2Vh8PddlmZddlmZdZdZedGddeZy) ) keras_export) TFDataLayerg@g@zkeras.layers.MelSpectrogramceZdZdZ d fd ZdZdZdZdZdZ d dZ d Z fd Z xZ S) MelSpectrograma3 A preprocessing layer to convert raw audio signals to Mel spectrograms. This layer takes `float32`/`float64` single or batched audio signal as inputs and computes the Mel spectrogram using Short-Time Fourier Transform and Mel scaling. The input should be a 1D (unbatched) or 2D (batched) tensor representing audio signals. The output will be a 2D or 3D tensor representing Mel spectrograms. A spectrogram is an image-like representation that shows the frequency spectrum of a signal over time. It uses x-axis to represent time, y-axis to represent frequency, and each pixel to represent intensity. Mel spectrograms are a special type of spectrogram that use the mel scale, which approximates how humans perceive sound. They are commonly used in speech and music processing tasks like speech recognition, speaker identification, and music genre classification. References: - [Spectrogram](https://en.wikipedia.org/wiki/Spectrogram), - [Mel scale](https://en.wikipedia.org/wiki/Mel_scale). Examples: **Unbatched audio signal** >>> layer = keras.layers.MelSpectrogram(num_mel_bins=64, ... sampling_rate=8000, ... sequence_stride=256, ... fft_length=2048) >>> layer(keras.random.uniform(shape=(16000,))).shape (64, 63) **Batched audio signal** >>> layer = keras.layers.MelSpectrogram(num_mel_bins=80, ... sampling_rate=8000, ... sequence_stride=128, ... fft_length=2048) >>> layer(keras.random.uniform(shape=(2, 16000))).shape (2, 80, 125) Input shape: 1D (unbatched) or 2D (batched) tensor with shape:`(..., samples)`. Output shape: 2D (unbatched) or 3D (batched) tensor with shape:`(..., num_mel_bins, time)`. Args: fft_length: Integer, size of the FFT window. sequence_stride: Integer, number of samples between successive STFT columns. sequence_length: Integer, size of the window used for applying `window` to each audio frame. If `None`, defaults to `fft_length`. window: String, name of the window function to use. Available values are `"hann"` and `"hamming"`. If `window` is a tensor, it will be used directly as the window and its length must be `sequence_length`. If `window` is `None`, no windowing is used. Defaults to `"hann"`. sampling_rate: Integer, sample rate of the input signal. num_mel_bins: Integer, number of mel bins to generate. min_freq: Float, minimum frequency of the mel bins. max_freq: Float, maximum frequency of the mel bins. If `None`, defaults to `sampling_rate / 2`. power_to_db: If True, convert the power spectrogram to decibels. top_db: Float, minimum negative cut-off `max(10 * log10(S)) - top_db`. mag_exp: Float, exponent for the magnitude spectrogram. 1 for magnitude, 2 for power, etc. Default is 2. ref_power: Float, the power is scaled relative to it `10 * log10(S / ref_power)`. min_power: Float, minimum value for power and `ref_power`. c ||_||_|xs||_||_||_||_||_|xst|dz |_| |_ | |_ | |_ | |_ | |_ t|<di|y)N) fft_lengthsequence_stridesequence_lengthwindow sampling_rate num_mel_binsmin_freqintmax_freq power_to_dbtop_dbmag_exp min_power ref_powersuper__init__)selfr r r r rrrrrrrrrkwargs __class__s ^/home/dcms/DCMS/lib/python3.12/site-packages/keras/src/layers/preprocessing/mel_spectrogram.pyrzMelSpectrogram.__init__Ss"%..<* *(   :C (9$: &  "" "6"c|jdvrdn |j}|jj||}|j|}|j |}|j r|j |}|jjj|dd}|jj||j}|S)N)float32float64r )dtype) compute_dtypebackendconvert_to_tensor _spectrogram _melscaler_dbscalenumpyswapaxescast)rinputsr"outputss rcallzMelSpectrogram.callss!!)?? ##  //e/D##F+..)   mmG,G,,$$--gr2>,,##GT-?-?@rc0|jjj||j|j|j |j d\}}|jjj|jjj|jjj||jjj|}|jjj||j}|S)NT)r r r r center) r&mathstftr r r r r+sqrtaddsquarepowerr)rr.realimagspecs rr(zMelSpectrogram._spectrograms\\&&++  00 00;; , d||!!&& LL   " " ""))$/1C1C1J1J41P   ||!!''dll; rc|j|j|jj|d|j|j |j }|jjj||dS)Nr#)rnum_spectrogram_binsrlower_edge_hertzupper_edge_hertz)axes) linear_to_mel_weight_matrixrr&shaperrrr+ tensordot)rr.matrixs rr)zMelSpectrogram._melscalesr11**!%!3!3F!;B!?,,!]]!]] 2 ||!!++FF+CCrcd|jjj|jjj||jz}|jjj |jj |j}|d|jjj|jjj||jzz}|jjj||jjj||jz }|S)Ng$@) r&r+log10maximumrabsr'rmaxr)rr.log_spec ref_values rr*zMelSpectrogram._dbscales LL   $ $ ""**64>>B   LL&&** LL * *4>> :  D4<<--33 LL   & &y$.. A   <<%%-- dll((,,X6D rcnt|jjjd|tz zzS)aoConverts frequencies in `frequencies_hertz` in Hertz to the mel scale. Args: frequencies_hertz: A tensor of frequencies in Hertz. name: An optional name for the operation. Returns: A tensor of the same shape and type of `frequencies_hertz` containing frequencies in the mel scale. ?)_MEL_HIGH_FREQUENCY_Qr&r+log_MEL_BREAK_FREQUENCY_HERTZ)rfrequencies_hertzs r _hertz_to_melzMelSpectrogram._hertz_to_mels7%t||'9'9'='= $'AA B(   rc$jj||}jj||}jj||}jjd|}d}|dz } jjj || ||d} jjj j | d} jjjjjj j |j |dzdd} tfdjjj| dd D\} }}| | z || z z }|| z ||z z }jjj|jjj||}jjj||d gd d ggS) a Returns a matrix to warp linear scale spectrograms to the mel scale. Returns a weight matrix that can be used to re-weight a tensor containing `num_spectrogram_bins` linearly sampled frequency information from `[0, sampling_rate / 2]` into `num_mel_bins` frequency information from `[lower_edge_hertz, upper_edge_hertz]` on the mel scale. This function follows the [Hidden Markov Model Toolkit (HTK)]( http://htk.eng.cam.ac.uk/) convention, defining the mel scale in terms of a frequency in hertz according to the following formula: ```mel(f) = 2595 * log10( 1 + f/700)``` In the returned matrix, all the triangles (filterbanks) have a peak value of 1.0. For example, the returned matrix `A` can be used to right-multiply a spectrogram `S` of shape `[frames, num_spectrogram_bins]` of linear scale spectrum values (e.g. STFT magnitudes) to generate a "mel spectrogram" `M` of shape `[frames, num_mel_bins]`. ``` # `S` has shape [frames, num_spectrogram_bins] # `M` has shape [frames, num_mel_bins] M = keras.ops.matmul(S, A) ``` The matrix can be used with `keras.ops.tensordot` to convert an arbitrary rank `Tensor` of linear-scale spectral bins into the mel scale. ``` # S has shape [..., num_spectrogram_bins]. # M has shape [..., num_mel_bins]. M = keras.ops.tensordot(S, A, 1) ``` References: - [Mel scale (Wikipedia)](https://en.wikipedia.org/wiki/Mel_scale) Args: num_mel_bins: Python int. How many bands in the resulting mel spectrum. num_spectrogram_bins: An integer `Tensor`. How many bins there are in the source spectrogram data, which is understood to be `fft_size // 2 + 1`, i.e. the spectrogram only contains the nonredundant FFT bins. sampling_rate: An integer or float `Tensor`. Samples per second of the input signal used to create the spectrogram. Used to figure out the frequencies corresponding to each spectrogram bin, which dictates how they are mapped into the mel scale. lower_edge_hertz: Python float. Lower bound on the frequencies to be included in the mel spectrum. This corresponds to the lower edge of the lowest triangular band. upper_edge_hertz: Python float. The desired top edge of the highest frequency band. dtype: The `DType` of the result matrix. Must be a floating point type. Returns: A tensor of shape `[num_spectrogram_bins, num_mel_bins]`. gr@@Nr)r r c3nK|],}jjj|dg.yw)r@N)r&r+reshape).0trrs r z=MelSpectrogram.linear_to_mel_weight_matrix...s5;  LL   & &q1l*; <; s25)axisr)r&r-r'r+linspace expand_dimsrSr3extract_sequencestuplesplitrHminimumpad)rrr=rr>r?r"zero bands_to_zero nyquist_hertzlinear_frequenciesspectrogram_bins_melband_edges_mellower_edge_mel center_melupper_edge_mel lower_slopes upper_slopesmel_weights_matrixs`` rrBz*MelSpectrogram.linear_to_mel_weight_matrixsT ))-? <<99    <<99   ||--c59 %+ !\\//88 -!5 . $||11==   1 2A **<< LL   ' '""#34""#34q   = 6;; \\''--naa-H; 6 2 N-~=  ' ')== Z ' "\\//77 $,,$$,,\<H  ||!!%% Q !Q (  rct|dk(r:|j|d%|d|jzdz|jzg}|Sdg}|S|d|j|d"|d|jzdz|jzndg}|S)Nr@r)lenrr )r input_shape output_shapes rcompute_output_shapez#MelSpectrogram.compute_output_shapeGs { q !!#1~1!^d&:&::Q>++,L( L(A!!#1~1!^d&:&::Q>++, Lrcft|}|j|j|j|j |j |j|j|j|j|j|j|j|j|jd |S)N) r r r r rrrrrrrrr)r get_configupdater r r r rrrrrrrrr)rconfigrs rrvzMelSpectrogram.get_config_s#% "oo#'#7#7#'#7#7++!%!3!3 $ 1 1 MM MM#//++<<!^^!^^  " r) iiNhanni>g4@NTgT@rUg|=rN)i@g@_@g@r )__name__ __module__ __qualname____doc__rr0r(r)r*rSrBrtrv __classcell__)rs@rrr s}FT#@ $D" $ E N0rrN)keras.src.api_exportr,keras.src.layers.preprocessing.tf_data_layerrrQrOrr rrrs=-D#+,h[h-hr