2Vh:ddlZddlZddlmZddlmZddlmZddlmZddlmZddl m Z edGd d ejZ y) N)backend) initializers)layers)ops) keras_export)scipyzkeras.layers.STFTSpectrogramc`eZdZdZ d fd ZdZdZdZdZdZ fdZ xZ S) STFTSpectrogramaLayer to compute the Short-Time Fourier Transform (STFT) on a 1D signal. A layer that computes Spectrograms of the input signal to produce a spectrogram. This layers utilizes Short-Time Fourier Transform (STFT) by The layer computes Spectrograms based on STFT by utilizing convolution kernels, which allows parallelization on GPUs and trainable kernels for fine-tuning support. This layer allows different modes of output (e.g., log-scaled magnitude, phase, power spectral density, etc.) and provides flexibility in windowing, padding, and scaling options for the STFT calculation. Examples: Apply it as a non-trainable preprocessing layer on 3 audio tracks of 1 channel, 10 seconds and sampled at 16 kHz. >>> layer = keras.layers.STFTSpectrogram( ... mode='log', ... frame_length=256, ... frame_step=128, # 50% overlap ... fft_length=512, ... window="hann", ... padding="valid", ... trainable=False, # non-trainable, preprocessing only ... ) >>> layer(keras.random.uniform(shape=(3, 160000, 1))).shape (3, 1249, 257) Apply it as a trainable processing layer on 3 stereo audio tracks of 2 channels, 10 seconds and sampled at 16 kHz. This is initialized as the non-trainable layer, but then can be trained jointly within a model. >>> layer = keras.layers.STFTSpectrogram( ... mode='log', ... frame_length=256, ... frame_step=128, # 50% overlap ... fft_length=512, ... window="hamming", # hamming windowing function ... padding="same", # padding to preserve the time dimension ... trainable=True, # trainable, this is the default in keras ... ) >>> layer(keras.random.uniform(shape=(3, 160000, 2))).shape (3, 1250, 514) Similar to the last example, but add an extra dimension so the output is an image to be used with image models. We apply this here on a signal of 3 input channels to output an image tensor, hence is directly applicable with an image model. >>> layer = keras.layers.STFTSpectrogram( ... mode='log', ... frame_length=256, ... frame_step=128, ... fft_length=512, ... padding="same", ... expand_dims=True, # this adds the extra dimension ... ) >>> layer(keras.random.uniform(shape=(3, 160000, 3))).shape (3, 1250, 257, 3) Args: mode: String, the output type of the spectrogram. Can be one of `"log"`, `"magnitude`", `"psd"`, `"real`", `"imag`", `"angle`", `"stft`". Defaults to `"log`". frame_length: Integer, The length of each frame (window) for STFT in samples. Defaults to 256. frame_step: Integer, the step size (hop length) between consecutive frames. If not provided, defaults to half the frame_length. Defaults to `frame_length // 2`. fft_length: Integer, the size of frequency bins used in the Fast-Fourier Transform (FFT) to apply to each frame. Should be greater than or equal to `frame_length`. Recommended to be a power of two. Defaults to the smallest power of two that is greater than or equal to `frame_length`. window: (String or array_like), the windowing function to apply to each frame. Can be `"hann`" (default), `"hamming`", or a custom window provided as an array_like. periodic: Boolean, if True, the window function will be treated as periodic. Defaults to `False`. scaling: String, type of scaling applied to the window. Can be `"density`", `"spectrum`", or None. Default is `"density`". padding: String, padding strategy. Can be `"valid`" or `"same`". Defaults to `"valid"`. expand_dims: Boolean, if True, will expand the output into spectrograms into two dimensions to be compatible with image models. Defaults to `False`. data_format: String, either `"channels_last"` or `"channels_first"`. The ordering of the dimensions in the inputs. `"channels_last"` corresponds to inputs with shape `(batch, height, width, channels)` while `"channels_first"` corresponds to inputs with shape `(batch, channels, height, weight)`. Defaults to `"channels_last"`. Raises: ValueError: If an invalid value is provided for `"mode`", `"scaling`", `"padding`", or other input arguments. TypeError: If the input data type is not one of `"float16`", `"float32`", or `"float64`". Input shape: A 3D tensor of shape `(batch_size, time_length, input_channels)`, if `data_format=="channels_last"`, and of shape `(batch_size, input_channels, time_length)` if `data_format=="channels_first"`, where `time_length` is the length of the input signal, and `input_channels` is the number of input channels. The same kernels are applied to each channel independently. Output shape: If `data_format=="channels_first" and not expand_dims`, a 3D tensor: `(batch_size, input_channels * freq_channels, new_time_length)` If `data_format=="channels_last" and not expand_dims`, a 3D tensor: `(batch_size, new_time_length, input_channels * freq_channels)` If `data_format=="channels_first" and expand_dims`, a 4D tensor: `(batch_size, input_channels, new_time_length, freq_channels)` If `data_format=="channels_last" and expand_dims`, a 4D tensor: `(batch_size, new_time_length, freq_channels, input_channels)` where `new_time_length` depends on the padding, and `freq_channels` is the number of FFT bins `(fft_length // 2 + 1)`. c  v|||kDs|dkrtd|d||||krtd|d||!|| z|k7rtjd|gd} || vr tddj| d |||d vrtd ||d vrtd |t |t r t jj|dt |(di| ||_ ||_ ||_ |xs|jdz|_||_|xs4dt!t#j$t#j&|z|_||_||_||_||_| |_t5j6| |_t:j<j?d|_y)Nz_`frame_step` should be a positive integer not greater than `frame_length`. Received frame_step=z, frame_length=zI`fft_length` should be not less than `frame_length`. Received fft_length=zF`fft_length` is recommended to be a power of two. Received fft_length=)log magnitudepsdrealimaganglestftz*Output mode is invalid, it must be one of z, z. Received: mode=)densityspectrumzQScaling is invalid, it must be `None`, 'density' or 'spectrum'. Received scaling=)validsamezDPadding is invalid, it should be 'valid', 'same'. Received: padding=)ndim) ValueErrorwarningswarnjoin isinstancestrrsignal get_windowsuper__init__mode frame_length frame_step _frame_step fft_lengthintmathceillog2 _fft_lengthwindowperiodicscalingpadding expand_dimsrstandardize_data_format data_formatr input_spec InputSpec)selfr&r'r(r*r0r1r2r3r4r6kwargs all_modes __class__s _/home/dcms/DCMS/lib/python3.12/site-packages/keras/src/layers/preprocessing/stft_spectrogram.pyr%zSTFTSpectrogram.__init__s  !  %a77AlC ,~/   !j<&?''1l/,Q   !zZK'?J&N MM''1l4  Q y <99Y'((9$A   72I#I33:)=  + +%%,I/  fc " LL # #FA . "6" ($%?):):a)?$% TYYtyy678 8      &"::;G ++5515=c |jd|jdzdzf}|jdk7rM|jd|t j d|j |j|j|_ |jdk7rN|jd|t j d|j |j|j|_ yy)Nr rr real_kernelr)nameshape initializer imag_kernel) r'r/r& add_weightrSTFTr0r2r1r@rD)r9 input_shaperBs r=buildzSTFTSpectrogram.builds""At'7'71'ctj|\}}}}d}|jdk(r\|jrtj|gd}|Stj ||||z|g}tj|gd}|S|jrtj|gd}|Stj ||||z|g}|S)N channels_last)rrrr rrr )rr rr)rrBr6r4 transposereshape)r9outputs_channels freq_channelstime_seq batch_sizes r=_adjust_shapeszSTFTSpectrogram._adjust_shapess/2yy/A,8]H    .-->$++M!98D --;--> ++M!98Dr>c |jdk(rKtj|\}}}tj|gd}tj|d|dg}n2tj|\}}}tj|dd|g}tj |tj |tj|j|j|j|j}d}|jdk(rMtj|\}}}tj|gd}tj|||||g}|Stj|\}}}tj|||||g}|S)NrKrLrJr )r3stridesr6) r6rrBrMrNconvcastrstandardize_dtypedtyper3r)) r9inputskernelrPrSrQrOrTrRs r= _apply_convzSTFTSpectrogram._apply_convsX    .$'IIf$5 !Ax]]695F[["h):;F$'IIf$5 !Ax[["a):;F((  HHVW66v||D ELL$$((      .),7); &AxmmGY7GkkX}h?G *-7); &A}hkkX}h?Gr>c |j}tj|dvrtd|d}d}d}|jdk7r|j ||j }|jdk7r|j ||j}|jdk(r|j|S|jdk(r|j|S|jdk(r%|jtj||S|jdk(r(|jtj||gdStj|tj|z}|jd k(rC|j|tj|ddddd d ddfd d gd d gd d gd d ggzS|jtjtj|tj }|jd k(r|Stj"tj|tj S)N>float16float32float64zUInvalid input type. Expected `float16`, `float32` or `float64`. Received: input type=rrrrr)axisrr rJrr)r[rrZ TypeErrorr&r^r@rDrUrarctan2 concatenatesquarepadsqrtmaximumepsilonr )r9r\r[ real_signal imag_signalpower linear_stfts r=callzSTFTSpectrogram.call"s   $ $U +4  338';     99 **643C3CDK 99 **643C3CDK 99 &&{3 3 YY& &&{3 3 YY' !&&s{{; 'LM M YY& &&k :C JJ{+cjj.EEE 99 &&''!Q"a-(Aq6Aq6Aq6Aq6*J  )) HHS[[(9: ;  99 # 773;;{GOO4EFG Gr>c|jdk(r|d}n|d}|jdzdz}|jdk(r|dz}tjj |||z|j f|j|j|j}|jdk(r|\}}}n|\}}}|jr|jdk(r||||fS||||fS|S)NrKrJr rr)rWr3r6) r6r/r&roperation_utilscompute_conv_output_shaper'r)r3r4)r9rGrQrRrBrTrSrPs r=compute_output_shapez$STFTSpectrogram.compute_output_shapeTs    ."2H"1~H((A-1 99  Q M##==  H $    $$LL(( >     .&+ #J!&+ #J8   ?2"HmXFF"Hh FF r>c $t|}|j|j|j|j |j |j|j|j|j|j|jd |S)N) r&r'r(r*r0r1r2r3r6r4) r$ get_configupdater&r'r(r*r0r1r2r3r6r4)r9configr<s r=rvzSTFTSpectrogram.get_configost#%  $ 1 1"oo"oo++ MM<<<<#//#//   r>) r NNhannFrrFN) __name__ __module__ __qualname____doc__r%rHrUr^rprtrv __classcell__)r<s@r=r r sWvtM>^(2@0Hd6r>r ) r,r keras.srcrrrrkeras.src.api_exportrkeras.src.utils.module_utilsrLayerr rr>r=rsE "-.,-rfllr.rr>