2Vh TddlmZddlmZddlmZedGddeZy))ops) keras_export)Layerzkeras.layers.RMSNormalizationc@eZdZdZdfd ZdZdZdZfdZxZ S)RMSNormalizationarRoot Mean Square (RMS) Normalization layer. This layer normalizes the input tensor based on its RMS value. The Keras layer performs the operation as described in [Root Mean Square Layer Normalization](https://arxiv.org/pdf/1910.07467) by Biao Zhang et al. If `scale` is enabled, the layer will scale the normalized outputs via a learnable scaling factor. So, with scaling enabled, the normalization equations are as follows: Let the intermediate activations for a mini-batch to be the `inputs`. ```python rms_normalization(x) = x * rsqrt(mean(square(x))) * scale ``` For example: >>> layer = keras.layers.RMSNormalization() >>> layer.build([5, 20, 30, 10]) >>> print(layer.scale.shape) (10,) >>> layer(np.random.rand(1, 10)).numpy() array([[0.35098287, 1.0495652 , 1.4645109 , 1.2944688 , 0.31124955, 1.2768592 , 1.184331 , 0.17474432, 0.49955517, 1.2428929 ]], dtype=float32) Args: axis: int. The axis on which to perform the normalization. epsilon: float. A small number to add to avoid division by zero. c @t|di|||_||_y)N)super__init__axisepsilon)selfr r kwargs __class__s `/home/dcms/DCMS/lib/python3.12/site-packages/keras/src/layers/normalization/rms_normalization.pyr zRMSNormalization.__init__-s! "6"  ct|jtr't|jDcgc]}|| c}}n"||jf}|jg|_|j d|d|_d|_ycc}w)Nscaleones)nameshape initializerT) isinstancer listtuple add_weightrbuilt)r input_shapedimrs rbuildzRMSNormalization.build2sw dii &tyyA;s+ABE +-E DI__6%   Bs Bcptj||j|j|jS)aApplies RMS normalization to the input tensor. Args: x: Input tensor of shape (batch_size, input_dim). Returns: The RMS-normalized tensor of the same shape (batch_size, input_dim), scaled by the learned `scale` parameter. )rr r )rrms_normalizationrr r )rxs rcallzRMSNormalization.call?s,$$ TZZdii  rc t|jtr|jg}n |j}|D]=}|t|k\s|t| ks!t d|d|d|j|S)NzAxis z" is out of bounds for input shape z. Received: axis=)rr intlen ValueError)rraxesr s rcompute_output_shapez%RMSNormalization.compute_output_shapeMs dii %II;D99D Ds;''43{3C2C+C D6"##.-0&&*ii[2 rc^|j|jd}t| }i||S)N)r r )r r r get_config)rconfig base_configrs rr,zRMSNormalization.get_config\s7II|| g(* (+(((r)gư>) __name__ __module__ __qualname____doc__r r r$r*r, __classcell__)rs@rrrs'#J    ))rrN) keras.srcrkeras.src.api_exportrkeras.src.layers.layerrrr rrr8s2-(-.[)u[)/[)r