BVh6dZddlmZddlmZddlmZddlmZddlmZddlm Z ddl m Z dd l m Z dd lmZd ZGd d e ZGdde ZGdde ZGdde ZdZGdde ZGdde Zy)z(Layers that act as activation functions.)dtypes)backend) constraints) initializers) regularizers)Layer) InputSpec)tf_utils)math_opsctSN)globalsc/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/keras/layers/advanced_activations.py get_globalsrs rcXeZdZdZdfd ZdZfdZejdZ xZ S) LeakyReLUaLeaky version of a Rectified Linear Unit. It allows a small gradient when the unit is not active: ``` f(x) = alpha * x if x < 0 f(x) = x if x >= 0 ``` Usage: >>> layer = tf.keras.layers.LeakyReLU() >>> output = layer([-3.0, -1.0, 0.0, 2.0]) >>> list(output.numpy()) [-0.9, -0.3, 0.0, 2.0] >>> layer = tf.keras.layers.LeakyReLU(alpha=0.1) >>> output = layer([-3.0, -1.0, 0.0, 2.0]) >>> list(output.numpy()) [-0.3, -0.1, 0.0, 2.0] Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the batch axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: alpha: Float >= 0. Negative slope coefficient. Default to 0.3. c tt| di||td|zd|_t j ||_y)NzKThe alpha value of a Leaky ReLU layer cannot be None, needs a float. Got %sTr)superr__init__ ValueErrorsupports_maskingrcast_to_floatxalphaselfrkwargs __class__s rrzLeakyReLU.__init__CsP )T#-f- }  "'( ))!D''.DJrcDtj||jS)N)rrrelurrinputss rcallzLeakyReLU.callLs <<djj 11rcdt|ji}tt|}t t |jt |jzSNr)floatrrr get_configdictlistitemsrconfig base_configrs rr)zLeakyReLU.get_configOsNuTZZ( )F 435K [&&()D,@@ AArc|Sr rr input_shapes rcompute_output_shapezLeakyReLU.compute_output_shapeT r)g333333? __name__ __module__ __qualname____doc__rr%r)r shape_type_conversionr3 __classcell__rs@rrr!s3B/2B  !!"rrceZdZdZ dfd Zej dZdZfdZ ej dZ xZ S)PReLUaParametric Rectified Linear Unit. It follows: ``` f(x) = alpha * x for x < 0 f(x) = x for x >= 0 ``` where `alpha` is a learned array with the same shape as x. Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: alpha_initializer: Initializer function for the weights. alpha_regularizer: Regularizer for the weights. alpha_constraint: Constraint for the weights. shared_axes: The axes along which to share learnable parameters for the activation function. For example, if the incoming feature maps are from a 2D convolution with output shape `(batch, height, width, channels)`, and you wish to share parameters across space so that each filter only has one set of parameters, set `shared_axes=[1, 2]`. c Htt| di|d|_t j ||_tj ||_tj ||_ |d|_ yt|ttfs |g|_ yt||_ yNTr)rr>rrrgetalpha_initializerralpha_regularizerralpha_constraint shared_axes isinstancer+tuple)rrBrCrDrErrs rrzPReLU.__init__{s  %)&) D)--.?@D)--.?@D'OO,<=Dd  dE] 3%dk*drct|dd}|j|jD] }d||dz < |j|d|j|j|j |_i}|jr1tdt|D]}||jvs||||<tt|||_ d|_ y)Nr)shapename initializer regularizer constraint)ndimaxesT) r+rE add_weightrBrCrDrrangelenr input_specbuilt)rr2 param_shapeirPs rbuildz PReLU.builds{12'K #! AE ****(( !*DJ D QK()#! D$$ $N$q'# S%5DADODJrc~tj|}|j tj| z}||zSr r!)rr$posnegs rr%z PReLU.calls5 ,,v C ::+ fW- -C 9rcptj|jtj|jt j|j |jd}tt|+}tt|jt|jzS)N)rBrCrDrE)r serializerBrrCrrDrErr>r)r*r+r,r-s rr)zPReLU.get_configs)33D4J4JK)33D4J4JK'11$2G2GH'' F t/1K [&&()D,@@ AArc|Sr rr1s rr3zPReLU.compute_output_shaper4r)zerosNNN) r6r7r8r9rr r:rXr%r)r3r;r<s@rr>r>Ys[D")!% $ +$ !!"( B !!"rr>cXeZdZdZdfd ZdZfdZejdZ xZ S)ELUaExponential Linear Unit. It follows: ``` f(x) = alpha * (exp(x) - 1.) for x < 0 f(x) = x for x >= 0 ``` Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: alpha: Scale for the negative factor. c tt| di||td|zd|_t j ||_y)Nz>Alpha of an ELU layer cannot be None, requires a float. Got %sTr)rrarrrrrrrs rrz ELU.__init__sN #t'' } 249: ;; D''.DJrcBtj||jSr )relurr#s rr%zELU.calls ;;vtzz **rcdt|ji}tt|}t t |jt |jzSr')r(rrrar)r*r+r,r-s rr)zELU.get_configsNuTZZ( )FT-/K [&&()D,@@ AArc|Sr rr1s rr3zELU.compute_output_shaper4r?r5r<s@rraras2*/+B  !!"rracXeZdZdZdfd ZdZfdZejdZ xZ S)ThresholdedReLUaThresholded Rectified Linear Unit. It follows: ``` f(x) = x for x > theta f(x) = 0 otherwise` ``` Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: theta: Float >= 0. Threshold location of activation. c tt| di||td|z|dkrtd|zd|_t j ||_y)NzJTheta of a Thresholded ReLU layer cannot be None, requires a float. Got %srzAThe theta value of a Thresholded ReLU layer should be >=0, got %sTr)rrjrrrrrtheta)rrlrrs rrzThresholdedReLU.__init__sr /4)3F3 } 8:?@ AA qy /167 88 D''.DJrctj|j|j}|tjtj|||jzSr )r castrldtypegreater)rr$rls rr%zThresholdedReLU.callsA MM$**fll 3E HMM("2"265"A6<<P PPrcdt|ji}tt|}t t |jt |jzS)Nrl)r(rlrrjr)r*r+r,r-s rr)zThresholdedReLU.get_configsNuTZZ( )F9;K [&&()D,@@ AArc|Sr rr1s rr3z$ThresholdedReLU.compute_output_shape r4rrgr5r<s@rrjrjs3* /QB  !!"rrjc^|tjk(rtjjSy)a Large negative number as Tensor. This function is necessary because the standard value for epsilon in this module (-1e9) cannot be represented using tf.float16 Args: tensor_type: a dtype to determine the type. Returns: a large negative number. ge)rfloat16min) tensor_types r_large_compatible_negativerws#FNN" >>   rcZeZdZdZdfd ZddZfdZejdZ xZ S)SoftmaxaSoftmax activation function. Example without mask: >>> inp = np.asarray([1., 2., 1.]) >>> layer = tf.keras.layers.Softmax() >>> layer(inp).numpy() array([0.21194157, 0.5761169 , 0.21194157], dtype=float32) >>> mask = np.asarray([True, False, True], dtype=bool) >>> layer(inp, mask).numpy() array([0.5, 0. , 0.5], dtype=float32) Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: axis: Integer, or list of Integers, axis along which the softmax normalization is applied. Call arguments: inputs: The inputs, or logits to the softmax layer. mask: A boolean mask of the same shape as `inputs`. Defaults to `None`. The mask specifies 1 to keep and 0 to mask. Returns: softmaxed output with the same shape as `inputs`. c Htt| di|d|_||_yr@)rryrraxis)rr{rrs rrzSoftmax.__init__Bs$ '4!+F+ DDIrc|>dtj||jz t|jz}||z }t |j t tfrtt|j dkDr8tj|tj||j dz Stj||j dStj||j S)NrhrIT)r{keepdimsr)r{) r rnrorwrFr{rGr+rSexpreduce_logsumexprsoftmax)rr$maskadders rr%z Softmax.callGs X]]466 $V\\ 24e  of$))eT]+ TYY! ||FX%>%> T&334 4vDIIaL99 ??6 22rcd|ji}tt|}t t |j t |j zS)Nr{)r{rryr)r*r+r,r-s rr)zSoftmax.get_configZsIdii F13K [&&()D,@@ AArc|Sr rr1s rr3zSoftmax.compute_output_shape_r4r)r r5r<s@rryry!s3@ 3&B  !!"rrycXeZdZdZdfd ZdZfdZejdZ xZ S)ReLUaYRectified Linear Unit activation function. With default values, it returns element-wise `max(x, 0)`. Otherwise, it follows: ``` f(x) = max_value if x >= max_value f(x) = x if threshold <= x < max_value f(x) = negative_slope * (x - threshold) otherwise ``` Usage: >>> layer = tf.keras.layers.ReLU() >>> output = layer([-3.0, -1.0, 0.0, 2.0]) >>> list(output.numpy()) [0.0, 0.0, 0.0, 2.0] >>> layer = tf.keras.layers.ReLU(max_value=1.0) >>> output = layer([-3.0, -1.0, 0.0, 2.0]) >>> list(output.numpy()) [0.0, 0.0, 0.0, 1.0] >>> layer = tf.keras.layers.ReLU(negative_slope=1.0) >>> output = layer([-3.0, -1.0, 0.0, 2.0]) >>> list(output.numpy()) [-3.0, -1.0, 0.0, 2.0] >>> layer = tf.keras.layers.ReLU(threshold=1.5) >>> output = layer([-3.0, -1.0, 1.0, 2.0]) >>> list(output.numpy()) [0.0, 0.0, 0.0, 2.0] Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the batch axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: max_value: Float >= 0. Maximum activation value. Default to None, which means unlimited. negative_slope: Float >= 0. Negative slope coefficient. Default to 0. threshold: Float >= 0. Threshold value for thresholded activation. Default to 0. c \tt| di|||dkrtd|z||dkrtd|z||dkrtd|zd|_|t j |}||_t j ||_t j ||_ y)Ngz=max_value of a ReLU layer cannot be a negative value. Got: %szBnegative_slope of a ReLU layer cannot be a negative value. Got: %sz=threshold of a ReLU layer cannot be a negative value. Got: %sTr) rrrrrrr max_valuenegative_slope threshold)rrrrrrs rrz ReLU.__init__s $((R (*34 55"!4 (*89 ::IN (*34 55!D((3iDN!00@D++I6DNrcptj||j|j|jS)N)rrr)rr"rrrr#s rr%z ReLU.calls. <<"11"&.."&.. 22rc|j|j|jd}tt|}t t|jt|jzS)N)rrr) rrrrrr)r*r+r,r-s rr)zReLU.get_configs[^^--^^F d.0K [&&()D,@@ AArc|Sr rr1s rr3zReLU.compute_output_shaper4r)Nrrr5r<s@rrrds3-^7&2B !!"rrN)r9tensorflow.python.frameworkrtensorflow.python.kerasrrrr)tensorflow.python.keras.engine.base_layerr)tensorflow.python.keras.engine.input_specr tensorflow.python.keras.utilsr tensorflow.python.opsr rrr>rarjrwryrrrrrs//+/00;?2*55pZEZz(%(V,e,^"@e@FV5Vr