AVh9dZddlZddlZddlmZddlmZddlmZddlm Z ddlm Z ddlm Z dd lm Z dd lm Z dd lmZdd lmZd ZdZGddZedgGddeZedgGddeZedgGddeZedgGddeZedgGdd eZGd!d"eZGd#d$eZGd%d&eZGd'd(eZGd)d*eZGd+d,eZeZ eZ!eZ"eZ#eZ$eZ%eZ&eZ'eZ(eZ)eZ*d4d-Z+d4d.Z,d4d/Z-d4d0Z.d1Z/Gd2d3Z0exZ1Z2exZ3Z4eZ5exZ6Z7exZ8Z9eZ:eZ;eZy)5zInitializers for TF 2.N) constant_op)dtypes) array_ops)gen_linalg_ops)linalg_ops_impl)math_ops) random_ops)stateless_random_ops) _compute_fans) tf_exportpartition_shapepartition_offsetc6eZdZdZddZdZedZddZy) Initializera[Initializer base class: all initializers inherit from this class. Initializers should implement a `__call__` method with the following signature: ```python def __call__(self, shape, dtype=None, **kwargs): # returns a tensor of shape `shape` and dtype `dtype` # containing values drawn from a distribution of your choice. ``` Nc t)aReturns a tensor object initialized as specified by the initializer. Args: shape: Shape of the tensor. dtype: Optional dtype of the tensor. If not provided will return tensor of `tf.float32`. **kwargs: Additional keyword arguments. Accepted values: `partition_shape` and `partition_offset`. Used when creating a single partition in a partitioned variable. `partition_shape` is the shape of the partition (i.e. the shape of the returned tensor) and `partition_offset` is a tuple of `int` specifying the offset of this partition w.r.t each axis. For example, a tensor of shape `(30, 100)` can be partitioned into two partitions: `p0` of shape `(10, 100)` and `p1` of shape `(20, 100)`; if the initializer is called with `partition_shape=(20, 100)` and `partition_offset=(10, 0)`, it should return the value for `p1`. )NotImplementedErrorselfshapedtypekwargss Q/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/ops/init_ops_v2.py__call__zInitializer.__call__0s $ ciS)zReturns the configuration of the initializer as a JSON-serializable dict. Returns: A JSON-serializable Python dict. rs r get_configzInitializer.get_configDs Irc6|jdd|di|S)a{Instantiates an initializer from a configuration dictionary. Example: ```python initializer = RandomUniform(-1, 1) config = initializer.get_config() initializer = RandomUniform.from_config(config) ``` Args: config: A Python dictionary. It will typically be the output of `get_config`. Returns: An Initializer instance. rNr)pop)clsconfigs r from_configzInitializer.from_configLs& JJw ==rc|D]X}|ttfvr$tdtttgd||r8t |j j dy)Nz"Keyword argument should be one of z . Received: z8 initializer doesn't support partition-related arguments)_PARTITION_SHAPE_PARTITION_OFFSET TypeErrorlist ValueError __class____name__)rrsupport_partitionkwargs r_validate_kwargszInitializer._validate_kwargsbs|+ '):; ; 0%'89:;<w PQ Q!~~&&'(* *+ + +rN)T) r+ __module__ __qualname____doc__rr classmethodr#r.rrrrr#s* (* +rrzeros_initializer)v1c.eZdZdZej fdZy)ZerosaiInitializer that generates tensors initialized to 0. Initializers allow you to pre-specify an initialization strategy, encoded in the Initializer object, without knowing the shape and dtype of the variable being initialized. Examples: >>> def make_variables(k, initializer): ... return (tf.Variable(initializer(shape=[k], dtype=tf.float32)), ... tf.Variable(initializer(shape=[k, k], dtype=tf.float32))) >>> v1, v2 = make_variables(3, tf.zeros_initializer()) >>> v1 >>> v2 >>> make_variables(4, tf.random_uniform_initializer(minval=-1., maxval=1.)) (, ##(', --6!%&e ??5% ((rNr+r0r1r2rfloat32rrrrr7r7ns.#)..)rr7ones_initializerc.eZdZdZej fdZy)OnesahInitializer that generates tensors initialized to 1. Initializers allow you to pre-specify an initialization strategy, encoded in the Initializer object, without knowing the shape and dtype of the variable being initialized. Examples: >>> def make_variables(k, initializer): ... return (tf.Variable(initializer(shape=[k], dtype=tf.float32)), ... tf.Variable(initializer(shape=[k, k], dtype=tf.float32))) >>> v1, v2 = make_variables(3, tf.ones_initializer()) >>> v1 >>> v2 >>> make_variables(4, tf.random_uniform_initializer(minval=-1., maxval=1.)) (, ##(', --6!%&e >>% ''rNr?rrrrCrCs.#)..(rrCconstant_initializerc&eZdZdZddZddZdZy)Constanta Initializer that generates tensors with constant values. Initializers allow you to pre-specify an initialization strategy, encoded in the Initializer object, without knowing the shape and dtype of the variable being initialized. `tf.constant_initializer` returns an object which when called returns a tensor populated with the `value` specified in the constructor. This `value` must be convertible to the requested `dtype`. The argument `value` can be a scalar constant value, or a list of values. Scalars broadcast to whichever shape is requested from the initializer. If `value` is a list, then the length of the list must be equal to the number of elements implied by the desired shape of the tensor. If the total number of elements in `value` is not equal to the number of elements required by the tensor shape, the initializer will raise a `TypeError`. Examples: >>> def make_variables(k, initializer): ... return (tf.Variable(initializer(shape=[k], dtype=tf.float32)), ... tf.Variable(initializer(shape=[k, k], dtype=tf.float32))) >>> v1, v2 = make_variables(3, tf.constant_initializer(2.)) >>> v1 >>> v2 >>> make_variables(4, tf.random_uniform_initializer(minval=-1., maxval=1.)) (, >> value = [0, 1, 2, 3, 4, 5, 6, 7] >>> init = tf.constant_initializer(value) >>> # Fitting shape >>> tf.Variable(init(shape=[2, 4], dtype=tf.float32)) >>> # Larger shape >>> tf.Variable(init(shape=[3, 4], dtype=tf.float32)) Traceback (most recent call last): ... TypeError: ...value has 8 elements, shape is (3, 4) with 12 elements... >>> # Smaller shape >>> tf.Variable(init(shape=[2, 3], dtype=tf.float32)) Traceback (most recent call last): ... TypeError: ...value has 8 elements, shape is (2, 3) with 6 elements... Args: value: A Python scalar, list or tuple of values, or a N-dimensional numpy array. All elements of the initialized variable will be set to the corresponding value in the `value` argument. support_partition: If true, the initizer supports passing partition offset and partition shape arguments to variable creators. This is particularly useful when initializing sharded variables where each variable shard is initialized to a slice of constant initializer. Raises: TypeError: If the input `value` is not one of the expected types. ctj|sGt|tttj fs"t dt|jd||_ ||_ y)Nz Invalid type for initial value: zD. Expected Python scalar, list or tuple of values, or numpy.ndarray.) npisscalar isinstancer(tuplendarrayr'typer+valuer,)rrPr,s r__init__zConstant.__init__sa KK *UT5"**4M"N  ,T%[-A-A,BCF F GGDJ.DrNc |j||j|tj|}t j |j ||S)aReturns a tensor object initialized as specified by the initializer. Args: shape: Shape of the tensor. dtype: Optional dtype of the tensor. If not provided the dtype of the tensor created will be the type of the inital value. **kwargs: Additional keyword arguments. Raises: TypeError: If the initializer cannot create a tensor of the requested dtype. r,)rr)r.r,rr;rconstantrPrs rrzConstant.__call__sJ &D4J4JK ooe$e    %u EErcd|jiS)NrP)rPrs rrzConstant.get_config*s TZZ  r)rFr/)r+r0r1r2rQrrrrrrHrHs@D/F$!rrHrandom_uniform_initializerc<eZdZdZddZej fdZdZy) RandomUniformaInitializer that generates tensors with a uniform distribution. Initializers allow you to pre-specify an initialization strategy, encoded in the Initializer object, without knowing the shape and dtype of the variable being initialized. Examples: >>> def make_variables(k, initializer): ... return (tf.Variable(initializer(shape=[k], dtype=tf.float32)), ... tf.Variable(initializer(shape=[k, k], dtype=tf.float32))) >>> v1, v2 = make_variables(3, tf.ones_initializer()) >>> v1 >>> v2 >>> make_variables(4, tf.random_uniform_initializer(minval=-1., maxval=1.)) (, 4 #)..E,rrXrandom_normal_initializerc<eZdZdZddZej fdZdZy) RandomNormalaInitializer that generates tensors with a normal distribution. Initializers allow you to pre-specify an initialization strategy, encoded in the Initializer object, without knowing the shape and dtype of the variable being initialized. Examples: >>> def make_variables(k, initializer): ... return (tf.Variable(initializer(shape=[k], dtype=tf.float32)), ... tf.Variable(initializer(shape=[k, k], dtype=tf.float32))) >>> v1, v2 = make_variables(3, ... tf.random_normal_initializer(mean=1., stddev=2.)) >>> v1 >>> v2 >> make_variables(4, tf.random_uniform_initializer(minval=-1., maxval=1.)) (, 4 #)..7&rric<eZdZdZddZej fdZdZy)TruncatedNormalasInitializer that generates a truncated normal distribution. Initializers allow you to pre-specify an initialization strategy, encoded in the Initializer object, without knowing the shape and dtype of the variable being initialized. These values are similar to values from a `tf.initializers.RandomNormal` except that values more than two standard deviations from the mean are discarded and re-drawn. This is the recommended initializer for neural network weights and filters. Examples: >>> def make_variables(k, initializer): ... return (tf.Variable(initializer(shape=[k], dtype=tf.float32)), ... tf.Variable(initializer(shape=[k, k], dtype=tf.float32))) >>> v1, v2 = make_variables( ... 3, tf.initializers.TruncatedNormal(mean=1., stddev=2.)) >>> v1 >>> v2 >> make_variables(4, tf.initializers.RandomUniform(minval=-1., maxval=1.)) (, >> def make_variables(k, initializer): ... return (tf.Variable(initializer(shape=[k], dtype=tf.float32)), ... tf.Variable(initializer(shape=[k, k], dtype=tf.float32))) >>> v1, v2 = make_variables(3, tf.initializers.VarianceScaling(scale=1.)) >>> v1 >>> v2 >> make_variables(4, tf.initializers.VarianceScaling(distribution='uniform')) (, fan_infan_avgfan_outzMArgument `mode` should be one of ('fan_in', 'fan_out', 'fan_avg'). Received: normalr>uniformruntruncated_normalzjArgument `distribution` should be one of ('uniform', 'truncated_normal', 'untruncated_normal'). Received: )r)lowerscalemode distributionr\r]r^)rrrrr\s rrQzVarianceScaling.__init__(s  { N" ## 33 004v7 88%%'Lx'l22 O&) **DJDI$DDI-d3Drc |j|t|}|j}t|\}}t|vr |t}|j dk(r|t d|z}n4|j dk(r|t d|z}n|t d||zdz z}|jdk(r6tj|dz }|jj|d||S|jdk(r3tj|}|jj|d||Stjd |z}|jj|| ||S) rrr?r@rg۶%?rzrg@)r.rsrr r%rmaxrmathsqrtr^rrtrb) rrrrrrrrmlimits rrzVarianceScaling.__call__BsE &!  &E JJE#E*OFG6!%&e yyH s2ve i  s2we s2(B.//e ..yy"44f  # # 4 4UC OO  2 2yyf  # # 1 1%fe LLiie $e  # # 2 255&% OOrc`|j|j|j|jdS)Nrrrr\rrs rrzVarianceScaling.get_configes* ))  r)rrrNrfrrrrrs0+\. 44#)..!PFrrc<eZdZdZddZej fdZdZy) OrthogonalaInitializer that generates an orthogonal matrix. Initializers allow you to pre-specify an initialization strategy, encoded in the Initializer object, without knowing the shape and dtype of the variable being initialized. If the shape of the tensor to initialize is two-dimensional, it is initialized with an orthogonal matrix obtained from the QR decomposition of a matrix of random numbers drawn from a normal distribution. If the matrix has fewer rows than columns then the output will have orthogonal rows. Otherwise, the output will have orthogonal columns. If the shape of the tensor to initialize is more than two-dimensional, a matrix of shape `(shape[0] * ... * shape[n - 2], shape[n - 1])` is initialized, where `n` is the length of the shape vector. The matrix is subsequently reshaped to give a tensor of the desired shape. Examples: >>> def make_variables(k, initializer): ... return (tf.Variable(initializer(shape=[k, k], dtype=tf.float32)), ... tf.Variable(initializer(shape=[k, k, k], dtype=tf.float32))) >>> v1, v2 = make_variables(3, tf.initializers.Orthogonal()) >>> v1 >> v2 >> make_variables(4, tf.initializers.Orthogonal(gain=0.5)) (>> def make_variable(k, initializer): ... return tf.Variable(initializer(shape=[k, k], dtype=tf.float32)) >>> make_variable(2, tf.initializers.Identity()) >>> make_variable(3, tf.initializers.Identity(gain=0.5)) Args: gain: Multiplicative factor to apply to the identity matrix. c||_yr/r)rrs rrQzIdentity.__init__s DIrc |j|dt|}t|dk7rtd|t j |d|i}|j |zS)aReturns a tensor object initialized as specified by the initializer. Args: shape: Shape of the tensor. dtype: Optional dtype of the tensor. Only floating point types are supported. **kwargs: Additional keyword arguments. Raises: ValueError: If the dtype is not floating point ValueError: If the requested shape does not have exactly two axes. FrSrrr)r.rsrr)reyer)rrrr initializers rrzIdentity.__call__so &E:  &E 5zQ L" ##"%%u:E:K 99{ ""rcd|jiS)Nrrrs rrzIdentity.get_config DII rN)rrfrrrrrs4#)..#,rrc*eZdZdZdfd ZdZxZS) GlorotUniformaThe Glorot uniform initializer, also called Xavier uniform initializer. Initializers allow you to pre-specify an initialization strategy, encoded in the Initializer object, without knowing the shape and dtype of the variable being initialized. Draws samples from a uniform distribution within [-limit, limit] where `limit` is `sqrt(6 / (fan_in + fan_out))` where `fan_in` is the number of input units in the weight tensor and `fan_out` is the number of output units in the weight tensor. Examples: >>> def make_variables(k, initializer): ... return (tf.Variable(initializer(shape=[k, k], dtype=tf.float32)), ... tf.Variable(initializer(shape=[k, k, k], dtype=tf.float32))) >>> v1, v2 = make_variables(3, tf.initializers.GlorotUniform()) >>> v1 >> v2 >> make_variables(4, tf.initializers.RandomNormal()) (>> def make_variables(k, initializer): ... return (tf.Variable(initializer(shape=[k, k], dtype=tf.float32)), ... tf.Variable(initializer(shape=[k, k, k], dtype=tf.float32))) >>> v1, v2 = make_variables(3, tf.initializers.GlorotNormal()) >>> v1 >> v2 >> make_variables(4, tf.initializers.RandomNormal()) (>> def make_variables(k, initializer): ... return (tf.Variable(initializer(shape=[k, k], dtype=tf.float32)), ... tf.Variable(initializer(shape=[k, k, k], dtype=tf.float32))) >>> v1, v2 = make_variables(3, tf.initializers.lecun_normal()) >>> v1 >> v2 >> make_variables(4, tf.initializers.RandomNormal()) (T KKrc tddd|S)aLeCun uniform initializer. Initializers allow you to pre-specify an initialization strategy, encoded in the Initializer object, without knowing the shape and dtype of the variable being initialized. Draws samples from a uniform distribution within [-limit, limit] where `limit` is `sqrt(3 / fan_in)` where `fan_in` is the number of input units in the weight tensor. Examples: >>> def make_variables(k, initializer): ... return (tf.Variable(initializer(shape=[k, k], dtype=tf.float32)), ... tf.Variable(initializer(shape=[k, k, k], dtype=tf.float32))) >>> v1, v2 = make_variables(3, tf.initializers.lecun_uniform()) >>> v1 >> v2 >> make_variables(4, tf.initializers.RandomNormal()) (>> def make_variables(k, initializer): ... return (tf.Variable(initializer(shape=[k, k], dtype=tf.float32)), ... tf.Variable(initializer(shape=[k, k, k], dtype=tf.float32))) >>> v1, v2 = make_variables(3, tf.initializers.he_normal()) >>> v1 >> v2 >> make_variables(4, tf.initializers.RandomNormal()) (T KKrc tddd|S)arHe uniform variance scaling initializer. Initializers allow you to pre-specify an initialization strategy, encoded in the Initializer object, without knowing the shape and dtype of the variable being initialized. Draws samples from a uniform distribution within [-limit, limit] where `limit` is `sqrt(6 / fan_in)` where `fan_in` is the number of input units in the weight tensor. Examples: >>> def make_variables(k, initializer): ... return (tf.Variable(initializer(shape=[k, k], dtype=tf.float32)), ... tf.Variable(initializer(shape=[k, k, k], dtype=tf.float32))) >>> v1, v2 = make_variables(3, tf.initializers.he_uniform()) >>> v1 >> v2 >> make_variables(4, tf.initializers.RandomNormal()) ( //% %    ""'+ ,, ,rcReZdZdZdfd Zddej fdZdZdZ xZ S) r]z5Random generator that selects appropriate random ops.cPtt| | |dg|_yd|_y)Nr)rr]rQr\rs rrQz_RandomGenerator.__init__0s) D*, )didirrzrc|jrtj}ntj}||||||jS)z0A deterministic random normal if seed is passed.rrlrmrr\)r\r stateless_random_normalr rtrrrlrmrops rrtz_RandomGenerator.random_normal8s= yy  7 7b  # #b $vU LLrc|jrtj}ntj}||||||jS)z1A deterministic random uniform if seed is passed.)rrZr[rr\)r\r stateless_random_uniformr rb)rrrZr[rrs rrbz_RandomGenerator.random_uniformAs= yy  8 8b  $ $b F6TYY PPrc|jrtj}ntj}||||||jS)z3A deterministic truncated normal if seed is passed.r)r\r stateless_truncated_normalr rrs rrz!_RandomGenerator.truncated_normalJs= yy  : :b  & &b $vU LLrr/) r+r0r1r2rQrr@rtrbrrrs@rr]r]-s*='*!6>>LPLrr]r/)?r2rnumpyrJtensorflow.python.frameworkrrtensorflow.python.opsrrrrr r tensorflow.python.ops.init_opsr tensorflow.python.util.tf_exportr r%r&rr7rCrHrXrir|rrrrrr4rArFrVrgtruncated_normal_initializervariance_scaling_initializerglorot_uniform_initializerglorot_normal_initializerorthogonal_initializeridentity_initializerrrrrrsr]zeror>onerErTrrbrrtridentity orthogonal glorot_normalglorot_uniformrrrrs 3.+01*,686$&H+H+V 2&+)K+)'+)\ "%+(;+(&+(\ !b)^!{^!*^!B 'B/AKA0AH &2.>;>/>BBkBJqkqhW2W2t5{5p+O+\+?+b*(..*(#*KZ(BV%KP%BV*$L$LRu d ((.%%"    r