AVh dZddlmZddlmZddlmZddlmZddlmZddl m Z ddl m Z dd lmZed e j d Zed e j ddZy )zGImplementation of Neural Net (NN) functions with distribution strategy.)distribute_lib)ops) array_ops) check_ops)math_ops)util)dispatch) tf_exportznn.scale_regularization_lossctjrtjr tdtjj }t j||z S)aScales the sum of the given regularization losses by number of replicas. Usage with distribution strategy and custom training loop: ```python with strategy.scope(): def compute_loss(self, label, predictions): per_example_loss = tf.keras.losses.sparse_categorical_crossentropy( labels, predictions) # Compute loss that is scaled by sample_weight and by global batch size. loss = tf.nn.compute_average_loss( per_example_loss, sample_weight=sample_weight, global_batch_size=GLOBAL_BATCH_SIZE) # Add scaled regularization losses. loss += tf.nn.scale_regularization_loss(tf.nn.l2_loss(weights)) return loss ``` Args: regularization_loss: Regularization loss. Returns: Scalar loss value. z|You are calling `scale_regularization_loss` in cross replica context, while it was expected to be called in replica context.)r has_strategyin_cross_replica_context RuntimeError get_strategynum_replicas_in_syncr reduce_sum)regularization_loss num_replicass X/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/ops/nn_impl_distribute.pyscale_regularization_lossrs\>!!#  1 1 3  A   ,,.CC,   0 1L @@znn.compute_average_lossNc tj|}|j}tj|5|+tj|}tj ||}t j||}|ntjrtjr tdtjj}tj|d}||z}t!j"|dt!j$|dt!j&|dt j(|}t j||}t j*||cdddS#1swYyxYw)aScales per-example losses with sample_weights and computes their average. Usage with distribution strategy and custom training loop: ```python with strategy.scope(): def compute_loss(labels, predictions, sample_weight=None): # If you are using a `Loss` class instead, set reduction to `NONE` so that # we can do the reduction afterwards and divide by global batch size. per_example_loss = tf.keras.losses.sparse_categorical_crossentropy( labels, predictions) # Compute loss that is scaled by sample_weight and by global batch size. return tf.nn.compute_average_loss( per_example_loss, sample_weight=sample_weight, global_batch_size=GLOBAL_BATCH_SIZE) ``` Args: per_example_loss: Per-example loss. sample_weight: Optional weighting for each example. global_batch_size: Optional global batch size value. Defaults to (size of first dimension of `losses`) * (number of replicas). Returns: Scalar loss value, obtained by summing the `per_example_loss` and dividing by `global_batch_size`. If `global_batch_size` is zero, the result is zero. NzwYou are calling `compute_average_loss` in cross replica context, while it was expected to be called in replica context.rz!global_batch_size must be scalar.)messagez%global_batch_size must be an integer.z'global_batch_size must be non-negative.)rconvert_to_tensordtype losses_utilcheck_per_example_loss_rankscale_losses_by_sample_weightrcastrr r rrrrshape_v2rassert_scalar_v2assert_integer_v2assert_non_negative_v2r div_no_nan)per_example_loss sample_weightglobal_batch_size input_dtyperper_replica_batch_sizelosss rcompute_average_lossr*FsfF**+;< &&+../?@"8 ++M:m$BB M }}%5{C  % % '557 E  $002GGl(112BCAF0<? #F#J$$#L   / 0D &7E   t%6 7E"8"8"8s D9E::F)NN)__doc__tensorflow.python.distributertensorflow.python.frameworkrtensorflow.python.opsrrrtensorflow.python.ops.lossesrrtensorflow.python.utilr tensorflow.python.util.tf_exportr add_dispatch_supportrr*rrr4s{N7+++*<+6 )* &A+&AR $% <@F8&F8r