BVh#dZddlmZddlmZddlmZddl m Z ddl m Z dZ Gdd ZGd d Ze d g  ddZy)z(Device function for replicated training.) node_def_pb2)device) tf_logging) server_lib) tf_export)Variable VariableV2AutoReloadVariableMutableHashTableMutableHashTableV2MutableHashTableOfTensorsMutableHashTableOfTensorsV2MutableDenseHashTableMutableDenseHashTableV2 VarHandleOp$BoostedTreesEnsembleResourceHandleOp*BoostedTreesQuantileStreamResourceHandleOpResourceConditionalAccumulatorDecisionTreeResourceceZdZdZdZdZy)_RoundRobinStrategyzReturns the next ps task index for placement in round-robin order. This class is not to be used directly by users. See instead `replica_device_setter()` below. c ||_d|_y)zgCreate a new `_RoundRobinStrategy`. Args: num_tasks: Number of ps tasks to cycle among. rN) _num_tasks _next_task)self num_taskss X/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/training/device_setter.py__init__z_RoundRobinStrategy.__init__'s  DODOc`|j}|jdz|jz|_|S)aChoose a ps task index for the given `Operation`. Args: unused_op: An `Operation` to be placed on ps. Returns: The next ps task index to use for the `Operation`. Returns the next index, in the range `[offset, offset + num_tasks)`. )rr)r unused_optasks r__call__z_RoundRobinStrategy.__call__0s+ ??D*doo=DO KrN)__name__ __module__ __qualname____doc__rr$rrrr s  rrceZdZdZdZdZy)_ReplicaDeviceChooserzClass to choose devices for Ops in a replicated training setup. This class is not to be used directly by users. See instead `replica_device_setter()` below. cX||_||_||_||_||_||_y)a3Create a new `_ReplicaDeviceChooser`. Args: ps_tasks: Number of tasks in the `ps` job. ps_device: String. Name of the `ps` job. worker_device: String. Name of the `worker` job. merge_devices: Boolean. Set to True to allow merging of device specs. ps_ops: List of strings representing `Operation` types that need to be placed on `ps` devices. ps_strategy: A callable invoked for every ps `Operation` (i.e. matched by `ps_ops`), that takes the `Operation` and returns the ps task index to use. N) _ps_tasks _ps_device_worker_device_merge_devices_ps_ops _ps_strategy)rps_tasks ps_device worker_device merge_devicesps_ops ps_strategys rrz_ReplicaDeviceChooser.__init__Fs1DNDO'D'DDL#Drc|js|jr |jStjj |jxsd}t |t jr|n |j}|jr|jr|j|jvrtjj |j}|j|j}}|r(|r||k(r!|j|j|}|j!|}|j#Stjj |j$xsd}|j!|}|j#S)z~Choose a device for `op`. Args: op: an `Operation`. Returns: The device to use for the `Operation`. )r#)r0rpydev DeviceSpec from_string isinstancerNodeDefnode_defr-r.opr1jobreplacer2make_merged_spec to_stringr/)rrAcurrent_devicer@r4 current_jobps_jobr5s rdevice_functionz%_ReplicaDeviceChooser.device_function\s)   299 YY%%11"))/rBN  L$8$89rr{{H ~~$//hkkT\\.I""..t?i*.. 6k  v(=%%4+<+BM  " " $$rN)r%r&r'r(rrIr)rrr+r+?s $,!%rr+ztrain.replica_device_setter)v1Nc|t|tjr|j}n#tj|j}tj j |j}||vs||yt||}|dk(ry|tt}|stjd| t|}t|s tdt!||||||} | j"S)a Return a `device function` to use when building a Graph for replicas. Device Functions are used in `with tf.device(device_function):` statement to automatically assign devices to `Operation` objects as they are constructed, Device constraints are added from the inner-most context first, working outwards. The merging behavior adds constraints to fields that are yet unset by a more inner context. Currently the fields are (job, task, cpu/gpu). If `cluster` is `None`, and `ps_tasks` is 0, the returned function is a no-op. Otherwise, the value of `ps_tasks` is derived from `cluster`. By default, only Variable ops are placed on ps tasks, and the placement strategy is round-robin over all ps tasks. A custom `ps_strategy` may be used to do more intelligent placement, such as `tf.contrib.training.GreedyLoadBalancingStrategy`. For example, ```python # To build a cluster with two ps jobs on hosts ps0 and ps1, and 3 worker # jobs on hosts worker0, worker1 and worker2. cluster_spec = { "ps": ["ps0:2222", "ps1:2222"], "worker": ["worker0:2222", "worker1:2222", "worker2:2222"]} with tf.compat.v1.device(tf.compat.v1.train.replica_device_setter(cluster=cluster_spec)): # Build your graph v1 = tf.Variable(...) # assigned to /job:ps/task:0 v2 = tf.Variable(...) # assigned to /job:ps/task:1 v3 = tf.Variable(...) # assigned to /job:ps/task:0 # Run compute ``` Args: ps_tasks: Number of tasks in the `ps` job. Ignored if `cluster` is provided. ps_device: String. Device of the `ps` job. If empty no `ps` job is used. Defaults to `ps`. worker_device: String. Device of the `worker` job. If empty no `worker` job is used. merge_devices: `Boolean`. If `True`, merges or only sets a device if the device constraint is completely unset. merges device specification rather than overriding them. cluster: `ClusterDef` proto or `ClusterSpec`. ps_ops: List of strings representing `Operation` types that need to be placed on `ps` devices. If `None`, defaults to `STANDARD_PS_OPS`. ps_strategy: A callable invoked for every ps `Operation` (i.e. matched by `ps_ops`), that takes the `Operation` and returns the ps task index to use. If `None`, defaults to a round-robin strategy across all `ps` devices. Returns: A function to pass to `tf.device()`. Raises: TypeError if `cluster` is not a dictionary or `ClusterDef` protocol buffer, or if `ps_strategy` is provided but not a callable. NrzQDEPRECATION: It is recommended to set merge_devices=true in replica_device_setterzps_strategy must be callable)r>r ClusterSpecas_dictr;r<r=rBlenlistSTANDARD_PS_OPSloggingwarningrcallable TypeErrorr+rI) r3r4r5r6clusterr7r8 cluster_spec ps_job_namechoosers rreplica_device_setterrYsD ':112__&l++G4<<>l""..y9==K,&,{*C*K < ,-H ]  ^/ "F  OO !%h/K +  2 33 !(I}"/ F'   r)rz/job:psz /job:workerTNNN)r(tensorflow.core.frameworkrtensorflow.python.frameworkrr;tensorflow.python.platformrrQtensorflow.python.trainingr tensorflow.python.util.tf_exportrrPrr+rYr)rrr_si/27<16+>>%>%B ,-.#$$-(5(,"&!%&* ^!/^!r