Vh%UddlZddlZddlZddlZddlmZddlmZddlm Z ddl Z ddl m Z dgZejeZdaej&ZdefdZer"e j.j1sederddlZddlm cmZdd lmZdd l m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;mZ>m?Z?m@Z@mAZAd d lBmCZCmDZDmEZEd d lCd dlDmFZFd dlGmHZHd dlImJZJeeKeeLeLfddfeMd<egdz ZeeCjzeDjzZ ddZNdZOeFjdddddfdZQeCjdZSyy)N) Generator) timedelta)urlparse is_availablereturnc6ttjdS)N _rpc_init)hasattrtorch_CN/home/dcms/DCMS/lib/python3.12/site-packages/torch/distributed/rpc/__init__.pyrrs 588[ ))rz*Failed to initialize torch.distributed.rpc)Store)!_cleanup_python_rpc_handler_DEFAULT_INIT_METHOD_DEFAULT_NUM_WORKER_THREADS_DEFAULT_RPC_TIMEOUT_SEC)_delete_all_user_and_unforked_owner_rrefs_destroy_rref_context_disable_jit_rref_pickle'_disable_server_process_global_profiler_enable_jit_rref_pickle&_enable_server_process_global_profiler_get_current_rpc_agent_invoke_remote_builtin_invoke_remote_python_udf_invoke_remote_torchscript_invoke_rpc_builtin_invoke_rpc_python_udf_invoke_rpc_torchscript_is_current_rpc_agent_set_reset_current_rpc_agent_rref_context_get_debug_info_set_and_start_rpc_agent_set_profiler_node_id_set_rpc_timeout _TensorPipeRpcBackendOptionsBase_UNSET_RPC_TIMEOUTenable_gil_profilingget_rpc_timeoutPyRRefRemoteProfilerManagerRpcAgentRpcBackendOptionsTensorPipeAgent WorkerInfo)apibackend_registry functions)*) BackendType)TensorPipeRpcBackendOptions)_server_process_global_profilerendezvous_iterator)init_rpcr7r8c tjjd|%t|tj s t d|t|ts t d|r|pt D].}t|tt j|s,|}nt d||t jk7rtjdd|i|t j}|t j|}|stj||}n5tj|j ||at%t"\}}}|j't)|j* t,5tj.t1d t2|}t2d z adddt5j6|t9|t;||||||y#1swY:xYw) ak Initializes RPC primitives such as the local RPC agent and distributed autograd, which immediately makes the current process ready to send and receive RPCs. Args: name (str): a globally unique name of this node. (e.g., ``Trainer3``, ``ParameterServer2``, ``Master``, ``Worker1``) Name can only contain number, alphabet, underscore, colon, and/or dash, and must be shorter than 128 characters. backend (BackendType, optional): The type of RPC backend implementation. Supported values is ``BackendType.TENSORPIPE`` (the default). See :ref:`rpc-backends` for more information. rank (int): a globally unique id/rank of this node. world_size (int): The number of workers in the group. rpc_backend_options (RpcBackendOptions, optional): The options passed to the RpcAgent constructor. It must be an agent-specific subclass of :class:`~torch.distributed.rpc.RpcBackendOptions` and contains agent-specific initialization configurations. By default, for all agents, it sets the default timeout to 60 seconds and performs the rendezvous with an underlying process group initialized using ``init_method = "env://"``, meaning that environment variables ``MASTER_ADDR`` and ``MASTER_PORT`` need to be set properly. See :ref:`rpc-backends` for more information and find which options are available. ztorch.distributed.init_rpcNz0Argument backend must be a member of BackendTypezEArgument rpc_backend_options must be an instance of RpcBackendOptionsz$Could not infer backend for options zRPC was initialized with no explicit backend but with options corresponding to %(backend)s, hence that backend will be used instead of the default BackendType.TENSORPIPE. To silence this warning pass `backend=%(backend)s` explicitly.backend)rank world_size)seconds rpc_prefix_r2)r r _log_api_usage_once isinstancer4r7 TypeErrorr/typeconstruct_rpc_backend_options TENSORPIPEloggerwarningdist_create_store_from_options rendezvous init_methodr:next set_timeoutr rpc_timeout_init_counter_lock PrefixStorestr _init_counter dist_autograd_initr&_init_rpc_backend)namer>r?r@rpc_backend_optionscandidate_backendstore_s rr;r;TsF $$%AB  z %11( NO O  *: !24 W  ?2>%0 !'(FF-0G  :;N:OP+000E(  ?!,,G  &"2"P"P#  334GNE #'//#//dz# 23KE1a ),?,K,KLM  $$S;}o)F%GOE Q M  D!d#'5$jBUV  s :0G##G,c 8|tj|tj|t|t j |t j tdf|ti}|jD].\}}t||rtd|d|dt|y)Nz Argument z must be of type z but got type ) r4r7rKrrTnumbersIntegralrFr/itemsrD RuntimeError) r>r\rYr?r@rZ type_mappingargarg_types r_validate_rpc_argsrfs %11 4:: # '"" ))4:6 !2 *//1 MCc8,"u$5hZ~dSVi[Y rct||||||tr tdtj||||||}t j |y)NzRPC is already initialized)r\rYr?r@rZ)rfr"rbr4 init_backendr3_init_rpc_states)r>r\rYr?r@rZ rpc_agents rrXrXs] 7E4zCVW $ &;< <%11 ! 3   Y'rct}|jtjj |jt j |S)N)r$updater3rget_debug_inforV_get_debug_info)infos rrnrnsB+- C..0??AB M1134 r)Nr<NN)Tloggingos threadingwarningscollections.abcrdatetimer urllib.parserr torch.distributed distributedrK__all__ getLogger__name__rIrULockrRboolrr r rbr_torch.distributed.autogradautogradrVtorch._C._distributed_c10drtorch._C._distributed_rpcrrrrrrrrrrrrrrrr r!r"r#r$r%r&r'r(r)r*r+r,r-r.r/r0r1r3r4r5r7optionsr8server_process_global_profilerr9tupleint__annotations__r;rfrHrX_require_initializedrnr rrrss %!      8 $ #Y^^%*d*>%((,,. C DD>660"""""""""H32-4N"5S#94#EFF IIG #&6&>&>>G  uWn"&&   (2 kr