Vh ddlZddlmZddlmZddlmZddlmZ ddl m Z eee edffZd ed ed dfd Zd ed dfdZy#e $rdZ Y)wxYw)N)Union)_mesh_resources) DeviceMesh) Placement) is_compilingcy)NFr X/home/dcms/DCMS/lib/python3.12/site-packages/torch/distributed/tensor/parallel/_utils.pyis_torchdynamo_compilingr sr . func_name extra_msgreturnc\ts"tj|d|tdyy)z Inject common validation logics for `_prepare_input` funcs via this decorator. Include verifying that input needs to be either a :class:`Tensor` or :class:`DTensor` and only 1D :class:`DeviceMesh` is passed in. z) is deprecated and will be removed soon. ) stacklevelN)r warningswarn FutureWarning)r rs r _deprecate_warningsrs0 $ % kB9+ N   &r device_meshc|jdkDrtd|jdtj|}|r>||k7r8tj|}||jdz k7rt d|ddyyy)a Check whether TP mesh dimension is valid or not. Args: device_mesh (:class:`DeviceMesh`): The `device_mesh` where we perform Tensor Parallelism on. Return: `True` if the mesh dimension is valid, `False` otherwise. z8Tensor Parallel only accepts a 1D DeviceMesh, but found zMD!If you have a 2-D or N-D device_mesh, consider passing in device_mesh["tp"]zFound TP device_mesh on the z dimension of its parent mesh.zeCurrently we only support intranode TP and TP needs to be the innermost dimension on its parent mesh.N)ndim ValueErrorr get_root_meshget_root_mesh_dim RuntimeError)r root_meshtp_mesh_dim_in_roots r _validate_tp_mesh_dimr!%s!F{GWGWFXYZ Z   --k:IY+--?? L )..1"4 4./B.CCabw  5.yr )rtypingrtorch.distributed.device_meshrtorch.distributed.tensorr(torch.distributed.tensor.placement_typesrtorch._dynamo.external_utilsrr Exceptiontuple LayoutsTypestrrr!r r r r+s{9/>UIuY^445   3  3  4   3sAA A