Vh6ddlZddlmZddlZddlmZejejjZ ejejjZ GddeZ dejdejfdZdZd Zd d Zd d Zd d Zy)N)Enumc$eZdZdZdZdZdefdZy) DQuantTypez Different quantization methods for auto_quantize API are identified here. auto_quantize API currently supports fp16 and bfp16 methods. )fp16bfp16returnc|jSN)value)selfs g/home/dcms/DCMS/lib/python3.12/site-packages/torch/distributed/algorithms/_quantization/quantization.py__str__zDQuantType.__str__s zzN)__name__ __module__ __qualname____doc__FP16BFP16strrrr rr s D Errtensorrc\tj|ttj Sr )torchclampTORCH_HALF_MINTORCH_HALF_MAXhalf)rs r _fp32_to_fp16_with_clamprs ;;v~~ > C C EErc6t|tjstdt ||t j k(r t|S|t jk(r)tjjj|Std|d)Nz;_quantize_tensor expecting torch.Tensor as input but found Quantization type  is not supported) isinstancerTensor RuntimeErrortyperrrrops quantization_FloatToBfloat16Quantized)rqtypes r _quantize_tensorr+s fell +I$v, X   '// *"" "yy%%??GG/w6GHIIrct|trtd|Dstdt ||Dcgc]}t ||}}|Scc}w)Nc3PK|]}t|tj ywr r#rr$.0ps r z(_quantize_tensor_list..- 4() 1ell#4$&zH_quantize_tensor_list expecting list of torch.Tensor as input but found )r#listallr%r&r+) tensor_listr*tquantized_tensor_lists r _quantize_tensor_listr:,sm k4 (4-841VW[\gWhVi j  BMMA-a7MM  NAc|t|tjstdt ||t j k(rx|jtjk7rtd|jd|jtjk(r||jS|j|z S|t jk(r_|jtjk7rtd|jdtjjj|Std|d)Nz=_dequantize_tensor expecting torch.Tensor as input but found ztensor dtype is z while expected to be FP16.r!r")r#rr$r%r&rrdtypefloat16floatrr'r(_Bfloat16QuantizedToFloat)rr* quant_losss r _dequantize_tensorrB7s  fell +KDQWL> Z    <<5== ("6<<.0KL \\U]] *z/A<<> !<<>J. . *"" " <<5== ("6<<.0KL 99))CCFK K/w6GHIIrct|trtd|Dstdt ||Dcgc]}t ||}}|Scc}w)Nc3PK|]}t|tj ywr r.r/s r r2z*_dequantize_tensor_list..Qr3r4zJ_dequantize_tensor_list expecting list of torch.Tensor as input but found )r#r5r6r%r&rB)r7r*rAr8dequantized_tensor_lists r _dequantize_tensor_listrFPsm k4 (4-841XY]^iYjXk l  FQQ1!U;QQ ""Rr;cJtjfd}|S)a Quantize the input tensors, choose the precision types, and pass other necessary arguments and then dequantizes the output. Currently it only supports: . FP16 and BFP16 quantization method supported for gloo and nccl backends . all_gather, all_to_all collective ops Note: BFP16 only supports 2D tensors. Args: func (Callable): A function representing collective operations. qtype (QuantType): Quantization method quant_loss (float, optional): This can be used to improve accuracy in the dequantization. Returns: (Callable): the same collective as func but enables automatic quantization/dequantization. cx|jdd}|jdd}|dur td tjk(r^|d}t |d }t | }tj||||t t| D] \}}|||< y tjk(r^|d}t |d }t | }tj||||t t| D] \}}|||< y tjk(r|d}|jd d} |jd d} t |d }t | }tj||| | | t t| D] \}}|||< ytd d)Ngroupasync_opFTz,The async_op=True mode is not supported yet.r)rIrJ)rA out_splits in_splits)rIzThe collective op z is not supported yet) getr%dist all_gatherr+r: enumeraterF all_to_allall_to_all_singlerB)argskwargsrIrJtensors input_tensors out_tensorsir8rLrMfuncr*rAs r wrapperzauto_quantize..wrapperks 7D)::j%0 t MN N 4?? "1gG,T!Weminrmaxrrr$rr+r:rBrFr^rrr res U]]+//U]]+//  FU\\FellF J!J2#;r