Vhz.ddlmZddlmZddlmZddlmZmZm Z m Z m Z ddl Z e rddl mZgdZdZd Zd Zd Zd Zd ZdZdZdZdZdZdZdZdZdZdZdZ dZ!dZ"dZ#GddeZ$eGddZ%eGd d!Z&Gd"d#Z'Gd$d%Z(y)&) annotations) dataclass)Enum)AnyCallableOptional TYPE_CHECKINGUnionN)Pattern) BackendConfigBackendPatternConfig DTypeConfigDTypeWithConstraintsObservationType input_dtype output_dtype weight_dtype bias_dtype is_dynamicnameconfigspatternpattern_complex_formatobservation_type dtype_configs root_module qat_module#reference_quantized_module_for_root fused_module fuser_methodroot_node_getterextra_inputs_getter#num_tensor_args_to_observation_typeinput_type_to_indexc eZdZdZdZ dZ dZy)rzfAn enum that represents different ways of how an operator/operator pattern should be observed rN)__name__ __module__ __qualname____doc__&OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT OUTPUT_SHARE_OBSERVER_WITH_INPUTINPUT_OUTPUT_NOT_OBSERVEDc/home/dcms/DCMS/lib/python3.12/site-packages/torch/ao/quantization/backend_config/backend_config.pyrr5s../* ()$ !"r0rcteZdZUdZdZded<dZded<dZded<dZded<dZ ded <dZ d ed <dZ d ed <y)ra Config for specifying additional constraints for a given dtype, such as quantization value ranges, scale value ranges, and fixed quantization params, to be used in :class:`~torch.ao.quantization.backend_config.DTypeConfig`. The constraints currently supported are: * `quant_min_lower_bound` and `quant_max_upper_bound`: Lower and upper bounds for the minimum and maximum quantized values respectively. If the QConfig's `quant_min` and `quant_max` fall outside this range, then the QConfig will be ignored. * `scale_min_lower_bound` and `scale_max_upper_bound`: Lower and upper bounds for the minimum and maximum scale values respectively. If the QConfig's minimum scale value (currently exposed as `eps`) falls below the lower bound, then the QConfig will be ignored. Note that the upper bound is currently not enforced. * `scale_exact_match` and `zero_point_exact_match`: Exact match requirements for scale and zero point, to be used for operators with fixed quantization parameters such as sigmoid and tanh. If the observer specified in the QConfig is neither `FixedQParamsObserver` nor `FixedQParamsFakeQuantize`, or if the quantization parameters don't match, then the QConfig will be ignored. NOptional[torch.dtype]dtypezUnion[int, float, None]quant_min_lower_boundquant_max_upper_boundscale_min_lower_boundscale_max_upper_boundzOptional[float]scale_exact_matchz Optional[int]zero_point_exact_match) r(r)r*r+r4__annotations__r5r6r7r8r9r:r/r0r1rrLsZ2$(E '5929592959295929)--,0M0r0rceZdZUdZded<ded<ded<ded<ded < d dd Zedd Zedd ZeddZ e ddZ ddZ y )raG Config object that specifies the supported data types passed as arguments to quantize ops in the reference model spec, for input and output activations, weights, and biases. For example, consider the following reference model: quant1 - [dequant1 - fp32_linear - quant2] - dequant2 The pattern in the square brackets refers to the reference pattern of statically quantized linear. Setting the input dtype as `torch.quint8` in the DTypeConfig means we pass in `torch.quint8` as the dtype argument to the first quantize op (quant1). Similarly, setting the output dtype as `torch.quint8` means we pass in `torch.quint8` as the dtype argument to the second quantize op (quant2). Note that the dtype here does not refer to the interface dtypes of the op. For example, the "input dtype" here is not the dtype of the input tensor passed to the quantized linear op. Though it can still be the same as the interface dtype, this is not always the case, e.g. the interface dtype is fp32 in dynamic quantization but the "input dtype" specified in the DTypeConfig would still be quint8. The semantics of dtypes here are the same as the semantics of the dtypes specified in the observers. These dtypes are matched against the ones specified in the user's QConfig. If there is a match, and the QConfig satisfies the constraints specified in the DTypeConfig (if any), then we will quantize the given pattern using this DTypeConfig. Otherwise, the QConfig is ignored and the pattern will not be quantized. Example usage:: >>> # xdoctest: +SKIP(failing) >>> dtype_config1 = DTypeConfig( ... input_dtype=torch.quint8, ... output_dtype=torch.quint8, ... weight_dtype=torch.qint8, ... bias_dtype=torch.float) >>> dtype_config2 = DTypeConfig( ... input_dtype=DTypeWithConstraints( ... dtype=torch.quint8, ... quant_min_lower_bound=0, ... quant_max_upper_bound=255, ... ), ... output_dtype=DTypeWithConstraints( ... dtype=torch.quint8, ... quant_min_lower_bound=0, ... quant_max_upper_bound=255, ... ), ... weight_dtype=DTypeWithConstraints( ... dtype=torch.qint8, ... quant_min_lower_bound=-128, ... quant_max_upper_bound=127, ... ), ... bias_dtype=torch.float) >>> dtype_config1.input_dtype torch.quint8 >>> dtype_config2.input_dtype torch.quint8 >>> dtype_config2.input_dtype_with_constraints DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=0, quant_max_upper_bound=255, scale_min_lower_bound=None, scale_max_upper_bound=None) rinput_dtype_with_constraintsoutput_dtype_with_constraintsweight_dtype_with_constraintsr3rOptional[bool]rNct|tr||_nt||_t|tr||_nt||_t|tr||_nt||_||_||_y)N)r4) isinstancerr=r>r?rr)selfrrrrrs r1__init__zDTypeConfig.__init__s| k#7 80;D -0D;0WD - l$8 91=D .1E"2D . l$8 91=D .1E"2D .%$r0c.|jjSN)r=r4rCs r1rzDTypeConfig.input_dtypes00666r0c.|jjSrF)r>r4rGs r1rzDTypeConfig.output_dtype11777r0c.|jjSrF)r?r4rGs r1rzDTypeConfig.weight_dtyperIr0c|jtd}|+t|tjt fs t d|jtd}|+t|tjt fs t d|jtd}|+t|tjt fs t d|jtd}|jtd}||||||S)at Create a ``DTypeConfig`` from a dictionary with the following items (all optional): "input_dtype": torch.dtype or ``DTypeWithConstraints`` "output_dtype": torch.dtype or ``DTypeWithConstraints`` "weight_dtype": torch.dtype or ``DTypeWithConstraints`` "bias_type": torch.dtype "is_dynamic": bool Nz@Expected input_dtype to be a torch.dtype or DTypeWithConstraintszAExpected output_dtype to be a torch.dtype or DTypeWithConstraintszAExpected weight_dtype to be a torch.dtype or DTypeWithConstraints) getINPUT_DTYPE_DICT_KEYrBtorchr4r ValueErrorOUTPUT_DTYPE_DICT_KEYWEIGHT_DTYPE_DICT_KEYBIAS_DTYPE_DICT_KEYIS_DYNAMIC_DICT_KEY)clsdtype_config_dictrrrrrs r1 from_dictzDTypeConfig.from_dicts(++,@$G  ": %++';<, R ),,-BDI  #J 5;;(<=- S ),,-BDI  #J 5;;(<=- S '**+>E &**+>E ; lJ SSr0c@i}|j|j|t<|j|j|t <|j |j|t<|j|j|t<|j|j|t<|S)z Convert this ``DTypeConfig`` to a dictionary with the items described in :func:`~torch.ao.quantization.backend_config.DTypeConfig.from_dict`. ) rr=rMrr>rPrr?rQrrRrrS)rCrUs r1to_dictzDTypeConfig.to_dict s -/    '6:6W6W 2 3    (22 %     (22 %  ?? &59__ 1 2 ?? &59__ 1 2  r0)NNNNN) r.Union[torch.dtype, DTypeWithConstraints, None]rrYrrYrr3rr@)returnr3)rUdict[str, Any]rZrrZr[) r(r)r*r+r;rDpropertyrrr classmethodrVrXr/r0r1rrpsCJ#76#77#77%%GKGKGK,0%) %C%E%E % * % # %<778888 T TD!r0rcleZdZdZd d dZdZd dZddZ ddZe ddZ e ddZ dd Z y )r ac Config that defines the set of patterns that can be quantized on a given backend, and how reference quantized models can be produced from these patterns. A pattern in this context refers to a module, a functional, an operator, or a directed acyclic graph of the above. Each pattern supported on the target backend can be individually configured through :class:`~torch.ao.quantization.backend_config.BackendPatternConfig` in terms of: (1) The supported input/output activation, weight, and bias data types (2) How observers and quant/dequant ops are inserted in order to construct the reference pattern, and (3) (Optionally) Fusion, QAT, and reference module mappings. The format of the patterns is described in: https://github.com/pytorch/pytorch/blob/master/torch/ao/quantization/backend_config/README.md Example usage:: import torch from torch.ao.quantization.backend_config import ( BackendConfig, BackendPatternConfig, DTypeConfig, ObservationType, ) weighted_int8_dtype_config = DTypeConfig( input_dtype=torch.quint8, output_dtype=torch.quint8, weight_dtype=torch.qint8, bias_dtype=torch.float) def fuse_conv2d_relu(is_qat, conv, relu): return torch.ao.nn.intrinsic.ConvReLU2d(conv, relu) # For quantizing Linear linear_config = BackendPatternConfig(torch.nn.Linear) .set_observation_type(ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT) .add_dtype_config(weighted_int8_dtype_config) .set_root_module(torch.nn.Linear) .set_qat_module(torch.ao.nn.qat.Linear) .set_reference_quantized_module(torch.ao.nn.quantized.reference.Linear) # For fusing Conv2d + ReLU into ConvReLU2d conv_relu_config = BackendPatternConfig((torch.nn.Conv2d, torch.nn.ReLU)) .set_observation_type(ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT) .add_dtype_config(weighted_int8_dtype_config) .set_fused_module(torch.ao.nn.intrinsic.ConvReLU2d) .set_fuser_method(fuse_conv2d_relu) # For quantizing ConvReLU2d fused_conv_relu_config = BackendPatternConfig(torch.ao.nn.intrinsic.ConvReLU2d) .set_observation_type(ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT) .add_dtype_config(weighted_int8_dtype_config) .set_root_module(torch.nn.Conv2d) .set_qat_module(torch.ao.nn.intrinsic.qat.ConvReLU2d) .set_reference_quantized_module(torch.ao.nn.quantized.reference.Conv2d) backend_config = BackendConfig("my_backend") .set_backend_pattern_config(linear_config) .set_backend_pattern_config(conv_relu_config) .set_backend_pattern_config(fused_conv_relu_config) c ||_i|_yrF)r!_pattern_complex_format_to_configrCrs r1rDzBackendConfig.__init__ds WY.r0c"d|jdS)NzBackendConfig())__dict__rGs r1__repr__zBackendConfig.__repr__ls a00r0c||_|S)z5 Set the name of the target backend. )rrbs r1set_namezBackendConfig.set_nameos  r0ctjjjjj |}||j |<|S)z Set the config for an pattern that can be run on the target backend. This overrides any existing config for the given pattern. )rNao quantizationbackend_configutils,_get_pattern_in_reversed_nested_tuple_formatra)rCconfigrs r1set_backend_pattern_configz(BackendConfig.set_backend_pattern_configvsH "'!6!6!E!E!K!K!x!x " JP../EF r0c6|D]}|j||S)z Set the configs for patterns that can be run on the target backend. This overrides any existing config for a given pattern if it was previously registered already. )rp)rCrconfs r1set_backend_pattern_configsz)BackendConfig.set_backend_pattern_configss& 2D  + +D 1 2 r0cHt|jjS)zS Return a copy of the list of configs set in this `BackendConfig`. )listravaluesrGs r1rzBackendConfig.configss D::AACDDr0cF||jtd}|jtgD]k}t|tr|j |%t|t r%|j tj|Ztdtd|S)z Create a ``BackendConfig`` from a dictionary with the following items: "name": the name of the target backend "configs": a list of dictionaries that each represents a `BackendPatternConfig` zExpected backend_config_dict['z'] to be a dictionary) rL NAME_DICT_KEYCONFIGS_DICT_KEYrBr rpdictrVrO)rTbackend_config_dictrrds r1rVzBackendConfig.from_dicts&**="=>$(()92> A!12//2At$//0D0N0Nq0QR 45E4FF[\   r0ct|jt|jDcgc]}|j c}iScc}w)z Convert this ``BackendConfig`` to a dictionary with the items described in :func:`~torch.ao.quantization.backend_config.BackendConfig.from_dict`. )ryrrzrrX)rCcs r1rXzBackendConfig.to_dicts4 499 DLLAqqyy{A  As? N)rx)rstr)rrrZr )ror rZr )rlist[BackendPatternConfig]rZr )rZr)r|r[rZr r\)r(r)r*r+rDrfrhrprsr]rr^rVrXr/r0r1r r !sa?BY1  1   EE * r0r ceZdZdZdddZdZddZ ddZddZ ddZ dd Z dd Z dd Z dd Z dd Zd dZ d!dZ d"dZ d#dZddZe d$dZd%dZy)&r z Config object that specifies quantization behavior for a given operator pattern. For a detailed example usage, see :class:`~torch.ao.quantization.backend_config.BackendConfig`. Nc||_tj|_g|_d|_d|_d|_d|_d|_ d|_ d|_ i|_ i|_ d|_yrF)rrr,rrrrreference_quantized_modulerr _root_node_getter_extra_inputs_getter$_num_tensor_args_to_observation_type_input_type_to_index_pattern_complex_formatrCrs r1rDzBackendPatternConfig.__init__sp*1 / V V02<@;?KO'=A046:8<!PR146!:>$r0c |jjDcic]D\}}t|ttfs|$t|ttfrt |dkDr||F}}}d|dScc}}w)NrzBackendPatternConfig(rd)reitemsrBrur{len)rCkv dict_nonemptys r1rfzBackendPatternConfig.__repr__sv ++- 1D$<0Q]q4,/CFQJ qD  '}oQ77 sA A0cB|j td||_|S)a& Set the pattern to configure. The pattern can be a float module, functional operator, pytorch operator, or a tuple combination of the above. Tuple patterns are treated as sequential patterns, and currently only tuples of 2 or 3 elements are supported. ._get_dtype_configs]#{+ #t$",,S1111G0H SWX[S\R]]^` r0N)rrrZr)PATTERN_DICT_KEYrOBSERVATION_TYPE_DICT_KEYrrLrrrROOT_MODULE_DICT_KEYrQAT_MODULE_DICT_KEYr#REFERENCE_QUANTIZED_MODULE_DICT_KEYrFUSED_MODULE_DICT_KEYrFUSER_METHOD_DICT_KEYrROOT_NODE_GETTER_DICT_KEYrEXTRA_INPUTS_GETTER_DICT_KEYr,NUM_TENSOR_ARGS_TO_OBSERVATION_TYPE_DICT_KEYrINPUT_TYPE_TO_INDEX_DICT_KEYPATTERN_COMPLEX_FORMAT_DICT_KEYr)rTbackend_pattern_config_dictrrrr}s r1rVzBackendPatternConfig.from_dictvs* u : :   89IJ K $(C C  % %+,EF -001GL 8A  ! !"3A"6 7 8  ' + +,@$ G  7;;    (AEARAR '(= >  ! ! -&& ()   $ $ 0)) (,  t88 9A =99 (<  t(( )A -)) (,   ' ' 3,, (/ +*I%NsF. rF)rzOptional[Pattern])rr rZr )rrrZr )rrrZr )rzlist[DTypeConfig]rZr )rtype[torch.nn.Module]rZr )rrrZr )rrrZr )rrrZr )r rrZr )r!rrZr )r"rrZr )r#zdict[int, ObservationType]rZr )r$zdict[str, int]rZr )rr[rZr r\)r(r)r*r+rDrfrrrrrrrrrrrrrrr^rVrXr/r0r1r r s ?" 8  / 4. 0 $ *?   1 $#+  3M  #1  I*8I IIV++r0r )) __future__r dataclassesrenumrtypingrrrr r rNtorch.ao.quantization.utilsr __all__rMrPrQrRrSryrzrrrrrrrrrrrrrrrrr r r/r0r1rs"!@@ 3 %&&"" ":.($"&K#&&.4/T,4 d.  1 1  1F m!m! m!`Q Q hx+x+r0