VhW5ddlmZddlmZddlZddlmcmZddlmZGddeZ GddejZ d ejd ejd e fd ZGd dejZGddejZGddejZGddejZGddejZGddejZGddejZGddejZGddejZGdd ejZGd!d"ejZGd#d$ejZGd%d&ejZy)')Any)BaseSparsifierN)nncheZdZdeeefddffd ZdejdedeeefddfdZ xZ S)ImplementedSparsifierkwargsreturnNc &t||y)N)defaults)super__init__)selfr __class__s V/home/dcms/DCMS/lib/python3.12/site-packages/torch/testing/_internal/common_pruning.pyr zImplementedSparsifier.__init__ s &)module tensor_namec d|jjdjd<|jd}|j dddz|d<y)Nrzlinear1.weight step_count)parametrizationsweightmaskstateget)rrrr linear_states r update_maskz!ImplementedSparsifier.update_mask sN45&&q)..q1zz"23 %1%5%5lA%F%J \"r) __name__ __module__ __qualname__dictstrrr rModuler __classcell__rs@rrr sR*c3h*D*K"))K#KcSVhK\`Krrc>eZdZdZedej ddfdZy)MockSparseLinearz This class is a MockSparseLinear class to check convert functionality. It is the same as a normal Linear layer, except with a different type, as well as an additional from_dense method. modr c@||j|j}|S)z ) in_features out_features)clsr(linears r from_densezMockSparseLinear.from_denses"S__%%' rN)rrr __doc__ classmethodrLinearr.rrr'r's. RYY+=rr' subset_tensorsuperset_tensorr cd}|D]@}|t|kr/tj|||s|dz }n1|t|kr/yy)zW Checks to see if all rows in subset tensor are present in the superset tensor rrFT)lentorchequal)r3r4irows rrows_are_subsetr;"s] A#o&&;;sOA$67Q #o&&  rc\eZdZdZdfd Zdej dej fdZxZS) SimpleLinearzModel with only Linear layers without biases, some wrapped in a Sequential, some following the Sequential. Used to test basic pruned Linear-Linear fusion.r c Rt|tjtjdddtjdddtjddd|_tjddd|_tjddd|_y)NFbias )r r r Sequentialr1seqlinear1linear2rrs rr zSimpleLinear.__init__6sx == IIa ' IIa ' IIa '  yyAE2 yyBU3 rxcl|j|}|j|}|j|}|SN)rGrHrIrrKs rforwardzSimpleLinear.forward@. HHQK LLO LLOrr N rrr r/r r7TensorrOr$r%s@rr=r=2s)V4%,,rr=c\eZdZdZdfd Zdej dej fdZxZS) LinearBiaszModel with only Linear layers, alternating layers with biases, wrapped in a Sequential. Used to test pruned Linear-Bias-Linear fusion.r c :t|tjtjdddtjdddtjdddtjdddtjddd|_y) Nr?r@TrArCFrE)r r rrFr1rGrJs rr zLinearBias.__init__Ksn == IIa & IIa ' IIa & IIa & IIa% (  rrKc(|j|}|SrM)rGrNs rrOzLinearBias.forwardUs HHQKrrQrRr%s@rrUrUGs)O %,,rrUc\eZdZdZdfd Zdej dej fdZxZS)LinearActivationzModel with only Linear layers, some with bias, some in a Sequential and some following. Activation functions modules in between each Linear in the Sequential, and each outside layer. Used to test pruned Linear(Bias)-Activation-Linear fusion.r c t|tjtjdddtj tjdddtj tjddd|_tjddd|_tj |_ tjdd d|_ tj |_ y) Nr?r@TrArCFrDrWrE) r r rrFr1ReLUTanhrGrHact1rIact2rJs rr zLinearActivation.__init___s == IIa & GGI IIa ' GGI IIa &  yyAD1 GGI yyBU3 GGI rrKc|j|}|j|}|j|}|j|}|j |}|SrM)rGrHr^rIr_rNs rrOzLinearActivation.forwardmH HHQK LLO IIaL LLO IIaLrrQrRr%s@rrZrZZs)B %,,rrZc\eZdZdZdfd Zdej dej fdZxZS)LinearActivationFunctionala,Model with only Linear layers, some with bias, some in a Sequential and some following. Activation functions modules in between each Linear in the Sequential, and functional activationals are called in between each outside layer. Used to test pruned Linear(Bias)-Activation-Linear fusion.r c  t|tjtjdddtj tjdddtj tjddd|_tjddd|_tjdd d|_tjd d d|_ tj |_ y) Nr?r@TrArCFrDrWrE) r r rrFr1r\rGrHrIlinear3r^rJs rr z#LinearActivationFunctional.__init__|s == IIa & GGI IIa ' GGI IIa &  yyAD1 yyAE2 yyBU3 GGI rrKc |j|}|j|}tj|}|j |}tj|}|j |}tj|}|SrM)rGrHFrelurIrfrNs rrOz"LinearActivationFunctional.forwardsb HHQK LLO FF1I LLO FF1I LLO FF1IrrQrRr%s@rrcrcvs)B %,,rrcc\eZdZdZdfd Zdej dej fdZxZS) SimpleConv2dzModel with only Conv2d layers, all without bias, some in a Sequential and some following. Used to test pruned Conv2d-Conv2d fusion.r c 4t|tjtjdddddtjddddd|_tjddddd|_tjddddd|_y) Nr rWFrA@04r r rrFConv2drGconv2d1conv2d2rJs rr zSimpleConv2d.__init__sx == IIaQ . IIb"a / yyRAE: yyRAE: rrKcl|j|}|j|}|j|}|SrMrGrsrtrNs rrOzSimpleConv2d.forwardrPrrQrRr%s@rrkrks(1;%,,rrkc\eZdZdZdfd Zdej dej fdZxZS) Conv2dBiaszModel with only Conv2d layers, some with bias, some in a Sequential and some outside. Used to test pruned Conv2d-Bias-Conv2d fusion.r c ft|tjtjdddddtjdddddtjddddd|_tjddddd|_tjdd ddd|_y NrrmrWTrArnFrorprqrJs rr zConv2dBias.__init__s == IIaQ - IIb"a . IIb"a /  yyRAD9 yyRAE: rrKcl|j|}|j|}|j|}|SrMrvrNs rrOzConv2dBias.forwardrPrrQrRr%s@rrxrxs(6;%,,rrxc\eZdZdZdfd Zdej dej fdZxZS)Conv2dActivationaModel with only Conv2d layers, some with bias, some in a Sequential and some following. Activation function modules in between each Sequential layer, functional activations called in-between each outside layer. Used to test pruned Conv2d-Bias-Activation-Conv2d fusion.r c t|tjtjdddddtj tjdddddtj tjdddddtj |_tjddddd|_tjdd ddd|_ yrz) r r rrFrrr\r]rGrsrtrJs rr zConv2dActivation.__init__s == IIaQ - GGI IIb"a . GGI IIb"a / GGI  yyRAE: yyRAD9 rrKc|j|}|j|}tj|}|j |}tj |}|SrM)rGrsrhrirthardtanhrNs rrOzConv2dActivation.forwardsH HHQK LLO FF1I LLO JJqMrrQrRr%s@rr}r}s)A :%,,rr}c\eZdZdZdfd Zdej dej fdZxZS) Conv2dPadBiasaQModel with only Conv2d layers, all with bias and some with padding > 0, some in a Sequential and some following. Activation function modules in between each layer. Used to test that bias is propagated correctly in the special case of pruned Conv2d-Bias-(Activation)Conv2d fusion, when the second Conv2d layer has padding > 0.r ct|tjtjddddddtj tjdddddtj tjddddddtj tjddddddtj tjdddddtj |_tjdd dddd|_tj |_ tjd d dddd|_ tj |_ y) NrrmrWT)paddingrBFrArnrorp) r r rrFrrr\r]rGrsr^rtr_rJs rr zConv2dPadBias.__init__s == IIaQ14 8 GGI IIb"a / GGI IIb"aAD 9 GGI IIb"aAD 9 GGI IIb"a . GGI  yyRAqtD GGI yyRAqtD GGI rrKc|j|}|j|}|j|}|j|}|j |}|SrM)rGrsr^rtr_rNs rrOzConv2dPadBias.forwardrarrQrRr%s@rrrs)c &%,,rrc\eZdZdZdfd Zdej dej fdZxZS) Conv2dPoolzModel with only Conv2d layers, all with bias, some in a Sequential and some following. Activation function modules in between each layer, Pool2d modules in between each layer. Used to test pruned Conv2d-Pool2d-Conv2d fusion.r c t|tjtjdddddtj dddtj tjdddddtjtjddd|_ tjdd ddd|_ tj ddd|_ tj |_ tjd d ddd|_ tjd d ddd|_y) NrrmrWT kernel_sizerrBrstriderrnrorp)r r rrFrr MaxPool2dr\r] AvgPool2drGrsmaxpoolaf1rtconv2d3rJs rr zConv2dPool.__init__s == IIaAD A LLQq! < GGI IIb"!QT B GGI LLQq! <  yyRQM ||!QG 779yyRQM yyRQM rrKc.|j|}|j|}|j|}|j|}|j |}t j |ddd}t j|}|j|}|SNrrr) rGrsrrrtrh avg_pool2drirrNs rrOzConv2dPool.forwardsv HHQK LLO LLO HHQK LLO LL!Q ? FF1I LLOrrQrRr%s@rrrs)8N  %,, rrc\eZdZdZdfd Zdej dej fdZxZS)Conv2dPoolFlattenFunctionalaModel with Conv2d layers, all with bias, some in a Sequential and some following, and then a Pool2d and a functional Flatten followed by a Linear layer. Activation functions and Pool2ds in between each layer also. Used to test pruned Conv2d-Pool2d-Flatten-Linear fusion.r c |t|tjtjdddddtj dddtj tjdddddtjtjddd|_ tjddddd|_ tj |_ tjdd ddd|_ tjd |_tjd d d |_y) NrrWTrrrr@r? )rr rA)r r rrFrrrr\r]rrGrsrrtAdaptiveAvgPool2davg_poolr1fcrJs rr z$Conv2dPoolFlattenFunctional.__init__#s == IIa14 @ LLQq! < GGI IIa14 @ GGI LLQq! <  yyA1adK 779yyBAqtL ,,V4 ))B.rrKc0|j|}|j|}tj|ddd}|j |}|j |}|j |}tj|d}|j|}|Sr) rGrsrh max_pool2drrtrr7flattenrrNs rrOz#Conv2dPoolFlattenFunctional.forward3sz HHQK LLO LL!Q ? HHQK LLO MM!  MM!Q  GGAJrrQrRr%s@rrrs)@ /  %,, rrc\eZdZdZdfd Zdej dej fdZxZS)Conv2dPoolFlattenaModel with Conv2d layers, all with bias, some in a Sequential and some following, and then a Pool2d and a Flatten module followed by a Linear layer. Activation functions and Pool2ds in between each layer also. Used to test pruned Conv2d-Pool2d-Flatten-Linear fusion.r c t|tjtjdddddtj dddtj tjdddddtjtjddd|_ tjddddd|_ tj |_ tjdd ddd|_ tjd |_tj|_tj"d d d |_y)NrrWTrrrr@r?r)rr,rrA)r r rrFrrrr\r]rrGrsrrtrrFlattenrr1rrJs rr zConv2dPoolFlatten.__init__Es == IIa14 @ LLQq! < GGI IIa14 @ GGI LLQq! <  yyA1adK 779yyBAqtL ,,V4 zz| ))B.rrKc&|j|}|j|}tj|ddd}|j |}|j |}|j |}|j|}|j|}|Sr) rGrsrhrrrtrrrrNs rrOzConv2dPoolFlatten.forwardVsw HHQK LLO LL!Q ? HHQK LLO MM!  LLO GGAJrrQrRr%s@rrr?s)@ /"  %,, rrc eZdZdZdededededdf fd Zd ejdeejejffd Z xZ S) LSTMLinearModelzCContainer module with an encoder, a recurrent module, and a linear. input_dim hidden_dim output_dim num_layersr Nct|tj||||_tj |||_yrM)r r rLSTMlstmr1r-rrrrrrs rr zLSTMLinearModel.__init__es6 GGIz:> ii J7 rinputcT|j|\}}|j|}||fSrM)rr-)rroutput_hiddendecodeds rrOzLSTMLinearModel.forwardls,))E*++f%r rrr r/intr r7rStuplerOr$r%s@rrrbs\M88*-8;>8LO8 8U\\eELL%,,4N.Orrc eZdZdZdededededdf fd Zd ejdeejejffd Z xZ S) LSTMLayerNormLinearModelz9Container module with an LSTM, a LayerNorm, and a linear.rrrrr Nct|tj||||_tj ||_tj|||_yrM) r r rrr LayerNormnormr1r-rs rr z!LSTMLayerNormLinearModel.__init__usF GGIz:> LL, ii J7 rrKcv|j|\}}|j|}|j|}||fSrM)rrr-)rrKrs rrOz LSTMLayerNormLinearModel.forward}s699Q<5 IIaL KKN%xrrr%s@rrrrs\C88*-8;>8LO8 8% ell0J*Krr)typingrtorch.ao.pruningrr7torch.nn.functionalr functionalrhrr1r'rSboolr;r#r=rUrZrcrkrxr}rrrrrrr2rrrs.+ KNK ryy  5<< %,, SW  299*&ryy8>299(*ryy8BIIDB"))D  F bii  ryyr