VhQHL)UddlZddlZddlZddlmZddlmZddlmZmZddl Z ddl m Z ddl Z ddl mZddlmZmZmZddlmZmZmZmZmZmZmZmZmZmZdd lmZm Z m!Z!m"Z"dd l#m$Z$m%Z%m&Z&m'Z'm(Z(dd l)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3dd l4m5Z5m6Z6d Z7ddZ8dZ9dZ:dZ;dZdZ?dZ@dddZAdZB ddZCddZD ddZEddZFddZGdd ZH dd!ZI dd"ZJ dd#ZKdd$ZLd%ZMd&ZNd'ZOd(ZP dd)ZQdd*ZRdd+ZSdd,ZT dd-ZU dd/ZV dd0ZWdd1ZXdd2ZYdd3ZZ dd4Z[d5Z\d6Z]ge0d7dd9e jje e je jd:e9e:d.d.d.e+ejd;d e0d?d@e jjdAe!d.d.e=eegdB e0dCdDdEe e je je je jdd.d.eOePF e0dGdHe!d.d.deQe-eegI e0dJdKe!d.d.deQe-eegI e0dLdMd8dNdOe"e je jeBdd.d.e+ejdPdQdRe je jfSe+ejd;de+ejd«d.ords>s{%dSD! 779>**U#C{ c8,f "66!9>!M "~c38K|]}dt|fyw))ordN)dict).0os r; z,sample_kwargs_vector_norm..Ls .!R!  .sr?)r:r9r<s`` r;sample_kwargs_vector_normrE<s  /tv ..r=Fc+Kt}t||||}d|jvgd}gd}fdfd} fd} fd} fd } | | | | f} rd nd }t|||d | D]$\}}}}}|||fz}t ||||i| &yw)Ndtypedevice requires_grad linalg.svdr?rrrOc|d}|jd}|ddd|f}r|dn|dj}|dd|ddf}|||fS)Nr0r.r/)shapemH)usvr'kUVh is_linalg_svds r; uniformizez%sample_inputs_svd..uniformizeYsa F GGBK F37O$SV#a&)) RaR ^!Rxr=c<|\}}}|jSNabs)rVrX_r[s r;fn_Uzsample_inputs_svd..fn_UasS/1auuwr=c|dS)Nr0r?)rVr[s r;fn_Szsample_inputs_svd..fn_Ses#q!!r=c@|\}}}||jfSr]r^)rVr`r'rYr[s r;fn_Vhz sample_inputs_svd..fn_Vhhs"c?1b"&&({r=c*|\}}}||z|fSr]r?)rVrXr'rYr[s r;fn_UVhz!sample_inputs_svd..fn_UVhms c?1b2vqyr= full_matricessomeTFr9output_process_fn_grad)rrnamerr()op_inforIrHrJr9 make_fullrankmake_argbatchesnsrarcrergfnsfullmatbatchnrW fullmat_valfnrTrZr[s @@r;sample_inputs_svdryOsHMU6H!GLL0MG B"  uf %C!.oFG(/RPS(T $q!["A e g{%;TV   sBB c+VKtt|||}t|tdf|tdfft|tdtf|tdtfft dt|d|tdfft dyw) NrGrOargsr0r1r|r9)r0rOrS)rrr(r'rArnrIrHrJr9rps r;sample_inputs_crossr|s5}H h1v&h1v.>-@ AA !Q8Q1I#6"8!  hv&h1v.>-@RT VVsB'B)c+DKtt|tj}t |t df|t dff}d}t ||tt |dt df|t dff}d}t ||tt |t d f|t d ff}d }t ||tt |t d f|t d fftd  }d }t ||tyw)NrIrHrOr0)inputr|z&inputs dimension -1 must have length 3 error_regex error_typerQz.inputs must have the same number of dimensionsr/zmust have length 3r})rr|r9zDimension out of range) rrtorchfloat32r(r'r! RuntimeErrorrA IndexError)rnrIr9rpsampleerrs r;error_inputs_crossrs{6GH xA/xA7G6I JF 2C V FF xAq 2(Aq6:J9L MF :C V FF xA/xA7G6I JF C V FF 1vh1v&6%8!F #C V DDsDD c+Ktt|||dd}t|ttf|tft|tdztf|tft|ddttf|ddtft|ddtdztf|ddtft|ddd|dddt|ttf|dddt|ttf|tdz fddt|ttdz f|tdz fddyw) a9 This function generates input for torch.linalg.householder_product (torch.orgqr). The first argument should be a square matrix or batch of square matrices, the second argument is a vector or batch of vectors. Empty, square, rectangular, batched square and batched rectangular input is generated. r/)rIrHrJlowhighr0rrN)rrrM)rrr(r'rs r;!sample_inputs_householder_productrsE #   H h1v&! 77 hAqz*HaTN ;; h1a|,h1ay.A BB h1a!eQ/0(Aq!92E FF T-4d+  h1v&4d(K LL h1v&!a%t$(O PP h1q5z*Ha!eX4d,S TTsE1E3c+Kt}tt|||}t||||}gd}|D]C\} } | | | fz} dD]} t|| | fdD]} t|| | fEyw)NrG))r0r?)r/rM)r/r/rPr{)rrS)rrrr() rnrIrHrJr9rorpmake_arg_fullrank test_sizes matrix_size batch_sizesr5rvs r;!sample_inputs_linalg_matrix_powerrsHM5}H U6J %/C [k;77 9AhtnA48 8 9 CA/6aTB B C CsA6A8c+Kt}t||||}gd}gd}|jdk(} t||D]\} } | | | fz} || } | r| j sp| j dkDr]t jj| j}| |jdjdz} | j|t| yw)NrGrL)rr0rQlogdetrrS) rrrmr is_complexr2rlinalgslogdetsign unsqueezerequires_grad_r()rnrIrHrJr9rorprqrr is_logdetrurvrTAss r;'sample_inputs_linalg_det_logdet_slogdetrsIMU6HG B (I "     A e  Q\\^ A  $$Q',,AAKKO--b11A  ] +!n sCCc +HKt}t|||}tt||}d}|jdk(} d} d} d} t | | | D]\} }}||| | fz}t j j|\}}||| |fz}|sdnd}t ||D]|\}}|r|s|s | rOt dd D]>\}}t||||||r|n |j|ft||  @^t|||||||f ~yw)z8Samples the inputs for both linalg.lu_solve and lu_solverHrIcH|j}|j||Sr])cloner)XrJYs r;rz%sample_inputs_lu_solve..clones GGI 'r=linalg.lu_solve)r?rMr)rOr0r)r.r0rFrjr/repeat)adjointleftr~r{N) rrrrmrrr lu_factorr(mTrA)rnrIrHrJr9make_fnmake_amake_bris_linalg_lu_solverqrrnrhsrvrurhsrLUpivotsBgradsLU_gradB_gradrrs r;sample_inputs_lu_solversPBG WE& 9F [f =F !)::G B D Wd3W 5# UaV^ %\\++A. F 5As8# $ -=&ue4 WOGVWV!%,]1%EMGT%b'*$eA144&HI#G$?"%6"2%G:Lf9UVV WWsD D"c+Kgdgdgdgdgdgdgdg}|D]H}g}t|dd|d dD]"}t|||| } |j| $t|Jyw) N)r0r/r0)r/rr/)rr/r/)r/r/r/r/)r/rOr.rQ)rQr.rr/)r/r.rOrQrOr/rSr0rG)ziprappendr() rnrIrHrJr9 test_casessizestensorsr5r:s r;sample_inputs_linalg_multi_dotrs J#cr E!"I. DE& A NN1    '""#sA&A(c+LKtjtjtjf}t t |||}d}||vrdt t ddf}nddt t ddddf}d } t||| d d gD]\} } } } t|| | | | f !yw) NrIrHrJ)r/r/)r/rOr/fror0rSnucr/r))rrS)rSrTFr{) rfloat16bfloat16 complex32rrrrr()rnrIrHrJr9low_precision_dtypesrprr<dimsr5r@r1keepdims r; sample_inputs_linalg_matrix_normr/s!MM5>>5??KF%}H E $$sSD!R(ucC4B26 D#*5$tUm#LDc3(4.S'/BCCDsB"B$)variantc+fK||dvrtd|tfdttfdtdfdtftttfdttftdtfdg }ddddd d td d d dt f }|tjtj tj hvrdtt dd f}ndddtt dd d d f }tt|||dd} |D]Dtdk(} td k(} | xsd dk7xrd dk7} dD] } |dk7r| rt| | | s| s%| r|n|}|D]}| r#d dk(r|tjk(s|Z|dkrU(| rRddtjdd dddd dd dtj dij|d}tfd|Dr||dk(r(ttj||||| t| || |dvst| || d Gyw)N)subgradient_at_zerozKUnsupported variant, expected variant to be 'subgradient_at_zero' but got: rMrr)rrrg?r0r/g @grSrg rr)rHrIrJrrFTr)r)rr0r0r?c3.K|] }|dk(yw)rNr?)rBr8 test_sizes r;rDz,sample_inputs_linalg_norm..sD9Q<1,DsrG)rr)r@rr1) ValueErrorr'rrrrrrrlenr(npr3anyzeros)rnrIrHrJrr9r vector_ords matrix_ordsrpis_vector_normis_matrix_normis_valid_for_p2rr<r@ dims_to_checkrs @r;sample_inputs_linalg_normrAsw.FFYZaYb c   A A A Aq Aq Aq  JCAsCr2tcTJK @@cC4B/ UE3aQC #   H 8 Y1,Y1,)WYr]a-?-VIbMUVDV$1 G//O!(9"5wGG"n"0;kD( !imq&8bff}S1W!#d64DF%c#rl"DmDD!33% %"'#)*7   '  &hy&93PPn,)$Y/S'vM( 1 8s H H1 H1c +&Ktt|||}d}d}t||D]g\}} || fz} t|| || ft t | D])} t|| || ft | +iyw)Nr)r?rMr)rQ)rr0rOrQr{r}r~)rrrr(rangerrA) rnrIrHrJr9rprqrrbrvrTis r;sample_inputs_linalg_vecdotrsF%}H%G B$1QD(5/%0BCCs5z" Ax&8!  sBBc+Kt}t||||}gd}ddg}t||D]\} } t|g| | | yw)a This function generates invertible inputs for linear algebra ops The input is generated as the itertools.product of 'batches' and 'ns'. In total this function generates 8 SampleInputs 'batches' cases include: () - single input, (0,) - zero batched dimension, (2,) - batch of two matrices, (1, 1) - 1x1 batch of matrices 'ns' gives 0x0 and 5x5 matrices. Zeros in dimensions are edge cases in the implementation and important to test for in order to avoid unexpected crashes. rGr?rMr)r0r0rQrNrrrr() rnrIrHrJr9rrprqrrrurvs r;sample_inputs_linalg_invertiblerscCGweF-XH&G QBGR(2q(0E010a0112sA A c+6Kfd}dD]r}td|gdD]^\}}|s|s t|||D]C} | jik(sJ||| j||| jd| _| E`tt|||Ed{y7w)zq This function produces inputs for matrix rank that test all possible combinations for atol and rtol cr|dk(ry|dk(ry|dk(sJtj|jddS)Nnonefloat?tensorrrI)ronesrT) kwarg_typeinprIs r; make_tol_argz/sample_inputs_matrix_rank..make_tol_argsB    X%%%zz#))CR.88r=)rrrr/ratolrtolN)rrr9r) rnrIrHrJr9rtol_type atol_type rtol_typers ` r;sample_inputs_matrix_rankrs 9($+VX,>q$I  Iyi9  }}***(FLLA(FLLA!    "/w}UUUsBBBBc+Kgd}gd}t|||D]\}}} ttd|| D]} tjg||| ||dj j j|} tjg|| | ||dj j j|} t| | fyw)aU This function produces factors `a` and `b` to generate inputs of the form `a @ b.t()` to test the backward method of `linalg_pinv`. That way we always preserve the rank of the input no matter the perturbations applied to it by the gradcheck. Note that `pinv` is Frechet-differentiable in a rank-preserving neighborhood. r)rrO2rOrr{N) rrminrrandqrQrr() rnrIrHrJr9rqr5rumrvrWars r;"sample_inputs_linalg_pinv_singularr s'G Dwd3, q!s1a|$ ,A DED1DaDeD>>-0   DED1DaDeD>>-0  aqd+ + ,,sCCc+Ktt|||}ttfdttfddttff}|D]}t||yw)NrGr/r0)rrr'r()rnrIrHrJr9rpshapesrTs r;sample_inputs_linalg_condr s]5}H A Aq Aq! F +(5/**+sAAc+XKtt|||}ddtfdtff}|D]}}t|dkDr|ddkDrt ||t|dkDr|dnd}t dD]/} || zdz } | dkrt ||t |  1yw) NrGr?rr/rrSr0rO)Nr9)rrr'rr(rrA) rnrIrHrJr9rprrTrvrrs r;sample_inputs_linalg_vanderrs5}H   A F A u:>eBi!mhuo. .UaE"IQq AAA A1uhuodQi@ @  A AsB(B*cF|jdk(r|tj}|jdk(rtj||d}|S||jd}tj|j |dj g|j|}|S)Nrr0T)r increasingrS)ndimrnewaxisvanderrTravelreshape)xrys r;np_vander_batchedr2svv{ bjjMvv{ IIa1 . 9 A IIaggi1 6 > >}}!} Mr=c+Kddlm}|tt||}|dtt||}||jz}||jz} t j dd||t j ddd|||| f} d| D} | D]} || _t| t| jjj|td| jjjjj|} t| td yw) Nrrandom_well_conditioned_matrixrr/c3^K|]%}tjj|d'yw)FupperN)rrcholeskyrBr s r;rDz8sample_inputs_linalg_cholesky_inverse..Us$HA%,,'''7H+-Fr"rT)$torch.testing._internal.common_utilsr r'rUrrrJr(detachrrrAr contiguous)rnrIrHrJr9r single_well_conditioned_matrixbatch_well_conditioned_matrices single_pdbatch_pdinputsrlus r;%sample_inputs_linalg_cholesky_inverser1@s6T&D 1E&&"'E 1auV'#/1O1R1RRI.1P1S1SSH  Aqf5 Aq!58 F IHJ  6'!n HHJ    - -m S   Aq!58PUAV  !++/U4F4HI4U V[[E15H &qf E$'   &q!5 H$'  6IsEEc +dKddlm}m}tj|}|t |||t d||tj dd||tj ddd||f}|jdk(r+|jr|t |||t d||fnd}d|D} d|D} tt||| } | D]} | \} }}|| _ d| jdd fD]}| g|| jd t }t| ||ftd | jj!j#|}t|||ftd | D]} | \} }}|| _ d| jdd fD]}| g|| jd t }t| ||ftd | jj!j#|}t|||ftd yw)Nrr3rr/r9r?c3^K|]%}tjj|d'yw)Fr6Nrr ldl_factor_exr%s r;rDz1sample_inputs_linalg_ldl_solve..s);< ""1"6r&c3^K|]%}tjj|d'yw)Tr6Nr?r%s r;rDz1sample_inputs_linalg_ldl_solve..s):; ""1"5r&rrrSFr6r~T)r'r4r5rrIr'rr:rrrrJrTr(rAr(rr)rnrIrHrJr9r4r5symmetric_inputshermitian_inputs test_cases1 test_cases2rp test_casefactorsrr` B_batch_shaper clone_factorss r;sample_inputs_linalg_ldl_solverJsT \\& !F"1E&A" qf   Aqf5 Aq!58  ;;% E$4$4 'qf E &q!5 H  @PK?OK F%}H!  & - '--"45 M?=?'--*;?Q?@AgVQKu@UV V#NN,224CCMRMVQKu8M    !  & - '--"45 M?=?'--*;?Q?@AgVQKt@TU U#NN,224CCMRMVQKt8L    sH.H0c +\Kddlm}tj|}|jdk(rd}nd}|jdk(s t rd}nd}t d ||D]K\}} } |d | zd fz} || ||d } | j|t| ||dd| } t| | | Myw)Nrrr8)gels)rLgelsygelssgelsdr9)rSrr0rMr?rN)rOrOrOr)rHrIrrrJdriver) r'r rrIr:r rrrr()rnrIrHrJr9r driversdeltasrurRdeltarTr rs r;sample_inputs_linalg_lstsqrVsS \\& !F{{f5{{e|~ '(:GV L /vuUA& *Ev N '  '  !Qv.. /sB*B,c+zKtjd|}tt||ftdywNr?rr{zat least 2 dimensions)rrrrandnr!r(rrnrIr9zero_ds r;error_inputs_lstsqr]s5 [[F +F F&++ s9;c+|Ktjd|}tt||dftdywrXrYr[s r; error_inputs_lstsq_grad_orientedr_s7 [[F +F F&$0+ s:<c+Ktt|||}ttfddf}tttff}itdtdtdf}tdddtdddtd ddf} t t ||t || D]\} } t || | yw) NrG)rOrQ)rQrOr/)offsetr0)radim1dim2rrr)rrr'rArrr() rnrIrHrJr9rp shapes_2d shapes_3d kwargs_2d kwargs_3drTkwargs r;!sample_inputs_diagonal_diag_embedris5}H Q(IQ IT^T^T^DI AAA& AAA& BQQ'I  9%wy)'D9 u(5/%889sCCc+`Ktt|tj}d}tt ff}tt t ff}i}tddtdtdf}|} t||} t||} t|| } t| | | D] \} }|| }t||}|jd }|jd }d |jvr|j}n2|jd vr| d vrp|jdz}n td| }|dz }t||dz}|xr||v}|xr||v}||k(rd|d|}t!||t|s|r(|r|n|}d|d|d|d}t!||t"tdyw)Nr)rr0rMrr0)rbrci')rb)rc)rr9rbrcdiagonal) diag_embedz_refs.diag_embed)rMrzshould be unreachablez(diagonal dimensions cannot be identical z, rz6Dimension out of range \(expected to be in range of \[z \], but got z\))rrrrr$r#r'rArrr(r3rmr1rrr!r)rnrIr9rpshapes1dshapes2dshapes3dkwargs1dkwargs2dkwargs3d samples1d samples2d samples3drTargrrbrcnum_dimbound1bound2 dim_range dim1_cond dim2_condrerr_dims r; error_inputs_diagonal_diag_embedr~s{6GH!HAyHAq |HH !! % % HH(+I(+I(+Iy)Y?"8 vuo3v6zz&!zz&!  %ggiG \\? ? $ggi!mG67 71&&1*- 2T2 2T2 4<*q% &q M5 Mf M'!5))*A A c+RKd}t||||}|D] }||_|yw)z< This function generates input for torch.linalg.eig c(|dt|dfSNrr0r^outputs r;out_fnz(sample_inputs_linalg_eig..out_fnpsay#fQi.((r=N)rrlrnrIrHrJr9rsamplesrs r;sample_inputs_linalg_eigrks9 ).gvumTG(.% s%'c+Kd}t||||}|D]+}dtjddgi|_||_|-yw)zm This function generates input for torch.linalg.eigh/eigvalsh with UPLO="U" or "L" keyword argument. cLt|tr|dt|dfS|Sr)r4tupler_rs r;rz)sample_inputs_linalg_eigh..out_fn~s* fe $!9c&)n, ,Mr=UPLOr#rXN)rrandomchoicer9rlrs r;sample_inputs_linalg_eighrysT .gvumTG Sz!:; (.% sAAc+Kt||||fi|D]h}|jr |jjjn|}ddt j d||fD]}t|}d|i|_|jyw)zd This function generates input for torch.linalg.pinv with hermitian=False keyword argument. Nrrr) rrrrealrHrrrr9)rnrIrHrJr9rC real_dtypers r;sample_inputs_linalg_pinvrs- 17,1+;+;QWW\\'' 3 S 6 RS DQA~AHG  sAc+NKt||||fi|D]}ddi|_|yw)zc This function generates input for torch.linalg.pinv with hermitian=True keyword argument. r7TN)rr9)rnrIrHrJr9rCs r;#sample_inputs_linalg_pinv_hermitianrs@ - 17 & s#%Tc+Kt}t||||}tt|||}gd} ddg} |rgd} nddg} t| | | D]-\} } }t |g| | | || | fz|zf/y w) a This function generates always solvable input for torch.linalg.solve We sample a fullrank square matrix (i.e. invertible) A The first input to torch.linalg.solve is generated as the itertools.product of 'batches' and 'ns'. The second input is generated as the product of 'batches', 'ns' and 'nrhs'. In total this function generates 18 SampleInputs 'batches' cases include: () - single input, (0,) - zero batched dimension, (2,) - batch of two matrices. 'ns' gives 0x0 and 5x5 matrices. and 'nrhs' controls the number of vectors to solve for: () - using 1 as the number of vectors implicitly (1,) - same as () but explicit (3,) - solve for 3 vectors. Zeros in dimensions are edge cases in the implementation and important to test for in order to avoid unexpected crashes. 'vector_rhs_allowed' controls whether to include nrhs = () to the list of SampleInputs. torch.solve / triangular_solve / cholesky_solve (opposed to torch.linalg.solve) do not allow 1D tensors (vectors) as the right-hand-side. Once torch.solve / triangular_solve / cholesky_solve and its testing are removed, 'vector_rhs_allowed' may be removed here as well. rG)r?rMrrrQr)r?rrNrrNr{N)rrrrr()rnrIrHrJvector_rhs_allowedr9rorrrqrrrrvrurs r;sample_inputs_linalg_solvers2IM U6F5}F'G QBd| Wd3T 5#&.%..A.fUaT\C=O6P5RSSTsA:A<c +Ktt||}d}d}d}t|||tddD]F\} } } \} } }| dk(r!| r || | fn || | f}|| | f}n#| r || | | fn || | | f}|| | | f}|r#|jd d d j d n)|jd d d }d ||j d k<| r|j n|j| | |d}|rgtddD]U\}}|s|s t|jj||jj|f|W6t||f|Iyw)Nr)r0r/r)rOr)r0rOrrjrOrr0rrrSrgư>)r#r unitriangularr/r~) rrrrkfill_r_triu_tril_r(rr)rnrIrHrJr9rpbsrrksrrvrWrr#unirrr8grad_Agrad_Bs r;%sample_inputs_linalg_solve_triangularrs{%?H B B B'. BGM!4(;#1a#$s 6$(!Q h1v.>A!Q A'+!Q#1a)1DA!Q#A  JJq"b ! ' ' , 1b"%A #Aaeegn   GGI GGI $E ")-"B f!GGI,,V4'')226:<!  aqd6: :?;sE=E?c+Kt||||d}d}|D]C}|jd|jfc|_|_|jdk(r||_|Eyw)aM This function generates always solvable input for legacy solve functions (the ones that are not in torch.linalg module). The difference from sample_inputs_linalg_solve is that here the right-hand-side of A x = b equation should have b.ndim >= 2, vectors are not allowed. Also the arguments order is swapped. F)rJrc |dS)Nrr?rs r;rz*sample_inputs_legacy_solve..out_fns ayr=rsolveN)rr|rrmrl)rnrIrHrJr9outrrs r;sample_inputs_legacy_solversk %mPU C$*KKNV\\O! fk <<7 ",2F ) sAAc+VKjdk(}|stnt}t||||}fd}d} t j |j dk(rdnd} d} t| | | D]9\} } }| t|ztfz}|s||n||}t|d | i| ;yw) Nlinalg.lu_factorrGc:jdk(r |d|dfS|S)N linalg.lur0r/)rm)rrns r;rz'sample_inputs_linalg_lu..out_fn"s' <<; &!9fQi' 'Mr=rPr8rj)T)rrSrr0r/pivotrk) rmrrrrrIr:rr'r()rnrIrHrJr9 full_rankrrpr batch_shapesrrT batch_shaperrUrTrs` r;sample_inputs_linalg_lurs 22I  A weF-XH &L#ll62776A]wF F%,\66%JU! UEq5y!n,#,HUO(E2B!We$4VTT UsB&B)c+Ktt|||}gd}gd}t|||D]\}} } t||| | fz ywNrGr)rQr/r)rrrr() rnrIrHrJr9rprqrrrur rvs r;sample_inputs_linalg_svdvalsr3s[5}H'G BwB/4 q!(5Aq6>2334rc+Kt}t||||}gd}gd}t|t||D]\} \} } | | | fz} t|| !ywrr) rnrIrHrJr9rorprqrrrur rvrTs r;sample_inputs_linalg_qr_geqrfr?snIMU6H'G B '"b/:, v1A(E*++,sAAc+Kddg}ddg}tt|||}tj||D](\}} ||} ||dd} t | | | *yw)N)r/rOr-)rOr.r.rO)rr/rGr/)r)rr itertoolsrr() rnrIrHrJr9a_shapesdimssrpa_shaperr rs r;sample_inputs_tensorsolverPsy<(H 6NE5}H#**8U;+  W  WRa[ !!QT**+sAA c+ Kt  fd}ddg}|D]U\}}|jg||j} | jt | t |Wyw)NcddS)N rr?)rIrHrprJsr; make_inputz+sample_inputs_tensorinv..make_inputdsBvU-XXr=))r/r/rO)rr0))r.rO)r-r0r/)ind)rrr(rr(r) rnrIrHrJr9rr shape_lhs shape_rhsrrps ``` @r;sample_inputs_tensorinvras{CHY F !'3 9"jl"":I: :AAC =)#3y>223sA.A4z linalg.crossrSc2tj|||S)N)axis)rcrossrrr1s r;rvs!QS!9r= linalg_crosszUnsupported on MPS for now TestCommontest_numpy_ref_mps) refopdtypes aten_namesample_inputs_funcerror_inputs_func supports_outsupports_fwgrad_bwgradsupports_forward_adskipsz linalg.det linalg_det)det) rraliasesrrrr decoratorscheck_batched_gradgradzlinalg.diagonallinalg_diagonaldiagonal_backward)raten_backward_namerrrrrrzlinalg.choleskylinalg_cholesky)rrrrcheck_batched_forward_gradrgradcheck_wrapperrzlinalg.cholesky_exlinalg_cholesky_exz linalg.vecdot linalg_vecdotr}cF|j|zj|Sr])conjsumrs r;rrsQVVX\$6$6s$;r=zSkipped!TestSchemaCheckModeOpInfotest_schema_correctness)rg~jt?g rh?rTestInductorOpInfotest_comprehensiver8) device_type)rrrrrrrrz linalg.cond linalg_condTestFakeTensortest_fake_crossref_backward_amp)rr active_if"test_fake_crossref_backward_no_amp) rrrrrrrgradcheck_nondet_tolrrz linalg.eig linalg_eigtest_outr9mps)rrtest_variant_consistency_eagerTestJittest_variant_consistency_jit) rrrrrcheck_batched_gradrrrrrzlinalg.eigvalslinalg_eigvals) rrrrrrrrrrrz linalg.eigh linalg_eigh) rrrrrrrrrrrzlinalg.eigvalshlinalg_eigvalshzlinalg.householder_productlinalg_householder_product)orgqrgMbP?zSkipped! FlakyTestFwdGradientstest_fn_fwgrad_bwgrad) rrrrgradcheck_fast_moderrrrrrrzlinalg.ldl_factorlinalg_ldl_factor)rrsupports_autogradrrzlinalg.ldl_factor_exlinalg_ldl_factor_exzlinalg.ldl_solvelinalg_ldl_solve) r.z not available before CUDA 11.3.1z linalg.lstsq linalg_lstsqTestBwdGradients)rrrrrrr grad_orientedcLtjj|||ddS)NrQr/)rrlstsq)r rrRs r;rrs# 2 21a 2 G Kr=TestOperatorSignatures(test_get_torch_func_signature_exhaustive) rvariant_test_namerrrrrrrrrrrzlinalg.matrix_power) matrix_powerlinalg_matrix_power) rrrrsupports_inplace_autogradrrrrrzlinalg.multi_dotlinalg_multi_dotz67470!test_noncontiguous_samples TestOpInfoxla TestNNCOpInfotest_nnc_correctness) rr dtypesIfCUDArrrrrrrrrz linalg.norm linalg_normtest_fn_gradgrad) rrrrrrrrrsubgradients_at_zerortest_forward_mode_AD test_fn_grad) rrrrrrrrrrzlinalg.matrix_normlinalg_matrix_norm) rrrrrrrrrz linalg.qr linalg_qr)rrrrrrrrzlinalg.slogdetlinalg_slogdet)rrrrrrrz linalg.vander linalg_vander) rrrrrrrrrzlinalg.vector_norm propagatelinalg_vector_normTestReductionstest_dim_emptytest_dim_empty_keepdim) ridentity nan_policysupports_multiple_dimscomplex_to_realrrrrgenerate_args_kwargsrrrlinalg_lu_factortest_compare_cpu) rrrrrrrrrzlinalg.lu_factor_exlinalg_lu_factor_exr linalg_lurlinalg_lu_solvezTests different backward paths'test_floating_inputs_are_differentiable) rrrrrrrrrrz linalg.inv linalg_inv)inverse) rrrrrrrrrrz linalg.inv_ex linalg_inv_ex) rrrrrrrrrz linalg.solve linalg_solveg'eRC>ga2U0*C?) rrrrrrrrrzlinalg.solve_exlinalg_solve_ex)rrrrrrrrzlinalg.solve_triangularlinalg_solve_triangular)rrrrrrrzlinalg.matrix_ranklinalg_matrix_rank)rrrrrrr7)rrrrrrrz linalg.pinv linalg_pinv) rrrrrrrrrrrsingularcZtjj||jzSr])rrpinvr)r rs r;rrs ))!add(3r=) rrrrrrrrrrrrgh㈵>) rrrrrrrrrrrrrK linalg_svd _linalg_svd) rrdecomp_aten_namerrrrrrrrrrzlinalg.svdvalslinalg_svdvals) rrr@rrrrrrrrzlinalg.tensorinv)rrrrrrrrzlinalg.tensorsolvecFtjj|||S)N)axes)rr tensorsolve)r rrs r;rrsBII$9$9!QT$9$Jr=g-C6J?g_>Y1\>)rrrrrrrop_dbz_refs.linalg.crosstest_python_ref_errors)torch_opinfo_namerrErz_refs.linalg.diagonal)rGrrEz_refs.linalg.vecdot)rGrEz_refs.linalg.vector_normz_refs.linalg.matrix_norm)rGrvalidate_view_consistencyrEz_refs.linalg.normz_refs.linalg.svdz_refs.linalg.svdvals python_ref_dbrr]r)rS)rrunittestcollections.abcr functoolsrrrnumpyrrr torch.testingr#torch.testing._internal.common_cudar r r *torch.testing._internal.common_device_typer r rrrrrrrr$torch.testing._internal.common_dtyperrrrr'rrrrr#torch.testing._internal.opinfo.corerr r!r"r#r$r%r&r'r(#torch.testing._internal.opinfo.refsr)r*rEryrrrrrrrrrrrrr rrrr1r<rJrVr]r_rir~rrrrrrrrrrrrrrrhalfrskiprboolrchalf complex64 complex128reigexpectedFailureeigvalshouseholder_productlongnormrrr vector_normr lu_factor_exlulu_solveinvinv_exrsolve_exsolve_triangularr=cdoublesvdsvdvals tensorinvrElist__annotations__rIr?r=r;rns $$ %       V/&* ZWE,U@C.6&WR#.D&7;]@  +022VF+0,> + A0 +0 6H+0% R+0:z /F9098z+0*4 .  +0 EI)TZ+0';T.U4 4+0,"+"3$E  9 <<  (U^^D .,#  :;$  E(  <<  )+ #B%'EF$ )E@ #.( JJ u{{  #<: AEZ #)+ ##(8;24EF [Er &)+ ##(8;24EF sEJ ! " ;-ejj%..I6#( #  j)+))9)9:    :;$  !5::sV/L"MN$$"   KEF )+4$#( #124E}U  j) 1" (    j) 4" (   GEB  <<  )+3#( $ # ((, PU   j)!     j)0!     j).!   ' 6&'8-HM'CER " <<  )+:#( $ #%'89  j)!     j)0!     j).!    #SEZ )+4;#( $ #%'8-H  j)!     j)0!     j).!    #[Eb #)+4;#( $ #%'89  j)!     j)0!     j).!   ! $cEl $. << + +)+  $ ##(<  !5??CT4M"NO   ./"'!((*    mEh  %)+:68IJ i Ex  ()+:68IJ y EH  $)+9 ')G35W  !    I Ef   )+5,%'89 z24F G z24F G z2L* M  j)!     j)0!     j).!   + %g Er   ) L)+5:  #%'89 (()5S  (((:   #s El  !')+ "' # 24E}U< m EH  $(U^^D3EJJO"' $ ##(91  h'7S   j)! }    j)&! }   !&I EX  <<  -emmU^^L24E}U4 #(# ((*<>P   j) 1" (    j) 4" (   YEX  <<  0-emmU^^L24E}U" %/D   $)# ((*<>P  ((*<>U  ((*<>T  113E~ V  YET &-emmU^^L #($#24E}U;  j) 1" (    j) 4" (   UEN  <?9  &'EF%qEX  <<  )+:$ #24EF  j)!     j)0!     j).!    "YE^ ! <<  )+:$ #24EF  j)!     j)0!     j).!    !_Eb  <<  )+5  # *  !5==#72O"PQ,!      j)!     j)0!     j).!    )+cEz # << )+5 # *  !5==#72O"PQ,!      j)!     j)0!     j).!    %){EN !+ << ( ()+@# "  OEd &)+424EF  j)!     j)0!     j).7  ! eEb &%)+>24EF  j)!     j).!    cET  <<  )+  $ #424EF  j)."   UEB $ 4)+ $ #=24EF (()5S   j)"'"     j)"""    %'CER %)+ $ ##(>;%'89  j)!     j)0!     j).!    !5==#4d2K"LM,"   &"'"  ;# 2SEx  <<  &)+ # #( $,24E}U  j)!     j)0!     j).!     j) 1" (    j) 4" (  =& !7yEh!  <<  "&)+#(# $724E}U  j) 1" (    j) 4" (   i!Eh"  II  )+2 ##(%'EF  :;$  i"EL#  J)+4 # *  !5==#5u2M"NO,"   !5==#5u2M"NO,!   "  :;$  /M#EtF|EV$( ((,8P  +  ) ". 113CEU V ((*: