AVhWdZddlZddlmZddlmZddlmZddlmZddlm Z ddlm Z dd lm Z dd lm Z dd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZe j4Zej8Zej:Zej<Zej@Z!edejDe!e jFZ$e jJZ&ejNZ(ejRZ*ejVZ+ejXZ,ejZZ.ej^Z0ejbZ1edejDe0ejdZ3ejhZ4ejjZ5e jlZ7ejpZ9ejtZ;ejxZe j~Z@ejZBedejDd9dZCedejDd9dZDdZEdZFdZGdZHdZIed ejDd9d!ZJed"g# d:d$ZKed%ejD d;d&ZLd'ZMed(ejDdd0ZRed1ejDd>d2ZSed3ejDd>d4ZTd5ZUd6ZVed7ejD d?d8ZWy)@zOperations for linear algebra.N) constant_op)dtypes)ops) tensor_shape) array_ops)array_ops_stack) check_ops)cond)control_flow_ops)gen_linalg_ops) linalg_ops)map_fn)math_ops)special_math_ops)stateless_random_ops while_loop)dispatch) tf_exportzlinalg.slogdetz linalg.logmz linalg.logdetc 0tj|d|g5tj|}dt j t j t jtj|dgzcdddS#1swYyxYw)aComputes log of the determinant of a hermitian positive definite matrix. ```python # Compute the determinant of a matrix while reducing the chance of over- or underflow: A = ... # shape 10 x 10 det = tf.exp(tf.linalg.logdet(A)) # scalar ``` Args: matrix: A `Tensor`. Must be `float16`, `float32`, `float64`, `complex64`, or `complex128` with shape `[..., M, M]`. name: A name to give this `Op`. Defaults to `logdet`. Returns: The natural log of the determinant of `matrix`. @compatibility(numpy) Equivalent to numpy.linalg.slogdet, although no sign is returned since only hermitian positive definite matrices are supported. @end_compatibility logdet@axisN) r name_scoper choleskyr reduce_sumlogrealrmatrix_diag_part)matrixnamechols X/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/ops/linalg/linalg_impl.pyrrDsu6 ~~dHvh/  " "6 *D $$ X]]9#=#=d#CDET s A)B  Bzlinalg.adjointctj|d|g5tj|d}tj|dcdddS#1swYyxYw)ahTransposes the last two dimensions of and conjugates tensor `matrix`. For example: ```python x = tf.constant([[1 + 1j, 2 + 2j, 3 + 3j], [4 + 4j, 5 + 5j, 6 + 6j]]) tf.linalg.adjoint(x) # [[1 - 1j, 4 - 4j], # [2 - 2j, 5 - 5j], # [3 - 3j, 6 - 6j]] ``` Args: matrix: A `Tensor`. Must be `float16`, `float32`, `float64`, `complex64`, or `complex128` with shape `[..., M, M]`. name: A name to give this `Op` (optional). Returns: The adjoint (a.k.a. Hermitian transpose a.k.a. conjugate transpose) of matrix. adjointr"r#T conjugateN)rrconvert_to_tensorrmatrix_transpose)r"r#s r%r'r'fsM0 ~~dIx0>  " "6 9F  % %f =>>>s .AAcgd}|Dcgc]"}tj||j$}}tjt j |dt j |dd|j}tj||}||d|zz}tj||}|d|z|d|zz}||fScc}w)z23rd-order Pade approximant for matrix exponential.)g^@gN@g(@N batch_shapedtyper rconstantr1r eyershapermatmul)r"bxidentmatrix_2tmpmatrix_umatrix_vs r%_matrix_exp_pade3r@s!678{Av||,8!8 ..oofb!//&)#2. LL %__VV ,(1Q4%<# __VS )( qTH_qte| +( 8 9s'C cgd}|Dcgc]"}tj||j$}}tjt j |dt j |dd|j}tj||}tj||}||d|zz|d|zz}tj||}|d|z|d|zz|d |zz}||fScc}w) z25th-order Pade approximant for matrix exponential.)g@g@g@@g@z@g>@r.Nr/r2r3rr4) r"r9r:r;r<matrix_4r=r>r?s r%_matrix_exp_pade5rEs-!678{Av||,8!8 ..oofb!//&)#2. LL %__VV ,( __Xx 0(1Q4(?"QqTE\1# __VS )( qTH_qth .1 =( 8 9s'C4cBgd}|Dcgc]"}tj||j$}}tjt j |dt j |dd|j}tj||}tj||}tj||}||d|zz|d|zz|d|zz}tj||}|d|z|d |zz|d |zz|d |zz} || fScc}w) z27th-order Pade approximant for matrix exponential.)g~pAg~`Ag@t>Ag@Ag@g@gL@r.Nr/rBr2rCr3rr4) r"r9r:r;r<rDmatrix_6r=r>r?s r%_matrix_exp_pade7rJsI!678{Av||,8!8 ..oofb!//&)#2. LL %__VV ,( __Xx 0( __Xx 0(1Q4(?"QqTH_4qte|C# __VS )( qTH_qth .1 @1Q4%< O( 8 9s'Dcgd}|Dcgc]"}tj||j$}}tjt j |dt j |dd|j}tj||}tj||}tj||}tj||}||d|zz|d|zz|d|zz|d|zz}tj||} |d |z|d |zz|d |zz|d |zz|d |zz} | | fScc}w)z29th-order Pade approximant for matrix exponential.) gynBgynBgAg@ Ag2|Ag~@Ag@g@gV@r.Nr/rGrBr2rHrCr3rr4) r"r9r:r;r<rDrImatrix_8r=r>r?s r%_matrix_exp_pade9rOsT!788{Av||,8!8 ..oofb!//&)#2. LL %__VV ,( __Xx 0( __Xx 0( __Xx 0(1 1Q4(?2QqTH_DdUl__VS )(dXo!x'!A$/9AaD8OKdUl  8 !9s'Ecgd}|Dcgc]"}tj||j$}}tjt j |dt j |dd|j}tj||}tj||}tj||}tj|||d|zz|d|zz|d|zz|d|zz|d |zz|d |zz}tj||}|d |z|d |zz|d |zz} tj|| |d|zz|d|zz|d|zz|d|zz} || fScc}w)z313th-order Pade approximant for matrix exponential.) gD`lCgD`\Cg`=Hb;Cg eCgJXBg"5Bg/cBg\L8BgpķAgsyAgS-Ag@gf@r.Nr/ rLrGrBr2 rMrHrCr3rr4) r"r9r:r;r<rDrItmp_ur>tmp_vr?s r%_matrix_exp_pade13rWs! 788{Av||,8!8 ..oofb!//&)#2. LL %__VV ,( __Xx 0( __Xx 0(ooh1R58+; ;adXo MNdXo!x(*+A$/:<=aD5LI__VU +( B%( QrUX- -!x ?%ooh&181Q4(?JdXo!u %  8 !9s'E<z linalg.expmc p tj|d|g5tj|djddddgk(r cdddSjdd}|j st jdd}t j t jdgt jddfdtjtjtjt jt jd z dd t jt jffd fd jtj tj"tj$fvrd }tj&tj(tj*|z tj*dz dt-\}}t/\}}t1tj2tj4djz \}} d} | |||f| ||| fnHjtj6tj8fvrd}tj&tj(tj*|z tj*dz dt-\}}t/\}}t1\}} t;\} } t=tj2tj4djz \} }d} | |||| | f| ||| | |fnt?djztj@tjtCjDtFjHjtKjLfdfd}tjd}fd}fd}tOjN||||g\}}jj sLt j |t j|t j|ddfdcdddSt j ||jQ|jddcdddS#1swYyxYw)aComputes the matrix exponential of one or more square matrices. $$exp(A) = \sum_{n=0}^\infty A^n/n!$$ The exponential is computed using a combination of the scaling and squaring method and the Pade approximation. Details can be found in: Nicholas J. Higham, "The scaling and squaring method for the matrix exponential revisited," SIAM J. Matrix Anal. Applic., 26:1179-1193, 2005. The input is a tensor of shape `[..., M, M]` whose inner-most 2 dimensions form square matrices. The output is a tensor of the same shape as the input containing the exponential for all input submatrices `[..., :, :]`. Args: input: A `Tensor`. Must be `float16`, `float32`, `float64`, `complex64`, or `complex128` with shape `[..., M, M]`. name: A name to give this `Op` (optional). Returns: the matrix exponential of the input. Raises: ValueError: An unsupported type is provided as input. @compatibility(scipy) Equivalent to scipy.linalg.expm @end_compatibility matrix_exponentialinputr(r.Nrrrr3.cDtj|jSN)rr5r1)r:l1_norms r%z$matrix_exponential..sk**1gmm<c Tt|t|dz k(sJt|dk(r:tjtj|d|d|dStjtj|d|d|dd|ddS)Nr2r)lenrwhere_v2rless)valscases _nest_whereconstr]s r%rfz'matrix_exponential.._nest_wheres Y#e*q. (( ( Ta!! MM'5a> 2E!HeAhH H!! MM'5a> 2E!H QR%) ,. .r_gj%eg@r)g#"ՀA?gNj?gC|@)g,?g|zی@?gQi?g d@z3tf.linalg.expm does not support matrices of type %sc>tj zzSr\)r matrix_solve)uvsr%r^z$matrix_exponential..Hs:22A261q5Ar_cVtjtjSr\)rfillr7)r"nansr%r^z$matrix_exponential..Is yv6<r_c>tjfddS)Nc0tjSr\rrc)i max_squaringssr%r^z/matrix_exponential..c..Os(--="Ar_c,tjdS)NF)rr5r_r%r^z/matrix_exponential..c..Ps+"6"6u"=r_)tf_condr )rs_ is_finiterts` r%czmatrix_exponential..cMs \\)A=??r_c|dztjtj|tj|||fSNr2)rrbrrcr8)rsr squaringss r%r9zmatrix_exponential..bRsA UI&& --9 %xq!' ~~d05':TQ  " "5w 7F ||BCQF" TQTQ,,s#K  ' ' )OOF+CR0k     2$ (?(D!EANPF!! LL   781< > i''):):: F'' 2M c A? A%%aQK8IAv << ( ( *       K)@)EFQ OQcTQTQh   V[%<%>> x = [[2., 3., 4.], [1., 2., 3.]] >>> x2 = [[2., 3., 4.], [10000., 2., 3.]] >>> y = tf.zeros([3, 3]) >>> z = tf.linalg.set_diag(y, x, align='LEFT_RIGHT', k=(-1, 0)) >>> z >>> soln = tf.linalg.banded_triangular_solve(x, tf.ones([3, 1])) >>> soln >>> are_equal = soln == tf.linalg.banded_triangular_solve(x2, tf.ones([3, 1])) >>> print(tf.reduce_all(are_equal).numpy()) True >>> are_equal = soln == tf.linalg.triangular_solve(z, tf.ones([3, 1])) >>> print(tf.reduce_all(are_equal).numpy()) True Storing 2 superdiagonals of a 4x4 matrix. Because of the 'LEFT_RIGHT' padding the last element of the first row is ignored. >>> x = [[2., 3., 4., 5.], [-1., -2., -3., -4.]] >>> y = tf.zeros([4, 4]) >>> z = tf.linalg.set_diag(y, x, align='LEFT_RIGHT', k=(0, 1)) >>> z >>> soln = tf.linalg.banded_triangular_solve(x, tf.ones([4, 1]), lower=False) >>> soln >>> are_equal = (soln == tf.linalg.triangular_solve( ... z, tf.ones([4, 1]), lower=False)) >>> print(tf.reduce_all(are_equal).numpy()) True Args: bands: A `Tensor` describing the bands of the left hand side, with shape `[..., K, M]`. The `K` rows correspond to the diagonal to the `K - 1`-th diagonal (the diagonal is the top row) when `lower` is `True` and otherwise the `K - 1`-th superdiagonal to the diagonal (the diagonal is the bottom row) when `lower` is `False`. The bands are stored with 'LEFT_RIGHT' alignment, where the superdiagonals are padded on the right and subdiagonals are padded on the left. This is the alignment cuSPARSE uses. See `tf.linalg.set_diag` for more details. rhs: A `Tensor` of shape [..., M] or [..., M, N] and with the same dtype as `diagonals`. Note that if the shape of `rhs` and/or `diags` isn't known statically, `rhs` will be treated as a matrix rather than a vector. lower: An optional `bool`. Defaults to `True`. Boolean indicating whether `bands` represents a lower or upper triangular matrix. adjoint: An optional `bool`. Defaults to `False`. Boolean indicating whether to solve with the matrix's block-wise adjoint. name: A name to give this `Op` (optional). Returns: A `Tensor` of shape [..., M] or [..., M, N] containing the solutions. banded_triangular_solve)lowerr'N)rrr r)bandsrhsrr'r#s r%rr^sFv ~~d5s|D2  1 1 s% 2222s =Azlinalg.tridiagonal_solvec &|r |s td|dk(rt|||||||S|dk(r/t|ttfrt |dk7r td|\}} } | j ddj| j ddr+|j ddj| j dds:tdj| j | j |j tj| j dfd } | | d d d g} | |d d d g}tj|| | fd}t|||||||S|dk(rtj|j d} tj|j d} | r"| r | | k7rtdj| | | xs| tj|ddd}t|||||||Stdj|)a/Solves tridiagonal systems of equations. The input can be supplied in various formats: `matrix`, `sequence` and `compact`, specified by the `diagonals_format` arg. In `matrix` format, `diagonals` must be a tensor of shape `[..., M, M]`, with two inner-most dimensions representing the square tridiagonal matrices. Elements outside of the three diagonals will be ignored. In `sequence` format, `diagonals` are supplied as a tuple or list of three tensors of shapes `[..., N]`, `[..., M]`, `[..., N]` representing superdiagonals, diagonals, and subdiagonals, respectively. `N` can be either `M-1` or `M`; in the latter case, the last element of superdiagonal and the first element of subdiagonal will be ignored. In `compact` format the three diagonals are brought together into one tensor of shape `[..., 3, M]`, with last two dimensions containing superdiagonals, diagonals, and subdiagonals, in order. Similarly to `sequence` format, elements `diagonals[..., 0, M-1]` and `diagonals[..., 2, 0]` are ignored. The `compact` format is recommended as the one with best performance. In case you need to cast a tensor into a compact format manually, use `tf.gather_nd`. An example for a tensor of shape [m, m]: ```python rhs = tf.constant([...]) matrix = tf.constant([[...]]) m = matrix.shape[0] dummy_idx = [0, 0] # An arbitrary element to use as a dummy indices = [[[i, i + 1] for i in range(m - 1)] + [dummy_idx], # Superdiagonal [[i, i] for i in range(m)], # Diagonal [dummy_idx] + [[i + 1, i] for i in range(m - 1)]] # Subdiagonal diagonals=tf.gather_nd(matrix, indices) x = tf.linalg.tridiagonal_solve(diagonals, rhs) ``` Regardless of the `diagonals_format`, `rhs` is a tensor of shape `[..., M]` or `[..., M, K]`. The latter allows to simultaneously solve K systems with the same left-hand sides and K different right-hand sides. If `transpose_rhs` is set to `True` the expected shape is `[..., M]` or `[..., K, M]`. The batch dimensions, denoted as `...`, must be the same in `diagonals` and `rhs`. The output is a tensor of the same shape as `rhs`: either `[..., M]` or `[..., M, K]`. The op isn't guaranteed to raise an error if the input matrix is not invertible. `tf.debugging.check_numerics` can be applied to the output to detect invertibility problems. **Note**: with large batch sizes, the computation on the GPU may be slow, if either `partial_pivoting=True` or there are multiple right-hand sides (`K > 1`). If this issue arises, consider if it's possible to disable pivoting and have `K = 1`, or, alternatively, consider using CPU. On CPU, solution is computed via Gaussian elimination with or without partial pivoting, depending on `partial_pivoting` parameter. On GPU, Nvidia's cuSPARSE library is used: https://docs.nvidia.com/cuda/cusparse/index.html#gtsv Args: diagonals: A `Tensor` or tuple of `Tensor`s describing left-hand sides. The shape depends of `diagonals_format`, see description above. Must be `float32`, `float64`, `complex64`, or `complex128`. rhs: A `Tensor` of shape [..., M] or [..., M, K] and with the same dtype as `diagonals`. Note that if the shape of `rhs` and/or `diags` isn't known statically, `rhs` will be treated as a matrix rather than a vector. diagonals_format: one of `matrix`, `sequence`, or `compact`. Default is `compact`. transpose_rhs: If `True`, `rhs` is transposed before solving (has no effect if the shape of rhs is [..., M]). conjugate_rhs: If `True`, `rhs` is conjugated before solving. name: A name to give this `Op` (optional). partial_pivoting: whether to perform partial pivoting. `True` by default. Partial pivoting makes the procedure more stable, but slower. Partial pivoting is unnecessary in some cases, including diagonally dominant and symmetric positive definite matrices (see e.g. theorem 9.12 in [1]). perturb_singular: whether to perturb singular matrices to return a finite result. `False` by default. If true, solutions to systems involving a singular matrix will be computed by perturbing near-zero pivots in the partially pivoted LU decomposition. Specifically, tiny pivots are perturbed by an amount of order `eps * max_{ij} |U(i,j)|` to avoid overflow. Here `U` is the upper triangular part of the LU decomposition, and `eps` is the machine precision. This is useful for solving numerically singular systems when computing eigenvectors by inverse iteration. If `partial_pivoting` is `False`, `perturb_singular` must be `False` as well. Returns: A `Tensor` of shape [..., M] or [..., M, K] containing the solutions. If the input matrix is singular, the result is undefined. Raises: ValueError: Is raised if any of the following conditions hold: 1. An unsupported type is provided as input, 2. the input tensors have incorrect shapes, 3. `perturb_singular` is `True` but `partial_pivoting` is not. UnimplementedError: Whenever `partial_pivoting` is true and the backend is XLA, or whenever `perturb_singular` is true and the backend is XLA or GPU. [1] Nicholas J. Higham (2002). Accuracy and Stability of Numerical Algorithms: Second Edition. SIAM. p. 175. ISBN 978-0-89871-802-7. z5partial_pivoting must be True if perturb_singular is.compactsequencerBz0Expected diagonals to be a sequence of length 3.NrzoTensors representing the three diagonals must have the same shape,except for the last dimension, got {}, {}, {}cJtj|jd}|r|k(r|S|dz k(rKtt |jdz Dcgc]}ddgc}|gz}t j ||Stdj|dz |cc}w)Nrr2rz/Expected {} to be have length {} or {}, got {}.) rdimension_valuer7rangerarpadrformat)tr#last_dim_paddingnrxpaddingsms r%pad_if_necessaryz+tridiagonal_solve..pad_if_necessaryHs  & &qwwr{ 3a !q& a!e%*3qwws B  subdiagonalr2r superdiagonalr.rr"CExpected last two dimensions of diagonals to be same, got {} and {}rr2ro LEFT_RIGHTk padding_valuealignz!Unrecognized diagonals_format: {})r!_tridiagonal_solve_compact_format isinstancetuplelistrar7is_compatible_withrrrrstackrr!) diagonalsrdiagonals_format transpose_rhs conjugate_rhsr#partial_pivotingperturb_singular superdiagmaindiagsubdiagrm1m2rs @r%tridiagonal_solvers2h. L MM" ,Y]-:Q3F I JJ#, Ix MM#2  1 1(.."2E F OOCR 3 3HNN3B4G H  ::@&mmX^^Y__;> ?? $$X^^B%78A w 1v>G OaVDI%%y(G&D2NI ,Y]-:  bA**WBlDI ,Y]-:NOPPr_c jjjj}}|r|dkrtdj||r,||k7r'||dz k7rtdj|dz |||rfjddj jd|dz s8tdjjddjd|dz jdr9jddk7r'td jjdfd } |rk|ri||dz k(ra|rt j tjd | tjtj|||d S|rtj| n|rt j | tj|||S) zCHelper function used after the input has been cast to compact form.r3z2Expected diagonals to have rank at least 2, got {}r2z/Expected the rank of rhs to be {} or {}, got {}Nr.z'Batch shapes {} and {} are incompatiblerBzExpected 3 diagonals got {}cjdrejdrUjdjdk7r5tdjjdjdyyy)Nrr.zRExpected number of left-hand sided and right-hand sides to be equal, got {} and {})r7rr)rrsr%check_num_lhs_matches_num_rhszH_tridiagonal_solve_compact_format..check_num_lhs_matches_num_rhsss " syy}, ::@&$??2. " ;? @@ -!.r_rr)) r7rankrrrrconjr expand_dimssqueezer rr,) rrrrrrr# diags_rankrhs_rankrs `` r%rrls#--syy~~h*A~  > E E  H *x:>/I HOO q.*h0 11"-@@ /:>"$ @GG //#2  /:> :< ==__RY__R0A5 299)//":MN OO@*Z!^!; MM# c   R (C!#   $$Y5E%5t =>@ BB  $ $SM BC -- C!  % %i6F&6 >>r_zlinalg.tridiagonal_matmulc~|dk(r|ddddf}|ddddf}|ddddf}n|dk(r|\}}}n|dk(rtj|jd }tj|jd }|r"|r ||k7rtd j ||t j |d d d} | ddddf}| ddddf}| ddddf}ntd|zt j|d }t j|d }t j|d }tj|||||S)aMultiplies tridiagonal matrix by matrix. `diagonals` is representation of 3-diagonal NxN matrix, which depends on `diagonals_format`. In `matrix` format, `diagonals` must be a tensor of shape `[..., M, M]`, with two inner-most dimensions representing the square tridiagonal matrices. Elements outside of the three diagonals will be ignored. If `sequence` format, `diagonals` is list or tuple of three tensors: `[superdiag, maindiag, subdiag]`, each having shape [..., M]. Last element of `superdiag` first element of `subdiag` are ignored. In `compact` format the three diagonals are brought together into one tensor of shape `[..., 3, M]`, with last two dimensions containing superdiagonals, diagonals, and subdiagonals, in order. Similarly to `sequence` format, elements `diagonals[..., 0, M-1]` and `diagonals[..., 2, 0]` are ignored. The `sequence` format is recommended as the one with the best performance. `rhs` is matrix to the right of multiplication. It has shape `[..., M, N]`. Example: ```python superdiag = tf.constant([-1, -1, 0], dtype=tf.float64) maindiag = tf.constant([2, 2, 2], dtype=tf.float64) subdiag = tf.constant([0, -1, -1], dtype=tf.float64) diagonals = [superdiag, maindiag, subdiag] rhs = tf.constant([[1, 1], [1, 1], [1, 1]], dtype=tf.float64) x = tf.linalg.tridiagonal_matmul(diagonals, rhs, diagonals_format='sequence') ``` Args: diagonals: A `Tensor` or tuple of `Tensor`s describing left-hand sides. The shape depends of `diagonals_format`, see description above. Must be `float32`, `float64`, `complex64`, or `complex128`. rhs: A `Tensor` of shape [..., M, N] and with the same dtype as `diagonals`. diagonals_format: one of `sequence`, or `compact`. Default is `compact`. name: A name to give this `Op` (optional). Returns: A `Tensor` of shape [..., M, N] containing the result of multiplication. Raises: ValueError: An unsupported type is provided as input, or when the input tensors have incorrect shapes. r.rNr2r3rr"rr.rrrorrz!Unrecognized diagonals_format: %s) rrr7rrrr!rr tridiagonal_mat_mul) rrrr#rrrrrdiagss r%tridiagonal_matmulrs`f"#q!)$Ia#HQ "G:%#, Ix8#  % %ioob&9 :B  % %ioob&9 :B bR2X  O 6"b>   & &WBl DEc1ai IS!QYHCAIG 8;KK LL##Ir2)  " "8R 0(  ! !'2 .'  ' ' 8Wc4 PPr_cg}|jjs.tdj|jj|j _|j j I|j j dkr.tdj|j j |S|r'|jtj|dd|S)z&Checks that input is a `float` matrix.z2Input `a` must have `float`-like `dtype` (saw {}).r3z4Input `a` must have at least 2 dimensions (saw: {}).z*Input `a` must have at least 2 dimensions.rmessage) r1 is_floating TypeErrorrr#r7rrappendr assert_rank_at_least)a validate_args assertionss r%_maybe_validate_matrixrs*     &qww|| 4 66WWQWW\\5ww||a $$*F177<<$8 ::  && AK MN r_zlinalg.matrix_rankctj|xsd5tj|tjd}t ||}|r3tj |5tj|}dddt|d}||jddjr1tj|jddj}n+tj tj|dd}tj"|j$j&j(}|tj*||j$ztj |dd z}tj,tj*||kDtj.d cdddS#1swYFxYw#1swYyxYw) aCompute the matrix rank of one or more matrices. Args: a: (Batch of) `float`-like matrix-shaped `Tensor`(s) which are to be pseudo-inverted. tol: Threshold below which the singular value is counted as 'zero'. Default value: `None` (i.e., `eps * max(rows, cols) * max(singular_val)`). validate_args: When `True`, additional assertions might be embedded in the graph. Default value: `False` (i.e., no graph assertions are added). name: Python `str` prefixed to ops created by this function. Default value: 'matrix_rank'. Returns: matrix_rank: (Batch of) `int32` scalars representing the number of non-zero singular values. matrix_rankr dtype_hintr#NF) compute_uvr.rT)rkeepdimsr)rrr+rrrcontrol_dependenciesridentitysvdr7rrmaxas_listrrfinfor1as_numpy_dtypeepsrrint32)rtolrr#rsrrs r%rrsl( ~~d+m,N aFNNEA'=9J  # #J /"   q !" A% A { ''"#, ( ( * FF17723<'') *     223 7 8 HHQWW++ , 0 0c  a) )   ab4 8 9    x}}QWfllC" M!NN"" NNs%A G $F>:D:G >G G  Gz linalg.pinvc tj|xsd5tjdt|}|r3tj|5t j dddjj}|fd}|d}|d}t|tr&t|trtt||} n*tjtj|||} d| zt!j"|j$z}tj||d }t'd d \} } } |tj(| dz} t j*| t j,| dkD| t!j.t j0|} tj2| t j,| dz | d }j4_j4j6I|j9j4ddj;j4dj4dg|cdddS#1swYxYw#1swYyxYw)a Compute the Moore-Penrose pseudo-inverse of one or more matrices. Calculate the [generalized inverse of a matrix]( https://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_inverse) using its singular-value decomposition (SVD) and including all large singular values. The pseudo-inverse of a matrix `A`, is defined as: 'the matrix that 'solves' [the least-squares problem] `A @ x = b`,' i.e., if `x_hat` is a solution, then `A_pinv` is the matrix such that `x_hat = A_pinv @ b`. It can be shown that if `U @ Sigma @ V.T = A` is the singular value decomposition of `A`, then `A_pinv = V @ inv(Sigma) U^T`. [(Strang, 1980)][1] This function is analogous to [`numpy.linalg.pinv`]( https://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.pinv.html). It differs only in default value of `rcond`. In `numpy.linalg.pinv`, the default `rcond` is `1e-15`. Here the default is `10. * max(num_rows, num_cols) * np.finfo(dtype).eps`. Args: a: (Batch of) `float`-like matrix-shaped `Tensor`(s) which are to be pseudo-inverted. rcond: `Tensor` of small singular value cutoffs. Singular values smaller (in modulus) than `rcond` * largest_singular_value (again, in modulus) are set to zero. Must broadcast against `tf.shape(a)[:-2]`. Default value: `10. * max(num_rows, num_cols) * np.finfo(a.dtype).eps`. validate_args: When `True`, additional assertions might be embedded in the graph. Default value: `False` (i.e., no graph assertions are added). name: Python `str` prefixed to ops created by this function. Default value: 'pinv'. Returns: a_pinv: (Batch of) pseudo-inverse of input `a`. Has same shape as `a` except rightmost two dimensions are transposed. Raises: TypeError: if input `a` does not have `float`-like `dtype`. ValueError: if input `a` has fewer than 2 dimensions. #### Examples ```python import tensorflow as tf import tensorflow_probability as tfp a = tf.constant([[1., 0.4, 0.5], [0.4, 0.2, 0.25], [0.5, 0.25, 0.35]]) tf.matmul(tf.linalg.pinv(a), a) # ==> array([[1., 0., 0.], [0., 1., 0.], [0., 0., 1.]], dtype=float32) a = tf.constant([[1., 0.4, 0.5, 1.], [0.4, 0.2, 0.25, 2.], [0.5, 0.25, 0.35, 3.]]) tf.matmul(tf.linalg.pinv(a), a) # ==> array([[ 0.76, 0.37, 0.21, -0.02], [ 0.37, 0.43, -0.33, 0.02], [ 0.21, -0.33, 0.81, 0.01], [-0.02, 0.02, 0.01, 1. ]], dtype=float32) ``` #### References [1]: G. Strang. 'Linear Algebra and Its Applications, 2nd Ed.' Academic Press, Inc., 1980, pp. 139-142. pinvrr(Nctjj|}||Stj|Sr\)rrr7r)dimdim_valrs r% get_dim_sizezpinv..get_dim_sizezs:..qwws|<  .q!#&&r_r.rg$@rcond)r1r#FT) full_matricesrr) adjoint_b)rrr+rrrrr1rrintfloatrrrrrrrrrrbexpand_dims_v2arrayinfr8r7r set_shaper)rrrr#rr1rnum_rowsnum_cols max_rows_colssingular_valuesleft_singular_vectorsright_singular_vectorscutoffa_pinvs` r%rr'sN ~~dnf%8 ac*A'=9J  # #J /"   q !" GG " "E }' b!hb!h Hc "z(C'@c(H56    Xx 0%9 M!BHHUO$7$77e  ! !%u7 CE  4 1  X((rB BF(()2262>> !O __!9!9/2!NNF  wwqww||7 qwws|//aggbk0JKL q88 "" 88s$;I,I,G)I,I) $I,,I5zlinalg.lu_solvec Xtj|xsd5tj|tjd}tj|tj d}tj||j d}t||||}|r]tj|5tj|}tj|}tj|}ddd|jjdk(r3|jjdk(rtj||d }nctj|}tj|dd tj|dd }|d |d } } tj|| | ggd } tj || } tj"| d | | g} tj || dd } tj"| d | g} t%j&|}tj t%j(|ddtj*f|| g}t-j.|| gd } tj0| | }tj"|| }t3t5|d d tj6tj|dd |j }t9|t9||dcdddS#1swY'xYw#1swYyxYw)aSolves systems of linear eqns `A X = RHS`, given LU factorizations. Note: this function does not verify the implied matrix is actually invertible nor is this condition checked even when `validate_args=True`. Args: lower_upper: `lu` as returned by `tf.linalg.lu`, i.e., if `matmul(P, matmul(L, U)) = X` then `lower_upper = L + U - eye`. perm: `p` as returned by `tf.linag.lu`, i.e., if `matmul(P, matmul(L, U)) = X` then `perm = argmax(P)`. rhs: Matrix-shaped float `Tensor` representing targets for which to solve; `A X = RHS`. To handle vector cases, use: `lu_solve(..., rhs[..., tf.newaxis])[..., 0]`. validate_args: Python `bool` indicating whether arguments should be checked for correctness. Note: this function does not verify the implied matrix is actually invertible, even when `validate_args=True`. Default value: `False` (i.e., don't validate arguments). name: Python `str` name given to ops managed by this object. Default value: `None` (i.e., 'lu_solve'). Returns: x: The `X` in `A @ X = RHS`. #### Examples ```python import numpy as np import tensorflow as tf import tensorflow_probability as tfp x = [[[1., 2], [3, 4]], [[7, 8], [3, 4]]] inv_x = tf.linalg.lu_solve(*tf.linalg.lu(x), rhs=tf.eye(2)) tf.assert_near(tf.matrix_inverse(x), inv_x) # ==> True ``` lu_solve lower_upperrpermrNr3r2r.rrr num_lower num_upperr1F)r)rrr+rrrr1_lu_solve_assertionsrrrr7rgatherbroadcast_dynamic_shaper broadcast_torr reduce_prodrrrr gather_ndset_diag band_partonestriangular_solve)rrrrr#r permuted_rhs rhs_shapebroadcast_batch_shapedrrhs_broadcast_shape broadcast_rhsbroadcast_permbroadcast_batch_sizebroadcast_batch_indicesrs r%rrsX ~~d(j)3'']DK  &,,V LD   0A0A NC%k4mLJ  # #J /&((5 !!$'  %& ! 1 4%%c4b9l//#&i'?? CR. //$  $&r]IbMa%,,.CaV-L235 ,,S2EFm'' AqzBm!--d4G4LMn (("aAn%112GH ) 6 6 ..- .q)2C2C/C D  #!%',, "N 3">n((Gl&&|5HIl +q9 OOK (" -[5F5F H IE  - a33&&33s&BL .AL.HL L L  L)zlinalg.lu_matrix_inversec >tj|xsd5tj|tjd}tj|tj d}t |||}|rHtj|5tj|}tj|}dddtj|}t||t|d|dd|jd cdddS#1swYRxYw#1swYyxYw) aComputes the inverse given the LU decomposition(s) of one or more matrices. This op is conceptually identical to, ```python inv_X = tf.lu_matrix_inverse(*tf.linalg.lu(X)) tf.assert_near(tf.matrix_inverse(X), inv_X) # ==> True ``` Note: this function does not verify the implied matrix is actually invertible nor is this condition checked even when `validate_args=True`. Args: lower_upper: `lu` as returned by `tf.linalg.lu`, i.e., if `matmul(P, matmul(L, U)) = X` then `lower_upper = L + U - eye`. perm: `p` as returned by `tf.linag.lu`, i.e., if `matmul(P, matmul(L, U)) = X` then `perm = argmax(P)`. validate_args: Python `bool` indicating whether arguments should be checked for correctness. Note: this function does not verify the implied matrix is actually invertible, even when `validate_args=True`. Default value: `False` (i.e., don't validate arguments). name: Python `str` name given to ops managed by this object. Default value: `None` (i.e., 'lu_matrix_inverse'). Returns: inv_x: The matrix_inv, i.e., `tf.matrix_inverse(tf.linalg.lu_reconstruct(lu, perm))`. #### Examples ```python import numpy as np import tensorflow as tf import tensorflow_probability as tfp x = [[[3., 4], [1, 2]], [[7., 8], [3, 4]]] inv_x = tf.linalg.lu_matrix_inverse(*tf.linalg.lu(x)) tf.assert_near(tf.matrix_inverse(x), inv_x) # ==> True ``` lu_matrix_inverserrrNrr.r/F)rr)rrr+rrrlu_reconstruct_assertionsrrrr7rr6r1)rrrr#rr7s r%r*r* s` ~~d112'']DK  &,,V LD*;mLJ  # #J /(((5 !!$'( OOK (E  b uSbz9J9J K   (( s%A1D +D6ADD DDzlinalg.lu_reconstructc tj|xsd5tj|tjd}tj|tj d}t |||}|rHtj|5tj|}tj|}dddtj|}tt|ddtj|dd|j }t|dd}tj ||}|j/|jj"|jj"d k7rtj$|dd } |d} tj&|| | | g}tj&|| | g}t)j(tj*|}tj,tj.| ddtj0f| | g} tj2|t5j6| |gd }tj&||}n)tj8|tj*|}|j;|j|cdddS#1swYxYw#1swYyxYw) aThe reconstruct one or more matrices from their LU decomposition(s). Args: lower_upper: `lu` as returned by `tf.linalg.lu`, i.e., if `matmul(P, matmul(L, U)) = X` then `lower_upper = L + U - eye`. perm: `p` as returned by `tf.linag.lu`, i.e., if `matmul(P, matmul(L, U)) = X` then `perm = argmax(P)`. validate_args: Python `bool` indicating whether arguments should be checked for correctness. Default value: `False` (i.e., don't validate arguments). name: Python `str` name given to ops managed by this object. Default value: `None` (i.e., 'lu_reconstruct'). Returns: x: The original input to `tf.linalg.lu`, i.e., `x` as in, `lu_reconstruct(*tf.linalg.lu(x))`. #### Examples ```python import numpy as np import tensorflow as tf import tensorflow_probability as tfp x = [[[3., 4], [1, 2]], [[7., 8], [3, 4]]] x_reconstructed = tf.linalg.lu_reconstruct(*tf.linalg.lu(x)) tf.assert_near(x, x_reconstructed) # ==> True ``` lu_reconstructrrrNrrrrr3r.r)rrr+rrrr+rrrr7rrrr1rr8rrrrinvert_permutationrrrrrrrr) rrrr#rr7rupperr: batch_sizer# batch_indicess r%r-r-LsNF ~~d../$ '']DK  &,,V LD*;mLJ  # #J /(((5 !!$'( OOK (E +q9uSbz):):; =E kQ" =Eu%A![%6%6%;%;%C!#''cr 3j )a   A Aq1 2a   tj!_ 5d ]]977 >d,, .. $Q (9(9%9 :ZOMm    _ " "M4#8r B Da   Au %a   1i::4@ AaKK !!" I$ $ (($ $ s%A1J> +J16G1J>1J; 6J>>Kclg}d}|jj$|jjdkr t||r'|jt j |d|d}|jjQ|jj;|jj|jjdzk7rJt||r=|jt j |tj|dz|d}|jddjr,|jd|jd k7r t||S|rWtjtj|ddd \}}|jt j||| |S) zCReturns list of assertions related to `lu_reconstruct` assumptions.z4Input `lower_upper` must have at least 2 dimensions.Nr3rz/`rank(lower_upper)` must equal `rank(perm) + 1`r2z`lower_upper` must be square.r.r)num_or_size_splitsr) r7rrrr assert_rank_at_least_v2 assert_rankrrsplit assert_equal)rrrrrrrs r%r+r+s* B''K,=,=,B,BQ,F W ))+AwOQ >''DJJOO,G1!44 w  innT2Q6 IJ ,'s,,. 1 1" 55 w    ?? $RS)a ADAqi,,Q7CD r_c,t|||}d}|jj$|jjdkr4t||r'|j t j |d|d}|jd;|jd,|jd|jdk7r t||S|rS|j t jtj|dtj|d||S)z=Returns list of assertions related to `lu_solve` assumptions.z,Input `rhs` must have at least 2 dimensions.r3rz3`lower_upper.shape[-1]` must equal `rhs.shape[-1]`.rr.r4) r+r7ndimsrrr rr8r)rrrrrrs r%rrs(dMJ* :'YY__  yy w &&sGDF B''CIIbM,E " - w   OOK ( , OOC  $  r_zlinalg.eigh_tridiagonalc tj|xsd5 fd}d}tj|d}|jd dkrt j |cdddStj|d }|j |j k7r td |||} |r | cdddS|||| } | | fcdddS#1swYyxYw) u Computes the eigenvalues of a Hermitian tridiagonal matrix. Args: alpha: A real or complex tensor of shape (n), the diagonal elements of the matrix. NOTE: If alpha is complex, the imaginary part is ignored (assumed zero) to satisfy the requirement that the matrix be Hermitian. beta: A real or complex tensor of shape (n-1), containing the elements of the first super-diagonal of the matrix. If beta is complex, the first sub-diagonal of the matrix is assumed to be the conjugate of beta to satisfy the requirement that the matrix be Hermitian eigvals_only: If False, both eigenvalues and corresponding eigenvectors are computed. If True, only eigenvalues are computed. Default is True. select: Optional string with values in {‘a’, ‘v’, ‘i’} (default is 'a') that determines which eigenvalues to calculate: 'a': all eigenvalues. ‘v’: eigenvalues in the interval (min, max] given by `select_range`. 'i’: eigenvalues with indices min <= i <= max. select_range: Size 2 tuple or list or tensor specifying the range of eigenvalues to compute together with select. If select is 'a', select_range is ignored. tol: Optional scalar. The absolute tolerance to which each eigenvalue is required. An eigenvalue (or cluster) is considered to have converged if it lies in an interval of this width. If tol is None (default), the value eps*|T|_2 is used where eps is the machine precision, and |T|_2 is the 2-norm of the matrix T. name: Optional name of the op. Returns: eig_vals: The eigenvalues of the matrix in non-decreasing order. eig_vectors: If `eigvals_only` is False the eigenvectors are returned in the second output argument. Raises: ValueError: If input values are invalid. NotImplemented: Computing eigenvectors for `eigvals_only` = False is not implemented yet. This op implements a subset of the functionality of scipy.linalg.eigh_tridiagonal. Note: The result is undefined if the input contains +/-inf or NaN, or if any value in beta has a magnitude greater than `numpy.sqrt(numpy.finfo(beta.dtype.as_numpy_dtype).max)`. TODO(b/187527398): Add support for outer batch dimensions. #### Examples ```python import numpy eigvals = tf.linalg.eigh_tridiagonal([0.0, 0.0, 0.0], [1.0, 1.0]) eigvals_expected = [-numpy.sqrt(2.0), 0.0, numpy.sqrt(2.0)] tf.assert_near(eigvals_expected, eigvals) # ==> True ``` eigh_tridiagonalc( dtjd5jjrVt j t j t j ||zt j}n*t j|t j|}tjjj}tj|dd|dd|ddz|ddgd}t j|j t j"|z}t j$|j&t j(|z }t j$t j|t j|}tj*gjj}tj$||j z ||j,z|j.z} | t j$|t j"zt j|j,|dz|j,|z rt j$ |j0dzd} d k(rt j2nrd k(r>t5j6ddd } t j2dddzn/d k(rt5j8ddd } n t;d| r4tj<| g5tj>dddd} t j@j| z|j,z|z} dvr|| z d| zzz } || z| zz}nMd| z d| zzz } d| z| zz}| }|}t j2||t j||j }t j$| |j&} tjB}tjD| |} tjD||}tjD|tjD|dfd}fd}tGjF||d| |g\}} }| |cdddS#1swYxYw#1swYyxYw)z;Computes all eigenvalues of a Hermitian tridiagonal matrix.c6tjd5jdtjtjt j tjtjt j fd}fd|\}}d}d} dz |z} | fd} | | ||\} }}|fd } tj| | | ||gd \} } }|cd d d S#1swYy xYw) z)Implements the Sturm sequence recurrence.sturmrrcdz }tj|dk}tjtjd|}||fS)Nr)rwhererequal)qcountalphaalpha0_perturbationrr:zeross r% sturm_step0zSeigh_tridiagonal.._compute_eigenvalues.._sturm..sturm_step0"sUa1 AOOAE47EuQx+-@!EAe8Or_c||dz |z z z }tj|k|dz|}tj|ktj| |}||fSr|)rrArminimum)rsrCrDrEbeta_sqpivminr:s r% sturm_stepzReigh_tridiagonal.._compute_eigenvalues.._sturm..sturm_step*ska71q5>A--1AOOAKEBEV X-=-=a&-I1MAe8Or_r2cTtD]}||z||\}}|z||fSr\)r)startrCrDjrM unroll_cnts r%unrolled_stepszVeigh_tridiagonal.._compute_eigenvalues.._sturm..unrolled_steps:s?:& 9#EAIq%8ha 9:%q%/ /r_c0tj|Sr\rr)rsrCrDrs r%r^zPeigh_tridiagonal.._compute_eigenvalues.._sturm..DsX]]1a%8r_F) back_propN) rrr7rrGrrrr)rErKrLrFr:rHrCrD blocksizerspeelrSr rxrrrMrRrGs````` @@@@@r%_sturmz>eigh_tridiagonal.._compute_eigenvalues.._sturms ^^G $, kk!n!//)//!"4FLLI%  2&,,G$   !](!U)!a%9$$* 0 'q!U3+!Q!*8$"--NQ5MUD+!QY, , , s C%DDcompute_eigenvaluesNr2rrrrrrsz&Got empty index range in select_range.r4rkz#Got empty interval in select_range.z-'select must have a value in {'a', 'i', 'v'}.g@>rrsr3)r7cd|zd|zzS)Ng?rv)rr/s r%midpointz@eigh_tridiagonal.._compute_eigenvalues..midpoints+#+. .r_c tjtj|tjtj||z Sr\)r logical_andrcr)rsrr/abs_tolmax_its r%continue_binary_searchzNeigh_tridiagonal.._compute_eigenvalues..continue_binary_searchsC%%mmAv&mmGX%8%8%GHJ Jr_c ||} |}tj| k||}tj| kD||}|dz||fSr|)rrA) rsrr/midcountsrXrErFrKr[rL target_countss r%binary_search_stepzJeigh_tridiagonal.._compute_eigenvalues..binary_search_stepsa&#%&2EsK&//&M"93F%//&="8#uE%Qu$ $r_)$rrr1 is_complexrr rsqrtsquarerrrrrrrJrrrmin reduce_minrrtinynmantrr assert_less_equal assert_lessrrrrr7rr)!rEbetabeta_absroff_diag_abs_row_sumlambda_est_maxlambda_est_mint_normonesafeminassertsfudge norm_slackrr/firstlast target_shaper`rerxrXr^rFrKr_r[rLrdrselect select_rangers!` @@@@@@@@r%_compute_eigenvaluesz.eigh_tridiagonal.._compute_eigenvaluess.` >>/ 0l& ;; ! !--&%MM(--"5"<=']]7+(OOD)'\\$'(334(// bq\8CR=8AB<7"# Ga Q!)) IIx**53G+GHJ!)) IIx**53G+GHJ!! LL ((,,~*FH ggb : :;**S599_sUYY%**.LM8++C1D1DW1MNN&ooeii(1+.EF))f$ $$S'2'q S="..+- s]//1o1o>@'#..a,q/A:MN- s]))1o1o;=' JK K '' 2 .&&u-E . ]]1ekk2U:UYYFO Z  :-E F0BB% :->%q/J.UV1CC%q/J.?%1DeL%w0CUK$"..5-   2   2 }5 &&uLA&&uLA'' ='445H5AC  / J  % %%//0F0B12E50AC5%u%Yl&l&p . .ql&l&s%L'T S; FT;T TTc tjd5tj|}tj|}t j ||j }|dd|ddz }tj|j jj}t jt j|dt j|d}tj||z}t j||} tjdg| gd} tj| dggd} t j t j"| | tj$tj&dt j | t j"| tj$tj&ddztjt j t)j*||fd d g |j } t-| d} | | ddtj.fz } t1j2d| j4| j }t j ||j }|tj.ddf|ddtj.fz }tj6|tj.ddf|dg}||t j8|gfd d}fd}t;j:||d| | |g\}}} }t=|cdddS#1swYyxYw)z5Implements inverse iteration to compute eigenvectors.compute_eigenvectorsrr2NrrFrrL*)r7seed)r7r1cRfd}tjfd|d|g\}}|S)Nc |} |}tjtj||d}|||ddf}t t |\}}t |}tj |||}|dz|fS)Nrr2)rrrrqr transposetensor_scatter_nd_update) cluster_idx eigenvectorsrPendupdate_indicesvectors_in_clusterrCrxvectors_to_updateortho_interval_endortho_interval_starts r%orthogonalize_clusterzxeigh_tridiagonal.._compute_eigenvectors..orthogonalize_close_eigenvectors..orthogonalize_clusters(5E$[1C&22uc*B0N!-eCil!; i 234DAq )! $==n.?AL?L0 0r_c0tj|Sr\rr)rsev num_clusterss r%r^zkeigh_tridiagonal.._compute_eigenvectors..orthogonalize_close_eigenvectors..sHMM!\:r_rr)rrrxrrrs r% orthogonalize_close_eigenvectorszYeigh_tridiagonal.._compute_eigenvectors..orthogonalize_close_eigenvectorss3 1'11:#a%68/!\ r_c Fd}d}tjd|z|j}tjtj ||tj tjtj|tj||zS)NrGg?r2r) rr5r1rr]rc reduce_any greater_equalr )rsrxnrm_v nrm_v_oldr_min_norm_growthnorm_growth_factors r%continue_iterationzKeigh_tridiagonal.._compute_eigenvectors..continue_iterations&/*33/! 6 %%mmAv&!!((mmE*mm$6$BCEFG Gr_ct|ddd}|}t|d}||ddtjfz }|}|dz|||fS)NrT)rrrr2r)rnormrr)rsrkrrrrs r%inverse_iteration_stepzOeigh_tridiagonal.._compute_eigenvectors..inverse_iteration_stepse)## %! )qq/%%9,,,--!.q1!Q5)+ +r_)rrrrrrr1rrrrrrrgrcrr] logical_notrrbrstateless_random_normalrrrr5r7tilerrr)rEroeigvalsrrgaprrtgaptolcloseleft_neighbor_closeright_neighbor_closev0rzero_nrm eigvals_cast alpha_shiftedrrrxrkrrrrrs @@@@@r%_compute_eigenvectorsz/eigh_tridiagonal.._compute_eigenvectorss >>0 1k NN7 # NN5 ! e4::6 abkGCRL(hhw}}33488!! LL $hll72;&?A& c6*'../?aH(//0@qI'33  !4 57K M(00   3 42 ?%11 !5!56J!KM&..   1 2=?@A ~~&89 ]] 8 8!fAr7 ,**Ra  %9,,,- -''EKKP  }}WDJJ? )##Q& ',q):K:K7K*L L ~~d9#4#4a#781a&A}hmmD&9: 6 G ,$../A/E012uh/GI1eQ|Wkkks MM--M6rEr(rr2Nroz+'alpha' and 'beta' must have the same type.)rrr+r7rr r1r) rEro eigvals_onlyr}r~rr#rrr eigvectorsrs ``` @r%r<r<sH ~~d001`_&Bm^  ! !%g 6E AAAv ]]5 !k``l  F 3D {{djj D EE"5$/G {``~'udGrs%3.+4+1+120,(*26,+6  & &   **##  / / 9H99'BC  & & ""  * *  nn&& -666t<=""]]  $ $%%nn      & & 55 ? @  >>:   "24 = qQqQh +3   \24\2~ %& (1$)$)'+',iQ'iQX/>d &' LQ(LQ^"   "N!"NJ = }}@  ]]@ %& <'<~ "# E $E P@4 $% #'"& b&br_