BVhܵ.dZddlZddlZddlZddlZddlmZddlmZddl m Z ddl m Z ddl m Z ddl m Z dd lmZdd lmZdd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlm Z ddl!m"Z"ddl!m#Z#dZ$dZ%dZ&dZ'e jPZ)dZ*dZ+ d;d Z,d!Z-d"Z. dd3Z?d=d4Z@d5ZAd6ZBd7ZCd8ZDd9ZEd:ZFy)>aControl flow statements: loops, conditionals, etc. Note: most of these operators accept pairs of get_state/set_state functions, to capture mutations that the corresponding code blocks might make. These mutations only need to be captured when staging the control flow, and they just work when reverting to Python behavior. __Examples__ ``` while cond: self.x += i ``` When the functionalized version is executed as a Python loop, it just works: ``` def loop_body(): self.x += i # works as expected for Python loops ``` But it won't work for TF loops: ``` def loop_body(): self.x += i # self.x has the wrong value! ``` get_state/set_state allow piping the mutations through the loop variables as well, in effect changing the loop body: ``` def loop_body(self_x): self.x = self_x # self.x now has the proper value self.x += i # the original block self_x = self.x # write self.x back into the loop vars return self_x self_x = tf.while_loop(...) self.x = self_x # the result is not properly captured ``` N) py_builtins) variables) ag_logging)misc)tensors) type_registry)dtypes) errors_impl) func_graph)ops)tensor_conversion) tensor_shape) tensor_util) array_ops)cond)control_flow_assert)control_flow_util)math_ops)tensor_array_ops) while_loop) distribute)nest)variable_utilsiTiPcz|duxs6t|tjxst|tjS)zTests whether a value is None or undefined. AutoGraph represents undefined symbols using special objects of type Undefined or UndefinedReturnValue. Args: value: value to test Returns: Boolean N) isinstancerUndefinedReturnValue Undefined)values b/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/autograph/operators/control_flow.py_is_none_or_undefr!fs: D= 4 y== > 4 y22 35c>d}tj|}|jtjk7rt dj ||||j|jjtj|d}|S|jjdkDrt|jjDcgc]}|| }}tj|dkDrt dj |||tj|d}|Scc}w)zz%verify_tf_loop_vars..r"Texpand_compositeszE'{}' does not have the same nested structure after one iteration. {}NFzV'{}' does not have the same nested structure as its corresponding shape invariant. {}) r map_structurer8rangeassert_same_structurer(rTrconvert_variables_to_tensorsr) functoolspartialr])r=iter_entry_varsiter_exit_varsr>opts check_shapesr_irArVrWrX invariant init_tensorses r verify_tf_loop_varsru sF(D0./))./J \ c"23 33 3 \ c)n ,, , \ c/2 22 2 \ c.1 11 1 \" #$!a ?D Q 55h?H T3sBJJ?@"77 CJ   *   ,  (G $ *g &  ^^z'''  &x~~22#))..0 11(r"ct||D]n\}}t|tjrt dj ||t|tj sVt dj |y)zBVerifies variables output by a conditional branch for consistency.z.'{}' must also be initialized in the {} branchz0the {} branch must also have a return statement.N)r9rrrr(r)r)vars_r> branch_namerAvar_s r _verify_tf_cond_branch_varsr~bs| e,jdD$ ++,  : A AK ! ""$ 667  < C C   r"c t|||}|D]N\}}} tj||dtjtjt|||Py#ttf$r t j |}t j |}tj||dn<#ttf$r*} t dj|t| | d} ~ wwxYwYwxYw)z@Verifies variables manipulated by a conditional for consistency.TrezK'{}' must have the same nested structure in the main and else branches: {}N) r9rrir(rTrrjr)rPrgrkrlry) body_vars orelse_varsr> named_varsrArwrxbody_var_tensorsorelse_var_tensorsrts r _verify_tf_cond_varsros<K8*$.O dHj<   :N 0$7:O#O  " < <)FFxP+HH  ""#35G59 ;) $< %vdCF35:; << ; rofor_fns r for_stmtrsR1  % %e ,F  D)Y dK  1 Fe$   '#' E:.. /$f E:.. /0f 1s&BB10B1c ~~~~ t}|j |j  | fd}|}&fd} | r|D]} || | ryyy|D] } ||  y)z5Overload of for_stmt that executes a Python for loop.c2|yr6r$)protected_iterafter_iterationbefore_iteration original_bodys r protected_bodyz$_py_for_stmt..protected_bodysN#r"Nc} t|S#tj$r)}tjdddt d|d}~wwxYw)Nrz1Caught error while evaluating loop stop conditionTexc_infoa+break and return statements which depend on a TF condition are not supported in Python for loops. Did you intend to make it a TF loop? See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/autograph/g3doc/reference/limitations.md#consistency-of-control-flow-types for more info.r'r OperatorNotAllowedInGraphErrorrlogr<)extra_test_resultrtrs r guarded_extra_testz(_py_for_stmt..guarded_extra_testsj$,H%&&  7 7 H ?  " @A GH  H H A$A  A_PythonLoopCheckerrr)rrrrrr>rocheckerrrtargetrrrs ` @@@r rrsL$ "G//--OM DH(& V !#  6lr"c^tjtjr||d<yy)Nmaximum_iterations)rGraphOrParentsInXlaContextr get_default_graph)rons r _add_max_iterations_hintrs*11#2G2G2IJ!"D Kr"c tj tjj }|j d  fd} fd}  fd}  fd} t | t| | || d|z|y) zHOverload of for_stmt that iterates over TF entities that admit a length.)sizercfzSr6r$)r iterate_indexsr aug_get_statez-_known_len_tf_for_stmt..aug_get_state s  ik ))r"c|^}|yr6r$) aug_loop_vars loop_varsrrs r aug_set_statez-_known_len_tf_for_stmt..aug_set_state s!.MIir"c>jdz yNr)read)rrrsr aug_bodyz(_known_len_tf_for_stmt..aug_bodysM "#QMr"cFk}tj|dS|S)NcyNFr$r$r"r rcz:_known_len_tf_for_stmt..aug_test..rdr"tf_condr) main_testrrrs r aug_testz(_known_len_tf_for_stmt..aug_tests,!I \\)Z ?? r"zN)rlen_r TensorArrayr&unstackr_tf_while_stmt)rrrrrr>rotarrrrrrs````` @@r rrsu! ##EKKa8" **U %-*  4# , r"c @ |jj\} |fd fd}fd}  fd} fd} ttjt j | tjt| | || dzy)zCOverload of for_stmt that iterates over a TF range (and elides it).cP|dk(rt|tjr|S|S)N iterate_names)rrr)rAvardefaultros r _value_orz%_tf_range_for_stmt.._value_or3s) _%%*S):M:M*N n Jr"c\}tfdt|D}f|zS)Nc38K|]\}}||ywr6r$).0rArriterates r z<_tf_range_for_stmt..aug_get_state..:s&8 D# $W%8s)tupler9) state_varsrrrr>s r rz)_tf_range_for_stmt..aug_get_state8s7J8\:688J : ""r"c|^}|yr6r$)rrrrs r rz)_tf_range_for_stmt..aug_set_state?s(Giir"c z yr6r$)rdeltarsr rz$_tf_range_for_stmt..aug_bodyGsM uGr"c tj}||dk\rk}nPkD}nJtjtjdk\ktjdkkD}t j |d}|S)Nrcyrr$r$r"r rcz6_tf_range_for_stmt..aug_test..Zrdr")rconstant_valuer logical_or logical_andrr) const_deltarrrrlimits r rz$_tf_range_for_stmt..aug_testLs,,U3K  eO eO %%   uz7U? ;   uqy'E/ :rostartrrrrrrrrs `````` @@@@r rr,s% ' # " mmD&&ueU;V\\JL ,  r"c dzd  fd fd  t   fd} fd}t|| y)zCOverload of for_stmt that iterates over TF Iterators. See for_loop.)zTcfzSr6r$)rhas_nextsr rz,_tf_iterator_for_stmt..aug_get_stateps ; $$r"c|^}|yr6r$)rrrrs r rz,_tf_iterator_for_stmt..aug_set_statess)Hy ir"c jj fd}fd}tj||y)z#Main body passed to _tf_while_stmt.c`j}t||Sr6) get_valueru) new_loop_varsrrr=r opt_iterateror>s r main_pathz:_tf_iterator_for_stmt..aug_body..main_paths7 ; "##om Y |TC r"cSr6r$)rsr noop_pathz:_tf_iterator_for_stmt..aug_body..noop_paths r"N)get_next_as_optional has_valuerr) rrrrrrrrr=rror>s @@r rz'_tf_iterator_for_stmt..aug_body|sQ,,.K$$&HI  Xy)46r"c@}tj|dS|S)Ncyrr$r$r"r rcz9_tf_iterator_for_stmt..aug_test..rdr"r)rrrs r rz'_tf_iterator_for_stmt..aug_tests)I \\)Z ?? r"N)rEr) rrrrrr>rorrrrrr=s ``````` @@@@r rrjs]*L8, (% o) <0664  r"c| tdtdvrtdd<fd}|j|y)z@Overload of for_stmt that iterates over TF distributed datasets.NzXbreak and return statements are not yet supported in for ... in distributed input loops.r_cT||}t|||Sr6ru) rrrrrr=rorr>s r reduce_bodyz6_tf_distributed_iterable_for_stmt..reduce_bodys3 iMKM9m\4A r")r<rE,_shape_invariants_mapping_to_positional_listreduce) rrrrrr>rorr=s ````` @r rrsr  . //k) <04K  ) -D  ELLK 01r"ctjdj5|}dddtjrt ||||||y|sy|t |||||y#1swYIxYw)aFunctional form of a while statement. The loop operates on a so-called state, which includes all symbols that are variant across loop iterations. In what follows we refer to state as either a tuple of entities that represent an actual state, or a list of arguments of the corresponding types. The inputs and outputs of the callables representing the loop blocks are not explicit - instead, these functions must use nonlocal/global for side effects. The inputs and outputs are instead controlled by the set_state/get_state functions. Args: test: Callable with boolean return type. The loop condition. body: Callable representing the actual loop body. get_state: Additional callable which can capture additional state (such as the values of composite symbols). This is only useful when staging the loop. set_state: Additional callable which save values captured by get_state back into the Python environment. This is only useful when staging the loop. symbol_names: Tuple containing the names of all loop variables. opts: Optional dict of extra loop parameters. Returns: Tuple containing the final state. tmpN)r FuncGraph as_defaultris_dense_tensorr_py_while_stmt)testrrrr>ro init_tests r while_stmtrsy>E"--/I  Y'4y)\4H  &tY 48!s A44A=c>eZdZdZdZdZdZdZdZdZ dZ d Z y ) rz-Verifies Python loops for TF-specific limits.) iterationscheck_inefficient_unrollcheck_op_count_after_iterationops_before_iterationc6d|_t|_d|_y)NrF)r WARN_INEFFICIENT_UNROLLr r selfs r __init__z_PythonLoopChecker.__init__sDO$;D!+0D'r"cXttjjSr6)setr rget_operationsrs r _get_opsz_PythonLoopChecker._get_opss s$$&557 88r"c@|jtkDr tdy)Nziteration limit exceeded)r PYTHON_MAX_ITERATIONSr(rs r _check_unroll_limitsz'_PythonLoopChecker._check_unroll_limits s  .. 1 22/r"c.d|_d|_d|_yr)r r r rs r !_stop_checking_inefficient_unrollz4_PythonLoopChecker._stop_checking_inefficient_unroll s$)D!*/D' $Dr"c jJj}tfd|D}t|tkryt j dj|djtjy)zAChecks for possibly-inefficient creation of ops in a Python loop.c3@K|]}|jvs|ywr6)r )rrrs r rz@_PythonLoopChecker._verify_inefficient_unroll..s%QBd6O6O,OQsFaLarge unrolled loop detected. Did you mean to use a TF loop? The following ops were created after iteration %s: %s See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/autograph/g3doc/reference/common_errors.md#warning-large-unrolled-loop-detected Location: %sr7T) r rrr8INEFFICIENT_UNROLL_MIN_OPSrwarningr join traceback format_stack)rops_after_iterationnew_opss` r _verify_inefficient_unrollz-_PythonLoopChecker._verify_inefficient_unrolls  $ $ 00 0--/Q(QQG 7|00   OOWdii 0F0F0H&I K r"c~|jr1|jtkDr|j|_d|_yyy)z.Called before each iteration in a Python loop.TN)r r !INEFFICIENT_UNROLL_MIN_ITERATIONSrr r rs r rz#_PythonLoopChecker.before_iteration)s8 %% ;;"&--/d,0d) < &r"c|xjdz c_|j|jrK|j}|r|j y|jt dzkDr|j yyy)z-Called after each iteration in a Python loop.rN)r rr r%rr')rdid_warns r rz"_PythonLoopChecker.after_iteration0skOOqO **002h  ..0 ??>B B ..0 C +r"N) __name__ __module__ __qualname____doc__ __slots__rrrrr%rrr$r"r rrs/5)093% .1 1r"rc ~~~ t}|j |j |  fd}|}fd}|r||ryy)z9Overload of while_stmt that executes a Python while loop.c0yr6r$)rrrsr rz&_py_while_stmt..protected_bodyJsor"c} t|S#tj$r)}tjdddt d|d}~wwxYw)Nrz2Caught error while evaluating while loop conditionTraThe condition of while loop started as non-Tensor, then changed to Tensor. This may happen either because variables changed type, or when a break or return statement inside the loop depends on a Tensor condition. In both cases, changing to a TF loop should remove the error. See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/autograph/g3doc/reference/limitations.md#consistency-of-control-flow-types for more info.r) test_resultrtrs r guarded_testz$_py_while_stmt..guarded_testPsf&KF+   5 5Fnn >  > ?EFF FrNr) rrrrrorrr4rrrs ` @@@r rr?s\ Iy "G//--OM DF* F r"c |Dcic]\}}t|||f}}}g}|D]L}|jt|d\}}|j||ur|ntjd|Nt |Scc}}w)NNNcyr6r$ras r rcz>_shape_invariants_mapping_to_positional_list..prdr")idgetappendrrgr)mappingkeyskvresultmap_keymap_vals r rris'. /tq!RUQF] /' / & La{{2a5,7GW MMa<T%7%7%JLL v 0sA?z9Tensor, int, float, bool or a list, tuple or dict thereofcX||dfSt|tjtjfr|dfSt|tt t frt|ddfStj|rE||n |j}| |j"tjd|j|fSg}d}t|j|D]\}}|d}n8t|t j"r|j$d}n|j$}n|}||j'|d}Yt|t j"r<|j$|j'|d}|j'|j$|j'||r|} nd} tj||j| fSt|t(t*t,fr|+t/j0dt/j2|} nAt/j0t4t/j2|t/j2|} t| \} } t/j6|| t/j6|| fSt9dj;t|j<|)aConstructs a (dummy) placeholder value for a loop-initialized variable. Args: like: Any object. The value created by the first iteration of the loop. If a Python scalar, the placeholder will be the zero value of that type. If a Tensor, the placeholder will be a zero tensor of matching shape and dtype. If a list, dict or tuple, the placeholder will be an identical structure of placeholders. shape_invariant: The shape invariant specified by the user (or None, if nothing was specified) for the respective variable. original: Any object. The value of the variable prior to entering the loop. Typically, this is one of the special "Undefined" value, because that's when a placeholder is needed. Returns: Either a zero value of structure, shape and dtype matching 'like', or 'original', if no such zero value could be created. Nrr$FTct|dSr6_placeholder_valuer>s r rcz$_placeholder_value..s,>q$,Gr"zFound an unsupported type '{}' while creating placeholder for {}. Supported types include Tensor, int, float, bool, list, tuple or dict.)rrrrrNrOr'r;rrRr*rankrzerosr&r9r Dimensionrr:listrdictrrgflattenrEpack_sequence_asrTr)r+) likerYoriginal like_shapeplaceholder_shapehas_dynamic_dimssrqlike_dimrrzippedvalsinvarss r rErEzsa& \ T>$,,i.L.LMN T>$eT*+ 4:a=$ d#$3$?TZZJZ__4 __R ,j 88DJJ +$1  a// 0 77?(WW(   * a// 0 77?  " "8 ,!   " "177 +  #+$.ii ??,djj 99 DD$ud+,!!"G"&,,t"46f!!"4dll46H"&,,"?Af)>tV)L NN P vd4j!!4( **r"c pd}d}d} tjdj5d} tj| |} || d}||}dddt dt ||||D} t | \}} d}|r || t#|||| | fS#1swYSxYw#ttttf$rQtjddd tj}d j|d j |d} YwxYw#|r ||wwxYw) apMakes a best-effort attempt to substitute undefineds with placeholders. Note: this substitution requires two things to happen: 1. the types of loop variables could be inferred (usually by staging one iteration) 2. these types could be replaced by placeholders (e.g. zero values, for tensors). Args: body: a function representing the loop body. See while_stmt. get_state: state getter for the loop statement. See while_stmt. set_state: state getter for the loop statement. See while_stmt. init_vars: loop variables before entering the loop. See while_stmt. nulls: list of boolean flags indicating whether the corresponding loop var is None or undefined. shape_invariants: user-specified shape invariant for each loop variable. symbol_names: list of loop variable names. See while_stmt. Returns: A tuple (success, new_init_vars, extra_shape_invariants, failure_message): * success is a boolean flag indicating whether types could be successfully inferred (step 1 above) * new_init_vars contains the loop vars, with None or undefined values replaced by default values, where possible (step 2 above) * extra_shape_invariants contains shape invariants that would be needed by while_stmt, for instance if the placeholder values had a shape different from the corresponding loop outputs FNrc t|ttttt t tjtjfr?tj|}tj|j|jS|Sr6)rrNrOr'rPrJrr.rQgenericr r%r placeholderr&r*)r>init_vals r autocast_to_tensorz4_try_handling_undefineds..autocast_to_tensorsY  UD#tUBJJ K M&;;A>(&&x~~x~~F Fr"Tc3LK|]\}}}}|r t|||n|dfywr6rD)rr>rivrqs r rz+_try_handling_undefineds.. s5!2 Aq"a*+ B1 %D 9!2s"$rz$Caught error while staging loop bodyrzGNote: AutoGraph tried to define it automatically, but ran into a {}: {}r)r@)r rrrrgrr9UnboundLocalErrorrTr(KeyErrorrrsysrr)r+rE)rrrr=nullsr_r>state_modifiedr?failure_messager]autocast_init_varsinits_and_invariantsextra_shape_invariantssuccessexcs r _try_handling_undefinedsrksW<.//*   e $ / / 1$   --.@)L"#n f! o#$,!!2y%/1!22),-A(B%I%G oO )3 33]$$: Y H =3NN1rcr= var_namessnrr>s r _runtime_zero_iterations_errmsgro%sy)lE95hb!Q !Y33423r ii ") ; B B9 MMr"c }tdD}t|rYjddD cic]\}} t|| c} }tfd|D} t || \} ndrst t | D]2\}} } jt|| }|%|t| <4tfdDrt did<fd}fd }dvrtdd<t }|jd dd |d <rd z}d|vrd|dz|d<n}tj|||fi|}r\tjtj|dt||gg5tj d|dd}ddd|ycc} }w#1swYxYw)z3Overload of while_stmt that stages a TF while_stmt.c32K|]}t|ywr6)r!)rr>s r rz!_tf_while_stmt..8s8!!$8sr_r$c3TK|]}jt|d!ywr6)r9r8)rr>shape_invars_by_init_valss r rz!_tf_while_stmt..=s*L78!%%beT2Ls%(FNc3XK|]!}t|vr|t|f#ywr6)r8)rnvmerged_shape_invariantss r rz!_tf_while_stmt..Qs7$J(*')"v1H'H&()@B)H$I$Js'*cDr|dd}|tdS)Nrz while loop)r4)rrequire_one_iterationrrs r rz _tf_while_stmt..aug_testXs(AB-i i  55r"cnr|dd}|}t||rd|z}|S)NrTr) rrrrr=rorxrr>s r rz _tf_while_stmt..aug_body_sNAB-i iFKM9m\4A -m r"rTreturn_same_structure)Fr6rcZtj|rtj|S|Sr6)rrRridentityrFs r rcz _tf_while_stmt..s"k.D.DQ.GY''*Qr"r)ranyr9r8rkrKr9rpoprr control_dependenciesrAssertrorrg)rrrrr>roorig_init_varsrcr>rqr_rhrunimerged_invariantrrwhile_loop_opts aug_init_varsfinal_loop_varsr=rvrxrss`````` @@@@r rr3sjk). 8i8 8%Z!XX&8"=!Q1q!L 0: x = -x ``` The state is represented by the variable `x`. The `body, `orelse` and `set_state` functions must bind to the original `x` symbol, using `nonlocal`. The inputs and outputs of the callables representing the loop blocks are not explicit - instead, these functions must use nonlocal/global for side effects. The inputs and outputs are instead controlled by the set_state/get_state functions. Args: cond: Boolean. body: Callable representing the main block of the conditional. orelse: Callable representing the else block of the conditional. get_state: Function that returns a tuple containing the values of all composite symbols modified within the conditional. This allows access to state that branches may mutate through side effects. This function is not needed and should not be called when dispatching to code matching Python's default semantics. This is useful for checkpointing to avoid unintended side-effects when staging requires evaluating all code-paths. set_state: Function to set the values of all composite symbols modified within the conditional. This is the complement to get_state, used to restore checkpointed values. The single argument a tuple containing values for each composite symbol that may be modified in a branch of the conditional. The is usually the result of a call to get_state. symbol_names: Tuple containing basic loop var names. nouts: Number of variables output by the statement. Vars which are not outputs will not be passed through staged control flow such as tf.cond. This includes variables that are defined before the conditional, but are not used after it. N)rr _tf_if_stmt _py_if_stmt)rrorelserrr>noutss r if_stmtrs3X T"dFIy,NdF#r"c t|d}sc  fdfddz d dg dg  fd} fd}tj|||d } | dz} | y) z*Overload of if_stmt that stages a TF cond.z if statementcdzS)N)rr$)prev_get_statesr rcz_tf_if_stmt..s~//r"c|ddSrr$)r>prev_set_states r rcz_tf_if_stmt..s.12/r")zrNc}|d}|d<t|ddt|d|S)Nrmainr~r) new_body_varsrrr=new_body_vars_new_orelse_vars_rrr>s r rz_tf_if_stmt..aug_bodys\ iFKM!&5)M%N1 |VD&=*:1*=|L r"c}|d}|d<t|ddtd||S)Nrelser) new_orelse_varsrr=rrrrrr>s r aug_orelsez_tf_if_stmt..aug_orelses\ i HkO%fu-O)QvFa$>!,o|L r"T)strict)r4rr)rrrrrr>rrrfinal_cond_varsr=rrrrs `````` @@@@@r rrs dN 3$ %. "NN/I/I''L Ek)6.V    LL Hj//#i&77/ Or"c"|r|S|S)z8Overload of if_stmt that executes a Python if statement.r$)rrrs r rrs%VX%r"r6rzr6)Gr.rkrbr!numpyr.%tensorflow.python.autograph.operatorsrr!tensorflow.python.autograph.utilsrrrrtensorflow.python.frameworkr r r r r rrtensorflow.python.opsrrrrrrrrtensorflow.python.typesrtensorflow.python.utilrrrrr'r TypeRegistryrr!r4rErLr]ruryr~rrrrrrrrrobjectrrrr:rErkrorrrrr$r"r rsF)V =;825;.32+943+153*2,.'1"$)!/M..05"8BF+"\ "6 ~ 7!t1< O29Lx0f# ,^; |; |26/9dG1G1T' TOS*lR4j N\~/$d-`&r"