AVhGdZddlmZddlmZddlmZddlmZddlmZddlm Z ddl m Z dd l m Z dd l mZdd lmZdd lmZdd lmZddlmZddlmZddlmZdadZej4Zej6Zej8ZdZdZdZdZ dZ!d&dZ"dZ#dZ$ejJdZ&dZ'dZ(dZ)dZ*d Z+d!Z,d"Z-ed#g$d%Z.y)'zUtilities for V2 control flow.)attr_value_pb2)context)atomic_function)concrete_function)tracing_compilation) transform)function_def_to_graph)ops) FuncGraph)control_flow_util)control_flow_v2_func_graphs)gradients_util) keras_deps) tf_contextlib) tf_exportNFcRtjrytj}t |t s t |t st |tr?|j}t |t rt |t r.t |tr?t |tS)z:Returns if the current graph is, or is nested in, a defun.F) rexecuting_eagerlyr get_default_graph isinstanceCondBranchFuncGraphWhileBodyFuncGraphWhileCondFuncGraph outer_graphr graphs Z/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/ops/control_flow_util_v2.pyin_defunr+sz     !% E. / E- . E- .   E E. / E- . E- . E9 %%cptjryt|txst|tS)z/Returns if the graph is a while loop FuncGraph.F)rrrrrrs rin_while_loop_defunr 7s0   U. / 0 U. /1rctj|tj|j|i}|j j ||jS)zConverts func_graph to a TF_Function and adds it to the current graph. Args: func_graph: FuncGraph Returns: The name of the new TF_Function. )rapply_func_graph_transformsrfrom_func_graphnamer_add_function_recursive) func_graphfuncs rcreate_new_tf_functionr(>sH '' 3  ( (*b I$ 006 rcX||dtjjddS)zReturns a unique name to use for a control flow function. Args: scope: A name scope string. name: An identifier for this function (e.g. "true", "body"). Returns: A string, the name to use for the function. _/)r uidreplace)scoper$s runique_fn_namer/Ns$dCGGI . 7 7S AArc4|dtjS)N_grad_)r r,) forward_names runique_grad_fn_namer3[s%swwy 11rcH|'|jdtj|ytsst j |j sStjjjdk7r'|jdtjdyyyy)a7Sets the flag to enable lowering on `op` if necessary. Lowering allows cond_v2 and while_v2 to avoid some of the limitations of Functions, allowing users to specify devices & colocation inside of cond_v2 and while_v2 input functions, and enabling non-strict evaluation & partial pruning. This brings v2 control flow closer to feature parity with v1 control flow. However, we do not lower in the following cases: - When the `If` or `While` ops are in the XLA context. Because it is easier for XLA to apply its own optimizations when dealing with un-lowered control flow operators than with low-level control flow primitives. - When the eager execution context specifies the executor of functions to be the single threaded executor (see context.function_executor_type()). Because the single threaded executor does not support v1 control flow ops. - When 'lower_using_switch_merge' is explicitly set to False. Args: op: An `If` or `While` Operation. lower_using_switch_merge: Explicit value to lower or not (optional). N_lower_using_switch_mergebSINGLE_THREADED_EXECUTORT) _set_attrr AttrValue!_DISABLE_LOWER_USING_SWITCH_MERGEr GraphOrParentsInXlaContextrrfunction_call_options executor_type)oplower_using_switch_merges rmaybe_set_lowering_attrrA_s,)LL,)),DEG .  8 8 B//==" #LL,n.F.F.NO # C .rctj|jr'|jdt j dyy)agTells XLA whether to propagate compile-time consts in the loop body. This is needed to make compile time constants available to ops, for example `max_num_elements` in `EmptyTensorList`, inside the loop body. Ideally this would always be turned on, but that doesn't work with legacy functionalized while_loops. Args: op: A `While` Operation. "_xla_propagate_compile_time_constsTr6N)r r<rr9rr:)r?s r*maybe_propagate_compile_time_consts_in_xlarDs811"((;LL5))D13 NodeDef for every op in the function. functions: a dict mapping function name -> AtomicFunction. Returns: The index into input_names corresponding to `tensor_name`. :rz%s:%dc j|jj}|j}|jj j}|j |}t||jjDcgc]}|jc}|jDcic]}|j|c}Scc}wcc}wN) attrr'r$cached_definition signature output_argretresource_input_index input_argnode_def) function_attribute_name func_namefdefoutput_arg_nameoutput_tensor_nameargndef functionsrS output_idxs r_extract_input_indexz2resource_input_index.._extract_input_indexs-- 78==BBi y ! 3 3d11*=BBo88O4 ! 4>>3K3KLCsxxL'+}} 5t499d? 5yBBL 5s C 'C )IdentityWhilePartitionedCallStatefulPartitionedCallf)If StatelessIf then_branch else_branchzXExpected cond branches ({} op) to each have the same input->output mapping of resources.zWTaking gradient of a while loop which creates a resource in its body is not supported: z ()) splitlenintr?inputAssertionErrorformat ValueErrorindex) tensor_name input_names node_defsr[partsop_namer*r] input_indexrSr\s ` @@rrQrQsv0 ;&   c "E 5zQ$gq* Uq!gz Z1__agj{J77k  # ZJ!HB{{++NN:.k F F NN#7#<=k - -(7k ,]; ;34:F8;;4GI INN /k !8;;0 11i ;&p   ; ''rc#Ktjj}tjd5tjj |ddddy#1swYyxYww)aClears the control inputs but preserves the ControlFlowContext. This is needed to preserve the XLAControlFlowControl when clearing control inputs for the gradient accumulators in while_v2. `ops.control_dependencies` does not allow that. Yields: A context manager in which the ops created will not have any control inputs by default but the control flow context is the same. N)r r_get_control_flow_contextcontrol_dependencies_set_control_flow_context)control_flow_contexts rclear_control_inputsr|s]..0JJL % 556JK    s8A7(A+" A7+A40A7cX|duxr%|jjjdS)N TPUStrategy) __class____name__ startswith)strategys r_is_tpu_strategyrs0 $  @    % % 0 0 ?ArcVtj}|r|jduSy)NF)rget_call_context_functionlayer)keras_call_context_functions r_is_building_keras_layerrs+ * D D F & ( . .d :: rcttStrytjt j ryt j jjdk(rytS)aWhether to output all intermediates of a functional control flow op. The default behavior is to output intermediates only when building a Keras Layer in graph mode and that too when certain other conditions are met: 1. We do not output intermediates if the functional control flow op is being built inside a FuncGraph which is not a If/While graph. This guards against outputting intermediates in eager mode since keras adds tensors to a FuncGraph named "keras_graph" in that case. Also because we do not output intermediates of tf.function (since this feature is only for backwards compatibility) outputting intermediates of functional control flow ops built inside tf.function is of no value. 2. We do not output intermediates when the compilation is using XLA or for a TPU. 3. We do not output intermediates when a single threaded executor is used since that does not perform inlining and pruning. Returns: A bool telling whether to output all intermediates. Fr8) /_EXPERIMENTAL_OUTPUT_ALL_INTERMEDIATES_OVERRIDErr r<r rrr=r>rrroutput_all_intermediatesr s\(5@ :: Z 11#2G2G2IJ  oo--;; !  ! ##rcd}|j}|=|j|}| |j}nt|dr |j}nn|=|t d|z|jj 5tj||}dddjD]}|jdvs|j|jdj} tjj|j|j|jj } | j%|_|S#1swYxYw#t"$rYwxYw)zDGenerates and returns a FuncGraph for the given op and input_shapes.Nrz%s cannot be found in the graph) input_shapesr`rc)attrs)r _get_functionrMhasattrrKeyError as_defaultr get_operationstypeget_attrr$rConcreteFunctionr# function_typerLAttributeError_get_gradient_function_gradient_function) r?rrUrVrr'r& operationrccfs rget_func_graphr*sd $ ((%    y )D   # #d um$e   \ 4y@ AA xx)&<< <)J),,. Ai~~GG   i005:: ;a  / / ? ? GG OO%%**@ &(%>%>%@i" A 1))&s8E%A EE E E clt|tjr|gfS|sdgfS|dj|fS)Nr)rr Operationr?) op_or_outputss rget_op_and_outputsr]s= s}}- "   8O     --rcH| dt|ddvryt|dd}| y)zDCheck if `graph` is wrapped by `run_as_function_for_tape_gradients`.Ncflow_gradient_wrapperr$TrF)getattrrs r-graph_wrapped_for_higher_order_tape_gradientsrfs7 75&"#==  E=$ /E  rc:tj|tjk(rntjj r dtjj vs0tj|ftj|dd}|S||S)aFix higher-order tape gradients by wrapping `make_op` in a function. Args: make_op: A function that takes a list of inputs and returns a list of output tensors. This function should set any handle data relevant to its outputs before returning. inputs: A list of tensors to check for tape gradients and pass to `make_op`. These should include all tensors used in `make_op`. Returns: Tensors corresponding to `make_op`'s output. rF) autograph)tracing_options) rPossibleTapeGradientTypes$POSSIBLE_GRADIENT_TYPES_HIGHER_ORDERr rbuilding_functionr$r call_functionTracingOptions)make_opinputsresultss r"run_as_function_for_tape_gradientsros(..v6  < <=$$&88+s/D/D/F/K/KK!//  +:: - G N 6?rz%experimental.output_all_intermediates)v1c|ay)aWhether to output all intermediates from functional control flow ops. The "default" behavior to is to output all intermediates when using v2 control flow inside Keras models in graph mode. This is needed to support taking gradients of v2 control flow. In graph mode, Keras can sometimes freeze the forward graph before the gradient computation which does not work for v2 control flow since it requires updating the forward ops to output the needed intermediates. We work around this by proactively outputting the needed intermediates when building the forward pass itself. Ideally any such extra tensors should be pruned out at runtime. However, if for any reason this doesn't work for you or if you have an inference-only model you can turn this behavior off using `tf.compat.v1.experimental.output_all_intermediates(False)`. If with the default behavior you are still seeing errors of the form "Connecting to invalid output X of source node Y which has Z outputs" try setting `tf.compat.v1.experimental.output_all_intermediates(True)` and please file an issue at https://github.com/tensorflow/tensorflow/issues. Args: state: True, False or None. None restores the default behavior. N)r)states rset_output_all_intermediatesrs 25:1rrK)/__doc__tensorflow.core.frameworkrtensorflow.python.eagerr,tensorflow.python.eager.polymorphic_functionrrrrtensorflow.python.frameworkr r &tensorflow.python.framework.func_graphr tensorflow.python.opsr r rtensorflow.python.utilrr tensorflow.python.util.tf_exportrrr;rrrrr r(r/r3rArDrQcontextmanagerr|rrrrrrrrrrrrs %4+HJLB=+<3=0-0626/$)!2EE0CC0CC &1  B2 PH3$P(f  &A $@0f."J 678:9:r