AVhYdZddlZddlmZddlmZddlmZddlmZddl m Z ddl m Z dd l m Z dd lmZdd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZdZdZ e!dgZ"e!gdZ#ededddd'dZ$GddejJZ&d'dZ'd Z(d!Z)d"Z*ejVd#Z,Gd$d%e-Z.d&Z/y)(z>xla is an experimental library that provides XLA support APIs.N)xla_ops) xla_ops_grad)attr_value_pb2)summary_op_util)context) def_function)ops) array_ops)control_flow_ops)variable_scope) tf_logging)compat)nest) tf_inspect)collections_abc) deprecated) tf_export_xla_compile_id Placeholder) AudioSummaryAudioSummaryV2HistogramSummary ImageSummary MergeSummaryPrint ScalarSummary TensorSummaryTensorSummaryV2zxla.experimental.compilezXxla.experimental.compile is deprecated. Consider using `@tf.function(jit_compile=True)`.T) warn_oncectjr tjfd}|St S)aBuilds an operator that compiles and runs `computation` with XLA. NOTE: In eager mode, `computation` will have `@tf.function` semantics. Args: computation: A Python function that builds a computation to apply to the input. If the function takes n inputs, 'inputs' should be a list of n `Tensor`s. `computation` may return a list of `Tensor`s and `Operation`s. `Tensor`s must come before `Operation`s in the returned list. All `Operation`s returned from `computation` will be executed when evaluating any of the returned output tensors. inputs: A list of inputs or `None` (equivalent to an empty list). Each input can be a nested structure containing values that can be converted to `Tensor`s. Note that passing an N-dimension list of compatible values will result in an N-dimension list of scalar `Tensor`s rather than a single Rank-N `Tensor`. If you need a different behavior, convert parts of `inputs` to `Tensor`s with `tf.convert_to_tensor`. Returns: List of `Tensor`s corresponding to the `Tensor`s from the output of `computation` i.e. the same return value as if computation(*inputs) is called directly, with the following exceptions: * None output: a NoOp would be returned with a control dependency on `computation`. * Single value output: a tuple containing the value would be returned. * Operation-only outputs: a NoOp would be returned with a control dependency on `computation`. TODO(b/121383831): Investigate into removing these special cases. Raises: RuntimeError: When eager execution is enabled. Known issues: When a tf.random operation is built with XLA, the implementation doesn't pass the user provided seed to the XLA compiler. As such, the XLA compiler generates a random number and uses it as a seed when compiling the operation. This implementation causes a violation of the Tensorflow defined semantics in two aspects. First, changing the value of the user defined seed doesn't change the numbers generated by the operation. Second, when a seed is not specified, running the program multiple times will generate the same numbers. ctSN)_compile_internal) computationinputssR/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/compiler/xla/xla.pyxla_compile_wrapperz$compile..xla_compile_wrapperss {F 33)rexecuting_eagerlyrfunctionr$)r%r&r(s`` r'compiler,>sBf  44   ; //r)ceZdZdZfdZdZdejfdZdejfdZ dZ dejfdZ e d Z e d ZxZS) XLACompileContextaA `ControlFlowContext` for nodes inside an XLA computation cluster. THIS IS ONLY FOR TENSORFLOW INTERNAL IMPLEMENTATION, DO NO USE DIRECTLY. The primary role of `XLACompileContext` is to mark operators inside a xla.compile() computation with attribute "_xla_compile_id=XYZ", where XYZ is a unique name. `ControlFlowContext` is used to perform the annotation since it integrates with Tensorflow constructs like ResourceVariables. For example, if a `ResourceVariable` is constructed inside a xla.compile() block, the `ResourceVariable` implementation can use `with ops.control_dependencies(None)` to build the variable's definition outside the compiled computation. ctt| ||_t j ||_g|_||_y)aBuilds a new XLACompileContext. Args: name: a unique name for the context, used to populate the `_xla_compile_id` attribute. pivot: a pivot node. Nodes in the XLACompileContext that do not have any inputs will have a control dependency on the pivot node. This ensures that nodes are correctly included in any enclosing control flow contexts. N) superr.__init___nameras_bytes_name_as_bytes_unsupported_ops_pivot)selfnamepivot __class__s r'r1zXLACompileContext.__init__s: T+-DJ //$/DDDKr)c |jrdj|jdtDcgc]}d|jd|jd!c}}t j dt|j|t|jtkDr1t j dt|jtz yyycc}w)N z z ()z$%d unsupported operations found: %sz... and %d more)r5join_MAX_WARNING_LINEStyper8loggingwarninglen)r7opop_strs r'report_unsupported_operationsz/XLACompileContext.report_unsupported_operationss yy))*=+=>"'' * foo=$//0&: T " " #&8 8)D1125GG I 9s$CrDcg}g}|jD]R}d}|j}|||k(rd}n|j}||r|j|B|j|T|j |j |||fS)z2Remove any external control dependency on this op.FT)control_inputs_get_control_flow_context_outer_contextappend_remove_all_control_inputs_add_control_inputs)r7rDinternal_control_inputsexternal_control_inputsxis_internal_opctxts r'_RemoveExternalControlEdgesz-XLACompileContext._RemoveExternalControlEdgess      *n  ( ( *d   4<. ""   &&q)&&q) *!!#23 "$; ;;r)c|jtvr+tjd|j|j|jt vr|j j|td|jDrtd|jzt|jjvrtd|jz|jtt!j"|j$|j&j)||j&j+||j-|\}}|js|sv|j/|j0nZt3t5|jD]9}|j|}|j7|}||us(|j9||;|rt;j<d5|j?|Dcgc]:}|j@r,tCjD|j@djF<}}|jIddd|jK||j@Dcgc]}|j}}|}|*|jLjO||jP}|*|jPr|jPjS|yycc}w#1swYxYwcc}w)zFCreate op in XLACompileContext and notifies outer context recursively.zhOperation of type %s (%s) is not supported in XLA. Execution will fail if this op is used in the graph. c3HK|]}|jjywr#)dtype _is_ref_dtype).0rPs r' z*XLACompileContext.AddOp..s 4Q177 4s "zTNon-resource Variables are not supported inside XLA computations (operator name: %s)z?XLA compiled computations cannot be nested, (operator name: %s))sNr)*r@_DENYLISTED_OPSrAerrorr8_UNSUPPORTED_OPSr5rKanyr&NotImplementedError_XLA_COMPILE_ATTRnode_defattr ValueError _set_attrr AttrValuer4graphprevent_feedingprevent_fetchingrS_add_control_inputr6rangerCAddValue _update_inputr control_dependenciesEnteroutputsr identityrDExitrM_valuesupdaterJ AddInnerOp) r7rDrNrOindexrPreal_x output_namesrs r'AddOpzXLACompileContext.AddOps ww/! mm 346GGRWWF  ww"" ""2& 4")) 44  "$'' * ++BKK,,, #%'WW- ..LL>33d6I6IJLHHR HHb! !% @ @ D 99 $ dkk*RYY(*% IIe q! ?  5& ) *   # #D ) -# yy   qyy| , / /# #  45%'JJ/qAFF/L/G   oo\*&&g     $$R('# 0s$,K*?K%K*9K6%K**K3c|j|jvr+|jj|j}||S|S|}|jj |j|j r@|j j |}|jj |j||j|j<|S)zCAdd `val` to the current context and its outer context recursively.)r8rr_external_valuesgetaddrJrk)r7valresults r'rkzXLACompileContext.AddValue s xx4<<$$((2fNS.. FLLSXX ""++C0f llv{{#&,D#((# Mr)cv|j||jr|jj|yyr#)rxrJrt)r7rDs r'rtzXLACompileContext.AddInnerOps/JJrN  $$R(r)cyr#r7s r' grad_statezXLACompileContext.grad_state!s r)cX|jr|jjSy)z0Forwards to the enclosing while context, if any.F)GetWhileContext back_proprs r'rzXLACompileContext.back_prop)s(   ! ! # - -- r))__name__ __module__ __qualname____doc__r1rFr OperationrSrxrkrtpropertyrr __classcell__)r:s@r'r.r.|ss " I?a&&q)?+?&&(44Y?,  lX&= >% w%G"nl%B7%K"nlG~&)G))+GLLN   ! !,Z @@N+ !Q  ):):1)=>.   - n- Aq 1;?3N **9N I@11*G))+GLLNsbJ+J, 0J:AJ,JAJ,30K;K $K.KJ,J)$J,,$KKK(cBt|tjrS|D]N}t|tjs1t|tjst |j dsNyt|tjryt |j dryy)auChecks if outputs is a flat structure. Following structures and values are considered flat: 1) None 2) A single object 3) A list or tuple of Tensors/Operations The only structures that this function understands are sequences, dictionaries and types defined using the attrs library. E.g. this means that if outputs contains a single user-defined Object, it is considered to be flat. Errors are raised later on if that Object cannot be converted to a Tensor. Args: outputs: Output from `computation` inside `xla.compile`. Returns: A boolean indicates whether outputs is flat. __attrs_attrs__FT)rrrMappinghasattrr:)rors r'rrs,112  Q00 1 Q// 0 !++0 1 001  W   12  r)c$| t}t|tjs|f}|t j fz } |Dcgc]3}t|t jr|nt j|5}}|Dcgc]}t|t js|!}}|Dcgc]}t|t jr|!}}|||zk7r tdg}|D]\}t j|jr |jnd5|jtj|ddd^||fScc}w#t$r}tdt|zd}~wwxYwcc}wcc}w#1swYxYw)zValidates flat outputs and adds back device assignments. Args: outputs: Output from `computation` inside `xla.compile`. Returns: Tensors and Operations extracted from outputs. NoXLA computation function return values must all either be Operations or convertible to Tensors. Got error: "%s"zdXLA computation function must return zero or more Tensor values followed by zero or more Operations.)tuplerrrr rr rr ExceptionrcstrdevicerKr rp)roreoutput_operationsrnew_output_tensorsts r'rrs  _gG G_55 6jG $$& (('@ 3== )s/D/DQ/GGG#*JQZ3==-IqJJ&K!jCMM.JAK.K !222  / 00 7a AHHb 17 2 21 56777 . ../ @  68;A ? @@@ KK77sNE8E?EE<%E</FF%FE E9E44E9F cg}tj|D]}t|tjrt d|j z tj|}tj|jr |jnd5|jtj|ddd|gfS#t$r}t dt|zd}~wwxYw#1swYxYw)aValidates non-flat outputs and adds back device assignments. Args: outputs: Output from `computation` inside `xla.compile`. Returns: Tensors extracted from outputs and an empty list because Operations are not allowed in non-flat outputs.. zxla.compile does not support Operation as return value in non-flat output structure. You can set returned Operations as control dependencies of returned Tensors so Operations are triggered when Tensors are evaluated. Operation found: "%s"rNr)rrrr rrcr8rrrrrKr rp)rorrrs r'rrs << 7a!S]]#  9<=66 B CC L   "a AHHb 17 2 21 5677#7( R  L  BDGF K LLL77s$C%C+ C( C##C(+C4 c#Ktj}dt_ d|t_y#|t_wxYww)aEnters a context where all summary ops are skipped. Summaries are not yet supported in xla.compile(). So we provide this context manager that can skip creating summary ops. This is a temporary workaround due to XLA not supporting summary ops. Yields: None. cy)NTrrr)r'z*_disable_summary_context..%sr)N)r skip_summary)original_skip_summary_funcs r'rrs5 /;;!-/> #=O #=O sA0 A =Ac"eZdZdZdZdZdZy)_CapturedObjectz#A placeholder to capture an object.cd|_yr#_objectrs r'r1z_CapturedObject.__init__0s DLr)c@|jr td||_y)NzFInternalError: _CapturedObject can capture only once. Please file bug.)r RuntimeError)r7rs r'capturez_CapturedObject.capture3s$ ||   DLr)c|jSr#rrs r'r{z_CapturedObject.get;s <<r)N)rrrrr1rr{rr)r'rr-s+r)rcvd}|}|||jz }tj|}t|j}|j d}nt|j }||z }||kr#|dk(r|j |d|S|d|S|j ||kDr|dk(r |d|S|d|Sy)aValidate the number of input arguments to an XLA function. Args: func: the Python function that will be called to generate the body of an XLA computation graph. input_arity: the number of explicit arguments supplied by the caller. infeed_queue: if not None, the infeed queue that will supply additional arguments to the function. Returns: None if function can be called with the supplied number of arguments, or an error string if it cannot. c&d|||dk(rdfzSdfzS)Nz%s %d argument%srrZr) complaintquantitys r' format_errorz3check_function_argument_count..format_errorMs6 H$,M79!< <<7:!< <>>  " "4 ((hmm$- H--."33-&A("2"2": )] 33 *m 44 "3m"CA )] 33 )] 33 r)r#)0r contextlibtensorflow.compiler.jit.opsrrtensorflow.core.frameworkrtensorflow.python.distributertensorflow.python.eagerrrtensorflow.python.frameworkr tensorflow.python.opsr r r tensorflow.python.platformr rAtensorflow.python.utilrrrtensorflow.python.util.compatr"tensorflow.python.util.deprecationr tensorflow.python.util.tf_exportrr`r?setr[r]r,XLAControlFlowContextr.r$rrrcontextmanagerrobjectrrrr)r'rsE0448+0++20<)'-996%    %&  (60 ' 60rr(>>rjdN'T1/h  F >>&f$-r)