AVhɆdZddlZddlZddlZddlZddlZddlmZddlmZddlm Z ddlm Z ddl m Z ddl mZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZddl m!Z!ddl"m#Z#gdZ$GddejJe!jLZ'GddejJe!jLZ(d)dZ)ejTe'e)d)dZ+ejTe(e+e(jYGddejJZ-Gdd ej\Z/Gd!d"ej`Z1ejdZ3d#Z4ejjd$Z6Gd%d&e7Z8Gd'd(e1Z9y)*z7Various classes representing distributed values for PS.N)distribute_lib)distribute_utils)values) values_util)coordinator_context)context)dtypes)ops)tensor)tensor_conversion_registry) array_ops)handle_data_util) lookup_ops)resource_variable_ops)variable_scope) save_context)base)core) numpy_compat)_create_resource _initialize_destroy_resourcec*eZdZdZdZdZdZedZdZ dZ dZ d Z d Z ed Zd Zed Zedej$fdZdZdZdDdZdDdZedZedZedZedZedZedZedZedZdZ dZ!dZ"d Z#d!Z$d"Z%d#Z&d$Z'd%Z(d&Z)d'Z*d(Z+d)Z,d*Z-d+Z.d,Z/d-Z0d.Z1d/Z2d0Z3d1Z4d2Z5d3Z6d4Z7d5Z8d6Z9dDd7Z:d8Z;d9Zd<Z?d=Z@d>ZAd?ZBd@ZCdAZDdBZEdEdCZFy)FAggregatingVariablezDA wrapper around a variable that aggregates updates across replicas.cb||_||_tj||_||_yN)_distribute_strategy_vweakrefref_aggregating_container _aggregation)selfstrategyv aggregations V/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/distribute/ps_values.py__init__zAggregatingVariable.__init__6s, (DDG '{{40A#Dctj|j5tj|j |}dddt ||j|j}||t|<|S#1swYAxYw)abPerform a deepcopy of the `AggregatingVariable`. Unlike the deepcopy of a regular tf.Variable, this keeps the original strategy and devices of the `AggregatingVariable`. To avoid confusion with the behavior of deepcopy on a regular `Variable` (which does copy into new devices), we only allow a deepcopy of a `AggregatingVariable` within its originating strategy scope. Args: memo: The memoization object for `deepcopy`. Returns: A deep copy of the current `AggregatingVariable`. Raises: RuntimeError: If trying to deepcopy into a different strategy. N)r$r%r&) renter_or_assert_strategyrcopydeepcopyrtyper"id)r#memor%copied_variables r' __deepcopy__z AggregatingVariable.__deepcopy__>s}$  0 01J1J K' -- &a'!d4j** %%'O %DDN ''s !BB c|jSrrr#s r'getzAggregatingVariable.get\ 77Nr)c|jSr)rr5s r'distribute_strategyz'AggregatingVariable.distribute_strategy_s  $ $$r)c.t|j|Srgetattrrr#names r' __getattr__zAggregatingVariable.__getattr__c 477D !!r)ctjj5|jdtjretj j g|i|cdddSjjj||cdddStj}|sJjtjjk(r)ttj j#d dfd }|j%|||cdddS#1swYyxYw)Nfargskwargsr) variable_typec tj||j}|r)t|tj r|jd}|j j|f|||dS)Nr) use_lockingr> read_valuerC)rapply_aggregationr" isinstancer PerReplicaextendedupdate)r$valuerHr>rIr%rBr#s r'merge_fnz2AggregatingVariable._assign_func..merge_fn~sz ++HeT=N=N,02! jv'8'89;;q>D""))4!, * * r))FNT)rr+rpopin_cross_replica_contextget_update_replica_idrrMrNget_replica_contextr"vsVariableAggregationNONE ValueErrorraggregation_error_msgformat merge_call)r#rDrEreplica_contextrPrBs` @r' _assign_funcz AggregatingVariable._assign_funcfs5  0 01J1J K)N **S/a  0 0 2  / / 1 =477,T,V, )N)N((1188 !$v9/)N)N)<<>    6 6 ; ; ;//66 5778 8 "' $  $))(f)MS)N)N)NsAD<;(D<-BD<<Ec0d}|j|d|i|S)Nc&|j|i|Sr) assign_subvarakws r'z0AggregatingVariable.assign_sub...#..!*Br*Br)rBr])r#rDrE assign_sub_fns r'r`zAggregatingVariable.assign_sub$BM 4  t >} >v >>r)c0d}|j|d|i|S)Nc&|j|i|Sr) assign_addras r'rez0AggregatingVariable.assign_add..rfr)rBrg)r#rDrE assign_add_fns r'rlzAggregatingVariable.assign_addrir)c0d}|j|d|i|S)Nc&|j|i|Sr)assignras r'rez,AggregatingVariable.assign..sjcjj!&:r&:r)rBrg)r#rDrE assign_fns r'rpzAggregatingVariable.assigns$:I 4  4 :y :6 ::r)c.|jjSrr initializerr5s r'rtzAggregatingVariable.initializer 77  r)c6|jjSrrinitialized_valuer5s r'rxz%AggregatingVariable.initialized_value 77 $ $ &&r)c.|jjSrr initial_valuer5s r'r|z!AggregatingVariable.initial_value 77  r)returnc.|jjSrropr5s r'rzAggregatingVariable.op 77::r)c6|jjSr)rrOr5s r'rOzAggregatingVariable.values 77==?r)c6|jjSr)rrIr5s r'rIzAggregatingVariable.read_values 77   r)Nc<|jj||SN)r>r sparse_readr#indicesr>s r'rzAggregatingVariable.sparse_read 77  wT  22r)c8|jj|Srrevalr#sessions r'rzAggregatingVariable.eval 77<<  r)c.|jjSrrgraphr5s r'rzAggregatingVariable.graph 77==r)c.|jjSrrdevicer5s r'rzAggregatingVariable.device 77>>r)c.|jjSrrshaper5s r'rzAggregatingVariable.shaperr)c|jSr)r"r5s r'r&zAggregatingVariable.aggregations   r)c.|jjSrrsynchronizationr5s r'rz#AggregatingVariable.synchronization 77 " ""r)c.|jjSrrr>r5s r'r>zAggregatingVariable.name 77<<r)c.|jjSrr trainabler5s r'rzAggregatingVariable.trainable 77  r)c.|jjSrrdtyper5s r'rzAggregatingVariable.dtyperr)ct|jtr|jjStj |jiSr)rKrCachingVariable _gather_saveables_for_checkpoint trackableVARIABLE_VALUE_KEYr5s r'rz4AggregatingVariable._gather_saveables_for_checkpoints8$''?+ WW 5 5 77  ( ($'' 22r)c h|jj|||fi|}||j||<|SFor implementing `Trackable`.r_export_to_saved_model_graphr# object_map tensor_mapoptionsrE resource_lists r'rz0AggregatingVariable._export_to_saved_model_graphD 9DGG88Z9@LDJLM!$''*Jt r)c|jj|||vr2t|j||j|j||<yyrN)r_copy_trackable_to_cpurrr"r#rs r'rz*AggregatingVariable._copy_trackable_to_cpusP  GG"":. :,T-F-F-7-@-1->->@jr)c |j|zSrr4r#os r'__add__zAggregatingVariable.__add__ 77Q;r)c ||jzSrr4rs r'__radd__zAggregatingVariable.__radd__ tww;r)c |j|z Srr4rs r'__sub__zAggregatingVariable.__sub__rr)c ||jz Srr4rs r'__rsub__zAggregatingVariable.__rsub__rr)c |j|zSrr4rs r'__mul__zAggregatingVariable.__mul__rr)c ||jzSrr4rs r'__rmul__zAggregatingVariable.__rmul__rr)c |j|z Srr4rs r' __truediv__zAggregatingVariable.__truediv__ rr)c ||jz Srr4rs r' __rtruediv__z AggregatingVariable.__rtruediv__ rr)c |j|zSrr4rs r' __floordiv__z AggregatingVariable.__floordiv__ 77a<r)c ||jzSrr4rs r' __rfloordiv__z!AggregatingVariable.__rfloordiv__s <r)c |j|zSrr4rs r'__mod__zAggregatingVariable.__mod__rr)c ||jzSrr4rs r'__rmod__zAggregatingVariable.__rmod__rr)c |j|kSrr4rs r'__lt__zAggregatingVariable.__lt__rr)c |j|kSrr4rs r'__le__zAggregatingVariable.__le__rr)c |j|kDSrr4rs r'__gt__zAggregatingVariable.__gt__!rr)c |j|k\Srr4rs r'__ge__zAggregatingVariable.__ge__$rr)c |j|zSrr4rs r'__and__zAggregatingVariable.__and__'rr)c ||jzSrr4rs r'__rand__zAggregatingVariable.__rand__*rr)c |j|zSrr4rs r'__or__zAggregatingVariable.__or__-rr)c ||jzSrr4rs r'__ror__zAggregatingVariable.__ror__0rr)c |j|z Srr4rs r'__xor__zAggregatingVariable.__xor__3rr)c ||jz Srr4rs r'__rxor__zAggregatingVariable.__rxor__6rr)c |j|Srr4rs r' __getitem__zAggregatingVariable.__getitem__9s 771:r)c0t|j||Srpowr)r#rmodulos r'__pow__zAggregatingVariable.__pow__<s tww6 ""r)c.t||jSrrrs r'__rpow__zAggregatingVariable.__rpow__?s q$''?r)c|jSrr4r5s r' __invert__zAggregatingVariable.__invert__B GG8Or)c|j Srr4r5s r'__neg__zAggregatingVariable.__neg__Err)c,t|jSr)absrr5s r'__abs__zAggregatingVariable.__abs__H tww<r)cd |jj|S#t$r tcYSwxYwr)r__div__AttributeErrorNotImplementedrs r'rzAggregatingVariable.__div__Ks/ WW__Q    //cd |jj|S#t$r tcYSwxYwr)r__rdiv__rrrs r'r zAggregatingVariable.__rdiv__Rs1 WW  a   rcd |jj|S#t$r tcYSwxYwr)r __matmul__rrrs r'r zAggregatingVariable.__matmul__Ys1 WW   ""  rcd |jj|S#t$r tcYSwxYwr)r __rmatmul__rrrs r'r zAggregatingVariable.__rmatmul__`s1 WW  ##  rc,t|jSrstrrr5s r'__str__zAggregatingVariable.__str__grr)c,t|jSrreprrr5s r'__repr__zAggregatingVariable.__repr__j =r)cyz6Pass resource_variable_ops.is_resource_variable check.Nr5s r' _should_act_as_resource_variablez4AggregatingVariable._should_act_as_resource_variablemr)c>|jj|||S)Nrr>as_ref)r_dense_var_to_tensorr#rr>rs r'rz(AggregatingVariable._dense_var_to_tensorqs 77 ' 'e$v ' NNr)rNNF)G__name__ __module__ __qualname____doc__r(r2r6propertyr9r?r]r`rlrprtrxr|r OperationrrOrIrrrrrr&rr>rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr r r rrrrrr)r'rr2sL$< % %"*NX??;  ' ! ! #--  3!         # #      3  @# Or)rceZdZdZdZdZdZdZd)dZdZ d Z d Z d Z e d Zd Ze dZe dej$fdZdZd)dZe dZe dZe dZe dZe dZe dZe dZe dZd)dZdZdZ dZ!dZ"d Z#d!Z$d"Z%d*d#Z&e'd$Z(e'd%Z)d&Z*d'Z+d(Z,y)+rz;A wrapper around a variable that caches read value locally.c.||_d|_d|_y)Nr)r_cache_current_new_cache_scope_count)r#r%s r'r(zCachingVariable.__init__xsDGDK*+D'r)c|jSrr4r5s r'r6zCachingVariable.get}r7r)c.t|j|Srr;r=s r'r?zCachingVariable.__getattr__r@r)ctjjr|jS|jj Sr)rcaching_scope_localin_caching_scopecached_read_valuerrIr5s r'rIzCachingVariable.read_values6++<<>  # # %% 77   r)Nc<|jj||Srrrs r'rzCachingVariable.sparse_readrr)ctjj|jkDr|xjdz c_d|_t j d5|j|jcdddStj|j|_|jcdddS#1swYyxYw)NzCPU:0) rr/new_cache_scope_countr+r*r rr identityrr5s r'r1z!CachingVariable.cached_read_values,,BB ++ , ))Q.)dk G   {{ ((1 {{ sB4;/B44B=c:|jj|i|Sr)rr`r#rDrEs r'r`zCachingVariable.assign_sub 477  t .v ..r)c:|jj|i|Sr)rrlr8s r'rlzCachingVariable.assign_addr9r)c:|jj|i|Sr)rrpr8s r'rpzCachingVariable.assigns 477>>4 *6 **r)c.|jjSrrsr5s r'rtzCachingVariable.initializerrur)c6|jjSrrwr5s r'rxz!CachingVariable.initialized_valueryr)c.|jjSrr{r5s r'r|zCachingVariable.initial_valuer}r)r~c.|jjSrrr5s r'rzCachingVariable.oprr)ctjjr|jS|jj Sr)rr/r0r1rrOr5s r'rOzCachingVariable.values3++<<>  # # %% 77==?r)c8|jj|Srrrs r'rzCachingVariable.evalrr)c.|jjSrrr5s r'rzCachingVariable.graphrr)c.|jjSrrr5s r'rzCachingVariable.devicerr)c.|jjSrrr5s r'rzCachingVariable.shaperr)c.|jjSrrr5s r'rzCachingVariable.synchronizationrr)c.|jjSrrr5s r'r>zCachingVariable.namerr)c.|jjSrrr5s r'rzCachingVariable.trainablerr)c.|jjSrrr5s r'rzCachingVariable.dtyperr)c.|jjSr)r constraintr5s r'rJzCachingVariable.constraints 77  r)cLtj|j|S)Nr)r np_asarraynumpy)r#rs r' __array__zCachingVariable.__array__s  " "4::  # # %% 77 ' 'e$u ' MMr)cztjjD]}|dk(s|dk(r|j| y)z%Register overloads for all operators.__eq____ne__N)r TensorOVERLOADABLE_OPERATORS_tensor_overload_operator)clsoperators r' _overload_overloadable_operatorsz0CachingVariable._overload_overloadable_operatorss=MM88. X X!5 ##H- .r)c^ttj|fd}t|||y)z1Delegate an operator overload to `tensor.Tensor`.c8|jg|i|SrrO)r%rDrEtensor_operators r' _operatorz._operators QWWY 8 8 88r)N)r<r rcsetattr)rfrgrmrls @r'rez)CachingVariable._tensor_overload_operators&fmmX6O9 C9%r)c:tj|jiSr)rrrr5s r'rz0CachingVariable._gather_saveables_for_checkpoint s  ( ($'' 22r)c h|jj|||fi|}||j||<|Srrrs r'rz,CachingVariable._export_to_saved_model_graphrr)cz|jj|||vrt||j||<yyr)rrrrs r'rz&CachingVariable._copy_trackable_to_cpus;  GG"":. :(DGG)<=jr)rr!)-r"r#r$r%r(r6r?rIrr1r`rlrpr&rtrxr|r r'rrOrrrrrr>rrrJrOrRrUrXrNrrrr classmethodrhrerrrrr)r'rrusC, " 3 //+  ' ! ! #--  !       # #        >)%'H N ..&&3 >r)rc(|j|||Srrrbrr>rs r'_tensor_conversion_aggregaterv' ! !%v 66r)c(|j|||Srrtrus r'_tensor_conversion_cachingry1rwr)cHeZdZdZdZedZedZdZ dZ dZ y) PerWorkerVariableaA wrapper around unsynced variables created on workers. `PerWorkerVariable`s are variables that are stored on workers and not synchronized. A `PerWorkerVariable` is really a wrapper around multiple independent `Variable`s stored on independent worker machines. `PerWorkerVariable` is currently only tested and supported when used with `ParameterServerStrategy`. A `PerWorkerVariable` can be created by creating a `Variable` within strategy scope and using the `per_worker_variable` flag, e.g.: ``` with strategy.scope(): var = tf.Variable(initial_value=0.0, per_worker_variable=True) ``` The implementation modifies the graph to ensure that a worker's local version of the variable is used for computation at call time, while needing only one function trace and requiring no code changes beyond the `per_worker_variable` flag. `PerWorkerVariable`s can thus be treated like a standard `Variable`, but support is experimental and not all ops have been tested. All per-worker values can be retrieved and read into a list via `PerWorkerVariable.read_all()`. Caveats: - `PerWorkerVariable`s should not be used as direct inputs to a `tf.function`. That is, they should not appear in a tf.function header as an input argument. However they can still be read and manipulated in a `tf.function`. - The `shape` argument must be fully-defined (no `None` entries) or left empty. Partially-defined shapes are not yet supported. - Automatic control dependencies do not work with `PerWorkerVariable`s, so returning a `PerWorkerVariable` is not supported, and `read_all()` should be used to retrieve values. (TODO: b/286052052) - `PerWorkerVariable`s should not be created within a `tf.function`. c p|j|_d|_tj|fi||_|d i||_|jd8tj5tj |_ dddn |d|_ d|_ |j j|_|j j |_d|_|jd|_|jdd|_n |ddz|_|jdd|_y#1swYxYw) N in_graph_modeF unique_id handle_namez Variable:0z:0validate_shapeTr)_cluster_coordinator _coordinator_per_worker_vars functoolspartial _var_creator_coordinator_instancer6r init_scoperrZ_in_graph_mode _cached_valuer_shaper_dtype _trainable _unique_id _handle_name_validate_shape)r#r$ next_creatorrEs r'r(zPerWorkerVariable.__init__as 55D D!)),A&AD!-!7!7Dzz/"* >> >")";";"==>>#?3dD,,22DK,,22DKDOjj-DO zz- (&d /$6d!::&6=D>>s &D,,D5cy)zEOverride to be a no-op to avoid metaclass creating ResourceVariables.Nr)rfrDrEs r'_variable_callz PerWorkerVariable._variable_callzs r)c tjstjr|jj S|j |j\}}tjj||Sr) rrZrin_save_contextrhandle_maybe_create_per_worker_varshandle_call_time_valuer get_default_graphcapture_call_time_valuer#closurespecs r'rzPerWorkerVariable.handlesk  "l&B&B&D  ' ' . .. ((*113mgt  " " $ < <  r)cRfd}|tjjfS)zReturns a closure to run for a handle at call time and its spec. This function is called in self.handle to create a placeholder which returns a handle on some worker or on the coordinator. ctj}|r@jj|j}|j |}|j Sjj Sr)rget_current_dispatch_contextr_values worker_indexmaybe_get_remote_valuerrdispatch_context remote_valueretr#s r'rz9PerWorkerVariable.handle_call_time_value..closuresb,IIK ,,44  ) )+ 55lCzz))000r)rk)PerWorkerVariableSpecrrr#rs` r'rz(PerWorkerVariable.handle_call_time_values. 1 )((//1 11r)cr|js+|jj|j|_yy)z9Create variable on each worker if it hasn't been created.N)rr_create_per_worker_variablesrr5s r'rz/PerWorkerVariable._maybe_create_per_worker_varss2     8 89J9J K  !r)cp|jjDcgc]}|jc}Scc}w)zESynchronously read variables from all workers into a list of Tensors.)rrr6)r#wvs r'read_allzPerWorkerVariable.read_alls(#44<< =BFFH == =s3N) r"r#r$r%r(rrrr&rrrrrr)r'r{r{;sE#J>2  1(M >r)r{c*eZdZdfd ZfdZxZS)rc`t||j|j|||_yr)superr(rr_value)r#rOr> __class__s r'r(zPerWorkerVariableSpec.__init__s& GU[[%++D9DKr)czt||}tj||jj |Sr)rplaceholder_valuerset_handle_datar _handle_data)r#placeholder_context placeholderrs r'rz'PerWorkerVariableSpec.placeholder_values3'+,?@K$$[$++2J2JK r))NN)r"r#r$r(r __classcell__rs@r'rrsr)rcNeZdZdZdZdZdZdZedZ edZ dZ y ) DistributedTablea/A distributed StaticHashTable for ParameterServerStrategy. An instance of DistributedTable has copies of a StaticHashTable and its resource handle on the coordinator of each worker, created at the DistributedTable instance initialization time with initializers on each worker. Users can call methods on a DistributedTable as if it were a StaticHashTable, which leads to execution with the resource local to the consumer worker (or the coordinator, if calling from the coordinator). This implementation relies on the fact that the methods of StaticHashTable are queried with the resource handle (instead of the python object). Currently, at saving time, a DistributedTable is saved as a StaticHashTable on the coordinator, and restoring a DistributedTable from SavedModel is not supported. c\tjj|jjdj d||_||_|j|_ d|_ tj|_ tjs|j!yy)NPSSDistributedLookupTabler4)r'distribution_strategy_input_api_counterget_cellrr" increase_byr_wrapped_creatorrr_distributed_table threadingLock _distributed_table_creation_lockrr_maybe_build_distributed_table)r#r$wrapped_creators r'r(zDistributedTable.__init__s::CC !<>>Ik!n!0!2D+D 55D#D,5NN,.wrappersD24262 2r))rr__dict__callablerKr&r<)r#attrrrs` @r'r?zDistributedTable.__getattr__s &&   t))222--66t.closures_-IIK ..66  ) )+ 55lC ))999r)rLr TensorSpecr resourcers` r'resource_handle_call_time_valuez0DistributedTable.resource_handle_call_time_values& : F%%b@ @@r)cj5js%fd}jj|_dddy#1swYyxYw)ICreate table objects and resources on each worker if hasn't been created.c@j}|j}|Sr)rr) new_tablerr#s r' create_copyzDDistributedTable._maybe_build_distributed_table..create_copy s!++-)))#*r)Nrrr_create_per_worker_resourcesr#rs` r'rz/DistributedTable._maybe_build_distributed_tablesQ . . I  $ $     : :; G  I I I 2A  Ac8tjstjr|jj S|j |j\}}tjj|||jj S)N) default_value) rrZrrrrrrr rrrs r'rz DistributedTable.resource_handles  "l&B&B&D  ' ' 7 77 ))+::.closure`s-IIK  6 8 ' *7@@PP,0088++-,55lC ))999r)r)rrrrs` r'rz8RestoredDistributedTable.resource_handle_call_time_valueYs':B F%%BfooF FFr)c`|tvrtdsi_|j|<tfdtDr/j5jj dddj j||Stt||S#1swY9xYw)N_restored_functionc3:K|]}|jvywrr.0methodr#s r' z7RestoredDistributedTable.__setattr__..s$ 7t.. . 7) TRACKABLE_RESOURCE_METHODShasattrrallr notify_allr __setattr__rr)r#r>rOrs` r'r z$RestoredDistributedTable.__setattr__s ))T/ 0"$&+dd#  75 7 7  ) ) 4  & & 1 1 3 4  ' ' 3 3D% @@ +T >tU KK  4 4s B$$B-c6|jjS)zEA function that creates a resource handle for a table on coordinator.)rrr5s r'rz)RestoredDistributedTable._create_resources  % % 6 6 88r)c6|jjS)z)A function that initializes the resource.)rrr5s r'rz$RestoredDistributedTable._initializes  % % 1 1 33r)c6|jjS)z&A function that destroys the resource.)rrr5s r'rz*RestoredDistributedTable._destroy_resources  % % 7 7 99r)cj5js%fd}jj|_dddy#1swYyxYw)rcj}j5tdrtfdtDr@jj tds'tfdtDr@dddtdrt |5jjD]\}}t||||j}tjs.tjtjj |ddd|j"}|S#1swYxYw#1swY#xYw)Nrc3:K|]}|jvywrrrs r'rz_RestoredDistributedTable._maybe_build_distributed_table..create_copy..s&A:d555A:r)rrr anyr waitrritemsrnrrrZr add_to_collection GraphKeysTABLE_INITIALIZERSr)rr> tf_functioninit_oprr#s r'rzLRestoredDistributedTable._maybe_build_distributed_table..create_copys,++-)++ 2d$89SA:8A:>:**//1d$89SA:8A:>: 2 T/ 04Y? Q'+'>'>'D'D'F6#$  456!--/g..0%%cmm&F&FP  Q))#* 2 2 Q QsA D:*D:"BE:EENrrs` r'rz7RestoredDistributedTable._maybe_build_distributed_tablesQ . .I  $ $ ,    : :; G 1IIIr) r"r#r$r%r(rr rrrrrrs@r'rrQs-O0 (GTL294:Ir)rr!):r% contextlibr,rrrtensorflow.python.distributerrrr(tensorflow.python.distribute.coordinatorrtensorflow.python.eagerrtensorflow.python.frameworkr r r r tensorflow.python.opsr rrrrrUtensorflow.python.saved_modelrtensorflow.python.trackablerrtensorflow.python.typesrtensorflow.python.utilrr BaseResourceVariablercrrrv#register_tensor_conversion_functionryrhr{rrStaticHashTablerlocalrrcontextmanagerrobjectrrrr)r'r,sk> 79/4H+.+.B+2,7669(/ @O/DD++@OF m>+@@$++m>d7?>>57 7?>>/1002k>-BBk>\F--wBz11wBt#2)//"3 == &  rI/rIr)