AVhWdZddlmZddlmZddlmZddlmZddlmZddlm Z ddl m Z dd l m Z dd l mZdd lmZdd lmZdd lmZddlmZddlmZddlmZej2e j4ej6dfdedej8fdZej2e j:dej6fdedej8fdZej>e j@eejBdddd3dZ ej>e jDed4dZ"ej>e jFe d5dZ#ej>e jHed6de%fdZ$ej>ejLed7dZ&ej>e jNed7dZ'ej>e jPed7dZ(ej>e jRed8d Z)ej>e jTed3d!Z*ej>e jVed8d"Z+ej>e jXed3d#Z,ej>e jZed4d$Z-d4d%Z.d&Z/d4d'Z0d4d(Z1d)Z2d*Z3d+Z4d,Z5d-Z6d.Z7d/Z8d0Z9d1Z:d2Z;y)9zStructuredTensor array ops.)Sequence)flags) constant_op)dtypes)ops)tensor) array_ops)math_ops) random_ops)dynamic_ragged_shape) ragged_tensor) RowPartition)StructuredTensor) deprecation)dispatchNinputreturnc:~|jj|Sz?Returns a DynamicRaggedShape containing the shape of the input. _ragged_shape with_dtyperout_typenames e/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/ops/structured/structured_array_ops.pyshape_v2r#    ' ' 11c:~|jj|Srr)rrrs rshape_v1r!+rrzUse the `axis` argument insteaddimcNtjd|d|}t|||S)aCreates a StructuredTensor with a length 1 axis inserted at index `axis`. This is an implementation of tf.expand_dims for StructuredTensor. Note that the `axis` must be less than or equal to rank. >>> st = StructuredTensor.from_pyval([[{"x": 1}, {"x": 2}], [{"x": 3}]]) >>> tf.expand_dims(st, 0).to_pyval() [[[{'x': 1}, {'x': 2}], [{'x': 3}]]] >>> tf.expand_dims(st, 1).to_pyval() [[[{'x': 1}, {'x': 2}]], [[{'x': 3}]]] >>> tf.expand_dims(st, 2).to_pyval() [[[{'x': 1}], [{'x': 2}]], [[{'x': 3}]]] >>> tf.expand_dims(st, -1).to_pyval() # -1 is the same as 2 [[[{'x': 1}], [{'x': 2}]], [[{'x': 3}]]] Args: input: the original StructuredTensor. axis: the axis to insert the dimension: `-(rank + 1) <= axis <= rank` name: the name of the op. dim: deprecated: use axis. Returns: a new structured tensor with larger rank. Raises: an error if `axis < -(rank + 1)` or `rank < axis`. axisr"r)rdeprecated_argument_lookup_expand_dims_impl)rr$rr"s r expand_dimsr(3s)<  / /eS I$ 5$T 22rct|||S)aCreates a StructuredTensor with a length 1 axis inserted at index `axis`. This is an implementation of tf.expand_dims for StructuredTensor. Note that the `axis` must be less than or equal to rank. >>> st = StructuredTensor.from_pyval([[{"x": 1}, {"x": 2}], [{"x": 3}]]) >>> tf.expand_dims(st, 0).to_pyval() [[[{'x': 1}, {'x': 2}], [{'x': 3}]]] >>> tf.expand_dims(st, 1).to_pyval() [[[{'x': 1}, {'x': 2}]], [[{'x': 3}]]] >>> tf.expand_dims(st, 2).to_pyval() [[[{'x': 1}], [{'x': 2}]], [[{'x': 3}]]] >>> tf.expand_dims(st, -1).to_pyval() # -1 is the same as 2 [[[{'x': 1}], [{'x': 2}]], [[{'x': 3}]]] Args: input: the original StructuredTensor. axis: the axis to insert the dimension: `-(rank + 1) <= axis <= rank` name: the name of the op. Returns: a new structured tensor with larger rank. Raises: an error if `axis < -(rank + 1)` or `rank < axis`. r%)r')rr$rs rexpand_dims_v2r*Us8 5$T 22rc|d}tj|5tj|jj dt jdfd}t||cdddS#1swYyxYw)atf.gather for structured tensors. Does not support (yet) checks on illegal axis values, et cetera. Indices must be a ragged or dense tensor. Args: params: a structured tensor to be gathered indices: a ragged tensor or tensor to gather by. validate_indices: whether to validate the indices name: the name of the op(s). axis: the axis in params to gather on. batch_dims: the number of batch dimensions. Returns: the params reorganized according to indices. Ngatherzparams.shape.rank) ndims_nameindicesr%c:tj|dS)N)validate_indicesr$ batch_dimsr)r r,)pr$r1r.r0s rleaf_opzgather..leaf_ops(    + r) r name_scoper get_positive_axisshaperankr "convert_to_tensor_or_ragged_tensor_extend_op_single)paramsr.r0rr$r1r3s `` `` rr,r,ts. \ D ~~d. | d  & &tV\\->->2E GD>>i!G VW -#...s ABB rc|d}t|fd}tj|djt j |d|5t ||cdddS#1swYyxYw)a#tf.concat for structured tensors. Does not support (yet) checks on illegal axis values, et cetera. Args: values: a sequence of StructuredTensors. axis: an axis to concatenate upon. name: the name of the op(s). Returns: the params reorganized according to indices. Nconcatc0tj|SN)r r<)valuesr$s rr3zconcat..leaf_ops   FD ))rrStructuredConcat),_assert_concat_compatible_structured_tensorsr r5r7rr4 _extend_op)r?r$rr3s ` rr<r<sh \ D.v6*  $ $T6!9>> :$ ~~d.7' fg &'''s  A&&A/c$tj|d||g5|jdk(r td|j }t j tj||}t||dcdddS#1swYyxYw)zShuffle a structured tensor on the zeroth axis. Args: value: a structured tensor of rank at least one. seed: the seed for shuffling. name: the name for shuffle. Returns: The shuffled structured tensor. shufflerz(Cannot shuffle a scalar StructuredTensor)seed)r$N) rr4r7 ValueErrornrowsr random_shuffler ranger,)valuerErfirst_dimensionindexs rrHrHsy ~~dIt}5( zzQ A BBkkmO  % %hnn_&E+/ 1E %Q ' (((s A"BBc|Mtjjjrtj }ntj }t|||S)Returns the size of a tensor.)rr)rconfigtf_shape_default_int64rJrint64int32sizers rsize_v2rTsC ||~,,224hh e$ 22rcR|Mtjjjrtj }ntj }tj|d|g5}|js\|j-tj|j|cdddStjd|cdddS|jdj}|| |cdddStj||cdddS#1swYyxYw)rNNrSdtype)rrOrPrJrrQrRrr4row_partitionsrGr castnvals)rrrr\s rrSrSs ||~,,224hh ~~dFUG, 0     "}}U[[]H5 0 0 }}Q) 0 0   $ * * ,E }(  0 0 ==h / 0 0 0s(AD2D"D=DD&c ~t|||Sz>> st = StructuredTensor.from_pyval([{"x":[3]}, {"x":[4,5]}]) >>> tf.zeros_like(st) >>> st = StructuredTensor.from_pyval([[{"x":[3]}], [{"x":[4,5]}, {"x":[]}]]) >>> tf.zeros_like(st, dtype=tf.int32) Args: input: a structured tensor. dtype: the dtype of the resulting zeros. (default is tf.float32) name: a name for the op. layout: Optional Layout. Only supports replicated layout. Returns: a tensor of zeros of the same shape. N4StructuredTensor only allows replicated layout. got rblayoutrWrX)is_fully_replicatedrFrfloat32rr4rZrGr zerosr RaggedTensor_from_nested_row_partitionsr\rrYrrflast_row_partitionresults rr_r_s*  : : <  >vhG  ] NNE ~~dL5'2 d     " fE   r58  --b1  ' ' C C*002%@ F     AD D 9AD  D)c ~t|||Sr^) ones_like_v2r`s r ones_likerr&s fE 55rcX||jstd||tj}t j |d|g5}|j sa|j0tj|jg||cdddStjg||cdddS|j d}tjjtj|j||j }|cdddS#1swYyxYw)amReplace every object with a zero. Example: >>> st = StructuredTensor.from_pyval([{"x":[3]}, {"x":[4,5]}]) >>> tf.ones_like(st) >>> st = StructuredTensor.from_pyval([[{"x":[3]}], [{"x":[4,5]}, {"x":[]}]]) >>> tf.ones_like(st, dtype=tf.int32) Args: input: a structured tensor. dtype: the dtype of the resulting zeros. (default is tf.float32) name: a name for the op. layout: Optional Layout. Only supports replicated layout. Returns: a tensor of zeros of the same shape. NrdrrrerWrX)rgrFrrhrr4rZrGr onesr rjrkr\rls rrqrq.s*  : : <  >vhG  ] NNE ~~dK%1 T     "~~u{{}ouVD   ~~b%7  --b1  ' ' C C)//1? F    roctj|d|g5}tj|jt j cdddS#1swYyxYw)zReturns the rank of a tensor.r7rXN)rr4rconstantr7rrR)rrs rr7r7YsI ~~dFUG,@    &,, ?@@@s /AAc tj||jdzdd}tj|d||g5|j j Dcic]\}}|tj||}}}|jd|dz|j|dz}t||}|dkDr|jnd}tj|||| cdddScc}}w#1swYyxYw) aCreates a StructuredTensor with a length 1 axis inserted at index `axis`. This is an implementation of tf.expand_dims for StructuredTensor. Note that the `axis` must be less than or equal to rank. >>> st = StructuredTensor.from_pyval([[{"x": 1}, {"x": 2}], [{"x": 3}]]) >>> tf.expand_dims(st, 0).to_pyval() [[[{'x': 1}, {'x': 2}], [{'x': 3}]]] >>> tf.expand_dims(st, 1).to_pyval() [[[{'x': 1}, {'x': 2}]], [[{'x': 3}]]] >>> tf.expand_dims(st, 2).to_pyval() [[[{'x': 1}], [{'x': 2}]], [[{'x': 3}]]] >>> tf.expand_dims(st, -1).to_pyval() # -1 is the same as 2 [[[{'x': 1}], [{'x': 2}]], [[{'x': 3}]]] Args: st: the original StructuredTensor. axis: the axis to insert the dimension: `-(rank + 1) <= axis <= rank` name: the name of the op. Returns: a new structured tensor with larger rank. Raises: an error if `axis < -(rank + 1)` or `rank < axis`. rVr$zrank(st)) axis_namer- ExpandDimsN)rVrr6rZrG) r r5r7rr4_fieldsitemsr(r6_expand_st_row_partitionsrGr from_fields) str$rkv new_fields new_shapenew_row_partitions new_nrowss rr'r'as6  $ $ BGGaK6j B$ ~~dL2t*5 8: 8H8H8J.4q!9 D ))J$$&$%8I22t<#ax aI  ' ')      sC/!C)?A C/)C//C8c~|dk(rS|jjdk(ry|j}tj||dd}|f|j zS||jk(r_|dz dk\r |j |dz j n|j}|j tjd||dfzS|dz dk\r |j |dz jn|j}|j d|dz tjd||dfz|j |dz dzS)z*Create the row_partitions for expand_dims.rrVF)rGvalidateN)r6r7rGrfrom_uniform_row_lengthrZr\)rr$r\ new_partitions rr}r}sG QY xx}}  HHJE 88 uA/M  b// // rww04qA $(#))+BHHJ   D D 5!/ 1 1115qA $(#))+BHHJ  YdQh '<+O+O 5,/+1 1353D3DTAXY3O PPrc t|ts td|s td| t|}|d}t|tr||}|j s|Si}|j D]/}t |Dcgc]}|j|c}||||<1t j||jS||Scc}w)aExtend an op from RaggedTensor and Tensor to StructuredTensor. Visits all children of the structured tensor, and children of children, applying leaf_op whenever it reaches a leaf, and empty_st_op whenever it reaches an internal node without children. Args: values: a list of structured tensors, ragged tensors, or tensors. All must have the same type. If they are structured tensors, they must have the same paths. leaf_op: an op for handling non-structured tensor. empty_st_op: op to create a structured tensor without fields. Returns: the result of the extended op (a StructuredTensor, RaggedTensor, or Tensor) Raises: ValueError: If values is not a Sequence or is empty. zExpected a listzList cannot be emptyrr6) isinstancerrFempty_st_op_like_zerosr field_namesrB field_valuer~r6)r?r3 empty_st_oprJ empty_resultrrrs rrBrBs* FH % & ''  + ,,(1K )%'(v&L     J    . F!Cq!--"2!CW!,.jm.  ' ' ,:L:L MM 6? "DsC c<d}t|g||||S)z4Extend an op to a value instead of a list of values.cyfd}|S)Nc|\}|Sr>r)r?rJ element_ops rlist_opz6_extend_op_single..to_list_op..list_opsgu  rr)rrs` r to_list_opz%_extend_op_single..to_list_ops  Nr)rB)rJr3rrs rr9r9s$ UGZ0*[2I JJrcfd}|S)Nc|Dcgc]}t|tj}}|}t|Scc}w)NrX)r_rrR_structured_tensor_like)r?rJas_zerosrnr3s rrz+empty_st_op_like_zeros..empty_st_opsD>D5: e6<<0HX F "6 ** s"=r)r3rs` rrrs+ rc|jjr!tji|jS|jj t d|jjdk(r8tji|jt j|dStjj|}t|j|jS)z> &d) a qA!%&//^A5F"GA4!8"GHimmQD  #Hs A= c|j}d|ji}|D]N}|j|}t|ts%t |j D]\}}|||f|z<P|S)z3Get ranks of all submessages of a StructuredTensor.r)rr7rrr_get_all_ranksr|)rr all_ranksrrk2v2s rrr.s| >> &277m) "a qA!%&$Q'--/"(2r! 1$)"" rc|Dcgc] }t|}}t}|ddD]%}|j|dj|}'|rt d|ycc}w)z/Raises an error if the paths are not identical.rVNrzASome paths are present in some, but not all, structured tensors: )rsetrsymmetric_differencerF)r?rpaths path_diff other_pathss r_assert_all_paths_matchr:sw(. /">"  /% /e)12YLka = =k JKIL     0sA!cx|Dcgc] }t|}}|ddD]}||dk7s tdycc}w)z>Raises an error if the ranks of submessages are not identical.rVNrz!Ranks of sub-message do not match)rrF)r?rranks other_rankss r_assert_all_ranks_matchrFsN(. /">"  /% /12Y)NNNr)r<)NN)NNT)<__doc__typingrtensorflow.core.configrtensorflow.python.frameworkrrrrrtensorflow.python.opsr r r tensorflow.python.ops.raggedr r *tensorflow.python.ops.ragged.row_partitionr2tensorflow.python.ops.structured.structured_tensorrtensorflow.python.utilrrdispatch_for_apirrRDynamicRaggedShaper6r!dispatch_for_typesr(deprecated_argsr*r,r<strrHrTrSrbr_rrrqr7r'r}rBr9rrrrrrrrrrArrrrsC"(3.+<+*,=6CO.+9--./5||2$2/BB2/29??++/ll2$2';'N'N2,2Y224DET#DeL3MF3@Y557GH3I3<Y--/?@! ).A).XY--/?@'s'A'0Z668HI(J((Y..0@A3B3Y^^-=>0?0,Y113CD7E7Y446FG&H&TY002BC6D6Y335EF'G'TY^^-=>@?@(VP,,^ K M"E - 7   < "r