AVhdZddlZddlZddlZddlZddlmZddlm Z ddl m Z ddl m Z ddl mZddl mZdd l mZdd l mZdd l mZdd l mZdd l mZddl mZddl mZddl mZddl mZddlmZddlmZddlmZddlmZddlmZddlm Z ddlm!Z!ddl"m#Z#ddl"m$Z$ddl"m%Z%ddl&m'Z'ddl(m)Z)ddl*m+Z,ddl*m-Z.dd l/m0Z0dd!l1m2Z2dd"l3m4Z4e'jjZ5e2d#Gd$d#e jle.jne.jpZ8d%Z9d&Z:e2d'ejvd(Gd)d'ejxe.jzZ=e)j|e)j~e=ejjeje$je=j dDd*ZEd+ZFd,ZGd-ZHd.ZId/ZJe je8eHeIeJGd0d1ZLd2ZMejd3d4ZOdEd5ZPdEd6ZQejfd7ZSd8ZTd9ZUd:ZVd;ZWd<ZXd=ZYd>ZZeje8fa\d?Z]d@Z^dAZ_dBZ`eje8e$jfZbejebe,jfZdddCl"meZey)Fz4Classes for storing ragged tensors and their values.N) struct_pb2)tf2)session)composite_tensor)composite_tensor_gradient) constant_op)dtypes)ops) sparse_tensor)tensor)tensor_conversion) tensor_shape) tensor_util) type_spec)type_spec_registry) array_ops)array_ops_stack) check_ops)cond)control_flow_assert)gen_ragged_conversion_ops)math_ops) ragged_config)ragged_tensor_value) ragged_util) RowPartition)nested_structure_coder)coreinternal)dispatch) tf_export) doc_controls RaggedTensorcjeZdZdZej dWdZedXdZee j dYdZ ee jdZdZ ee jdZdZ ee jdZdZee jdZd Zee j d[d Zee j dYd Zee j dZd Zee j dZd Ze dZdZedZedZedZdej4fdZedZedZedZedZedZ edZ!edZ"d\dZ#d\dZ$d]dZ%d\dZ&d\dZ'd^d!Z(d\d"Z)d_d#Z*d$Z+d%Z,d&Z-d'Z.d(Z/ee jddd de0jbfd)Z2d_d*Z3ee jde0jbfd+Z4d\d,Z5ede0jbdfd-Z6d`d.Z7d/Z8d0Z9d1Z:d2Z;d3Ze=d6Z?e=d7Z@e=d8ZAe=d9ZBe=d:ZCe=d;ZDe=d<ZEe=d=ZFe=d>ZGe=d?ZHe=d@ZIe=dAZJe=dBZKe=dCZLe=dDZMe=dEZNe=dFZOe=dGZPe=dHZQe=dIZRe=dJZSe=dKZTe=dLZUe=dMZVe=dNZWe=dOZXe=dPZYe=dQZZe=dRZ[[=dSZ\edTZ]dUZ^dVZ_e`jZby)ar$au Represents a ragged tensor. A `RaggedTensor` is a tensor with one or more *ragged dimensions*, which are dimensions whose slices may have different lengths. For example, the inner (column) dimension of `rt=[[3, 1, 4, 1], [], [5, 9, 2], [6], []]` is ragged, since the column slices (`rt[0, :]`, ..., `rt[4, :]`) have different lengths. Dimensions whose slices all have the same length are called *uniform dimensions*. The outermost dimension of a `RaggedTensor` is always uniform, since it consists of a single slice (and so there is no possibility for differing slice lengths). The total number of dimensions in a `RaggedTensor` is called its *rank*, and the number of ragged dimensions in a `RaggedTensor` is called its *ragged-rank*. A `RaggedTensor`'s ragged-rank is fixed at graph creation time: it can't depend on the runtime values of `Tensor`s, and can't vary dynamically for different session runs. Note that the `__init__` constructor is private. Please use one of the following methods to construct a `RaggedTensor`: * `tf.RaggedTensor.from_row_lengths` * `tf.RaggedTensor.from_value_rowids` * `tf.RaggedTensor.from_row_splits` * `tf.RaggedTensor.from_row_starts` * `tf.RaggedTensor.from_row_limits` * `tf.RaggedTensor.from_nested_row_splits` * `tf.RaggedTensor.from_nested_row_lengths` * `tf.RaggedTensor.from_nested_value_rowids` ### Potentially Ragged Tensors Many ops support both `Tensor`s and `RaggedTensor`s (see [tf.ragged](https://www.tensorflow.org/api_docs/python/tf/ragged) for a full listing). The term "potentially ragged tensor" may be used to refer to a tensor that might be either a `Tensor` or a `RaggedTensor`. The ragged-rank of a `Tensor` is zero. ### Documenting RaggedTensor Shapes When documenting the shape of a RaggedTensor, ragged dimensions can be indicated by enclosing them in parentheses. For example, the shape of a 3-D `RaggedTensor` that stores the fixed-size word embedding for each word in a sentence, for each sentence in a batch, could be written as `[num_sentences, (num_words), embedding_size]`. The parentheses around `(num_words)` indicate that dimension is ragged, and that the length of each element list in that dimension may vary for each item. ### Component Tensors Internally, a `RaggedTensor` consists of a concatenated list of values that are partitioned into variable-length rows. In particular, each `RaggedTensor` consists of: * A `values` tensor, which concatenates the variable-length rows into a flattened list. For example, the `values` tensor for `[[3, 1, 4, 1], [], [5, 9, 2], [6], []]` is `[3, 1, 4, 1, 5, 9, 2, 6]`. * A `row_splits` vector, which indicates how those flattened values are divided into rows. In particular, the values for row `rt[i]` are stored in the slice `rt.values[rt.row_splits[i]:rt.row_splits[i+1]]`. Example: >>> print(tf.RaggedTensor.from_row_splits( ... values=[3, 1, 4, 1, 5, 9, 2, 6], ... row_splits=[0, 4, 4, 7, 8, 8])) ### Alternative Row-Partitioning Schemes In addition to `row_splits`, ragged tensors provide support for five other row-partitioning schemes: * `row_lengths`: a vector with shape `[nrows]`, which specifies the length of each row. * `value_rowids` and `nrows`: `value_rowids` is a vector with shape `[nvals]`, corresponding one-to-one with `values`, which specifies each value's row index. In particular, the row `rt[row]` consists of the values `rt.values[j]` where `value_rowids[j]==row`. `nrows` is an integer scalar that specifies the number of rows in the `RaggedTensor`. (`nrows` is used to indicate trailing empty rows.) * `row_starts`: a vector with shape `[nrows]`, which specifies the start offset of each row. Equivalent to `row_splits[:-1]`. * `row_limits`: a vector with shape `[nrows]`, which specifies the stop offset of each row. Equivalent to `row_splits[1:]`. * `uniform_row_length`: A scalar tensor, specifying the length of every row. This row-partitioning scheme may only be used if all rows have the same length. Example: The following ragged tensors are equivalent, and all represent the nested list `[[3, 1, 4, 1], [], [5, 9, 2], [6], []]`. >>> values = [3, 1, 4, 1, 5, 9, 2, 6] >>> RaggedTensor.from_row_splits(values, row_splits=[0, 4, 4, 7, 8, 8]) >>> RaggedTensor.from_row_lengths(values, row_lengths=[4, 0, 3, 1, 0]) >>> RaggedTensor.from_value_rowids( ... values, value_rowids=[0, 0, 0, 0, 2, 2, 2, 3], nrows=5) >>> RaggedTensor.from_row_starts(values, row_starts=[0, 4, 4, 7, 8]) >>> RaggedTensor.from_row_limits(values, row_limits=[4, 4, 7, 8, 8]) >>> RaggedTensor.from_uniform_row_length(values, uniform_row_length=2) ### Multiple Ragged Dimensions `RaggedTensor`s with multiple ragged dimensions can be defined by using a nested `RaggedTensor` for the `values` tensor. Each nested `RaggedTensor` adds a single ragged dimension. >>> inner_rt = RaggedTensor.from_row_splits( # =rt1 from above ... values=[3, 1, 4, 1, 5, 9, 2, 6], row_splits=[0, 4, 4, 7, 8, 8]) >>> outer_rt = RaggedTensor.from_row_splits( ... values=inner_rt, row_splits=[0, 3, 3, 5]) >>> print(outer_rt.to_list()) [[[3, 1, 4, 1], [], [5, 9, 2]], [], [[6], []]] >>> print(outer_rt.ragged_rank) 2 The factory function `RaggedTensor.from_nested_row_splits` may be used to construct a `RaggedTensor` with multiple ragged dimensions directly, by providing a list of `row_splits` tensors: >>> RaggedTensor.from_nested_row_splits( ... flat_values=[3, 1, 4, 1, 5, 9, 2, 6], ... nested_row_splits=([0, 3, 3, 5], [0, 4, 4, 7, 8, 8])).to_list() [[[3, 1, 4, 1], [], [5, 9, 2]], [], [[6], []]] ### Uniform Inner Dimensions `RaggedTensor`s with uniform inner dimensions can be defined by using a multidimensional `Tensor` for `values`. >>> rt = RaggedTensor.from_row_splits(values=tf.ones([5, 3], tf.int32), ... row_splits=[0, 2, 5]) >>> print(rt.to_list()) [[[1, 1, 1], [1, 1, 1]], [[1, 1, 1], [1, 1, 1], [1, 1, 1]]] >>> print(rt.shape) (2, None, 3) ### Uniform Outer Dimensions `RaggedTensor`s with uniform outer dimensions can be defined by using one or more `RaggedTensor` with a `uniform_row_length` row-partitioning tensor. For example, a `RaggedTensor` with shape `[2, 2, None]` can be constructed with this method from a `RaggedTensor` values with shape `[4, None]`: >>> values = tf.ragged.constant([[1, 2, 3], [4], [5, 6], [7, 8, 9, 10]]) >>> print(values.shape) (4, None) >>> rt6 = tf.RaggedTensor.from_uniform_row_length(values, 2) >>> print(rt6) >>> print(rt6.shape) (2, 2, None) Note that `rt6` only contains one ragged dimension (the innermost dimension). In contrast, if `from_row_splits` is used to construct a similar `RaggedTensor`, then that `RaggedTensor` will have two ragged dimensions: >>> rt7 = tf.RaggedTensor.from_row_splits(values, [0, 2, 4]) >>> print(rt7.shape) (2, None, None) Uniform and ragged outer dimensions may be interleaved, meaning that a tensor with any combination of ragged and uniform dimensions may be created. For example, a RaggedTensor `t4` with shape `[3, None, 4, 8, None, 2]` could be constructed as follows: ```python t0 = tf.zeros([1000, 2]) # Shape: [1000, 2] t1 = RaggedTensor.from_row_lengths(t0, [...]) # [160, None, 2] t2 = RaggedTensor.from_uniform_row_length(t1, 8) # [20, 8, None, 2] t3 = RaggedTensor.from_uniform_row_length(t2, 4) # [5, 4, 8, None, 2] t4 = RaggedTensor.from_row_lengths(t3, [...]) # [3, None, 4, 8, None, 2] ``` c.|s tdt|t|tst d|d|j j dt|tr%|j|jjk(sJ||_ ||_ y)aiCreates a `RaggedTensor` with a specified partitioning for `values`. This constructor is private -- please use one of the following ops to build `RaggedTensor`s: * `tf.RaggedTensor.from_row_lengths` * `tf.RaggedTensor.from_value_rowids` * `tf.RaggedTensor.from_row_splits` * `tf.RaggedTensor.from_row_starts` * `tf.RaggedTensor.from_row_limits` * `tf.RaggedTensor.from_nested_row_splits` * `tf.RaggedTensor.from_nested_row_lengths` * `tf.RaggedTensor.from_nested_value_rowids` Args: values: A potentially ragged tensor of any dtype and shape `[nvals, ...]`. row_partition: A `RowPartition` object, representing the arrangement of the lists at the top level. internal: True if the constructor is being called by one of the factory methods. If false, an exception will be raised. Raises: ValueError: If internal = False. Note that this method is intended only for internal use. TypeError: If values is not a `RaggedTensor` or `Tensor`, or row_partition is not a `RowPartition`. zzRaggedTensor constructor is private; please use one of the factory methods instead (e.g., RaggedTensor.from_row_lengths()):Argument `row_partition` must be a RowPartition. Received .N) ValueError'_assert_is_supported_ragged_values_type isinstancer TypeErrorshapewith_rank_at_leastr$dtype_row_partition_values)selfvalues row_partitionr s Z/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/ops/ragged/ragged_tensor.py__init__zRaggedTensor.__init__s<  : ;;,F3 m\ 2 ""/3 44 LL##A&&,'  F$9$9$?$? ?? ?DL'Dct|tstd|dt|tstd|d|j ||d\}}|j r8|j j}|jddj||rd}t||j}tjtj|j|j||g}t|t s%|j#tj$|d|j'|}||d | S) aoCreates a `RaggedTensor` with a row partition. This is used as a way for RaggedTensors to share row partitions. The outer dimension of values must be equal to `partition.nvals()`. Args: values: A potentially ragged tensor. row_partition: a `RowPartition`: can be shared between tensors. validate: If true, then use assertions to check that the arguments form a valid `RaggedTensor`. Returns: A `RaggedTensor`. `result.rank = values.rank + 1`. `result.ragged_rank = values.ragged_rank + 1`. Raises: ValueError: If partition.nvals() != _nrows(values) r'r(2Argument `validate` must have type bool. Received partitionNr)zAArguments to _from_row_partition do not form a valid RaggedTensormessageT)r4r r5)r,rr-bool_convert_values_and_partition_has_precomputed_value_rowids value_rowidsr.assert_is_compatible_with_nrowsr0r assert_equalrcastnvalsr$appendassert_rank_at_least_with_dependencies)clsr4r5validatevalue_rowids_shapemsgrFcheckss r6_from_row_partitionz RaggedTensor._from_row_partition:sA* m\ 2 ""/3 44 h % ""*1. //== {,FM224(557== ll2A001CD OcV]001e  mmM//1=3F3FG f  - i44VQ?@#66v>m ft= IIr8Nc  t|tstd|dtj|d|||g5t j |||t|}|j|||cdddS#1swYyxYw)aiCreates a `RaggedTensor` with rows partitioned by `value_rowids`. The returned `RaggedTensor` corresponds with the python list defined by: ```python result = [[values[i] for i in range(len(values)) if value_rowids[i] == row] for row in range(nrows)] ``` Args: values: A potentially ragged tensor with shape `[nvals, ...]`. value_rowids: A 1-D integer tensor with shape `[nvals]`, which corresponds one-to-one with `values`, and specifies each value's row index. Must be nonnegative, and must be sorted in ascending order. nrows: An integer scalar specifying the number of rows. This should be specified if the `RaggedTensor` may containing empty training rows. Must be greater than `value_rowids[-1]` (or zero if `value_rowids` is empty). Defaults to `value_rowids[-1] + 1` (or zero if `value_rowids` is empty). name: A name prefix for the RaggedTensor (optional). validate: If true, then use assertions to check that the arguments form a valid `RaggedTensor`. Note: these assertions incur a runtime cost, since they must be checked for each tensor value. Returns: A `RaggedTensor`. `result.rank = values.rank + 1`. `result.ragged_rank = values.ragged_rank + 1`. Raises: ValueError: If `nrows` is incompatible with `value_rowids`. #### Example: >>> print(tf.RaggedTensor.from_value_rowids( ... values=[3, 1, 4, 1, 5, 9, 2, 6], ... value_rowids=[0, 0, 0, 0, 2, 2, 2, 3], ... nrows=5)) r:r(RaggedFromValueRowIds)rAnrowsrK dtype_hintrKN) r,r>r-r name_scoperfrom_value_rowids_get_optional_partition_dtyperO)rJr4rArRnamerKr5s r6rVzRaggedTensor.from_value_rowidshs^ h % ""*1. // 5u5 7O"44#26: >> print(tf.RaggedTensor.from_row_splits( ... values=[3, 1, 4, 1, 5, 9, 2, 6], ... row_splits=[0, 4, 4, 7, 8, 8])) r:r(RaggedFromRowSplits) row_splitsrKrSrTN) r,r>r-r rUrfrom_row_splitsrWrO)rJr4r[rXrKr5s r6r\zRaggedTensor.from_row_splitssJ h % ""*1. // 3fj5I JO"2226:>> print(tf.RaggedTensor.from_row_lengths( ... values=[3, 1, 4, 1, 5, 9, 2, 6], ... row_lengths=[4, 0, 3, 1, 0])) r:r(RaggedFromRowLengths) row_lengthsrKrSrTN) r,r>r-r rUrfrom_row_lengthsrWrO)rJr4r`rXrKr5s r6razRaggedTensor.from_row_lengthssB h % ""*1. // 4v{6K LO"33!26:0`, then `row_starts[0]` must be zero. name: A name prefix for the RaggedTensor (optional). validate: If true, then use assertions to check that the arguments form a valid `RaggedTensor`. Note: these assertions incur a runtime cost, since they must be checked for each tensor value. Returns: A `RaggedTensor`. `result.rank = values.rank + 1`. `result.ragged_rank = values.ragged_rank + 1`. #### Example: >>> print(tf.RaggedTensor.from_row_starts( ... values=[3, 1, 4, 1, 5, 9, 2, 6], ... row_starts=[0, 4, 4, 7, 8])) r:r(RaggedFromRowStarts) row_startsrFrKrSrTN) r,r>r-r rU _convert_to_ragged_tensor_valuesrfrom_row_startsrCrWrO)rJr4rdrXrKr5s r6rfzRaggedTensor.from_row_startss: h % ""*1. // 3fj5I JO/7f"22v26: 0`, then `row_limits[-1]` must be `nvals`. name: A name prefix for the RaggedTensor (optional). validate: If true, then use assertions to check that the arguments form a valid `RaggedTensor`. Note: these assertions incur a runtime cost, since they must be checked for each tensor value. Returns: A `RaggedTensor`. `result.rank = values.rank + 1`. `result.ragged_rank = values.ragged_rank + 1`. #### Example: >>> print(tf.RaggedTensor.from_row_limits( ... values=[3, 1, 4, 1, 5, 9, 2, 6], ... row_limits=[4, 4, 7, 8, 8])) r:r(RaggedFromRowLimits) row_limitsrKrSrTN) r,r>r-r rUrerfrom_row_limitsrWrO)rJr4rirXrKr5s r6rjzRaggedTensor.from_row_limits)s8 h % ""*1. // 3fj5I JO/7f"2226:>> values = tf.ragged.constant([[1, 2, 3], [4], [5, 6], [7, 8, 9, 10]]) >>> print(values.shape) (4, None) >>> rt1 = tf.RaggedTensor.from_uniform_row_length(values, 2) >>> print(rt1) >>> print(rt1.shape) (2, 2, None) Note that `rt1` only contains one ragged dimension (the innermost dimension). In contrast, if `from_row_splits` is used to construct a similar `RaggedTensor`, then that `RaggedTensor` will have two ragged dimensions: >>> rt2 = tf.RaggedTensor.from_row_splits(values, [0, 2, 4]) >>> print(rt2.shape) (2, None, None) Args: values: A potentially ragged tensor with shape `[nvals, ...]`. uniform_row_length: A scalar integer tensor. Must be nonnegative. The size of the outer axis of `values` must be evenly divisible by `uniform_row_length`. nrows: The number of rows in the constructed RaggedTensor. If not specified, then it defaults to `nvals/uniform_row_length` (or `0` if `uniform_row_length==0`). `nrows` only needs to be specified if `uniform_row_length` might be zero. `uniform_row_length*nrows` must be `nvals`. validate: If true, then use assertions to check that the arguments form a valid `RaggedTensor`. Note: these assertions incur a runtime cost, since they must be checked for each tensor value. name: A name prefix for the RaggedTensor (optional). Returns: A `RaggedTensor` that corresponds with the python list defined by: ```python result = [[values.pop(0) for i in range(uniform_row_length)] for _ in range(nrows)] ``` `result.rank = values.rank + 1`. `result.ragged_rank = values.ragged_rank + 1`. r:r(RaggedFromUniformRowLengthUniformRowLength)uniform_row_lengthrFrRrKrSrTN) r,r>r-r rUre_convert_row_partitionrW_nvals_uniform_row_lengthrfrom_uniform_row_lengthrO)rJr4rnrRrKrXrFr5s r6rqz$RaggedTensor.from_uniform_row_lengthPst h % ""*1. // :!3U; = O/7f1 0 ' /1(0BCe"::/26: r- tensor_libTensorlenr*r rUlistreversedziprV) rJ flat_valuesnested_value_rowids nested_nrowsrXrKresultrArRs r6from_nested_value_rowidsz%RaggedTensor.from_nested_value_rowidssF h % ""*1. //%z'8'89 ++>*?qB CCVc"566l L*"3"3 4--9N!=> > \ c"56 6C< !!B&'( +, , ;k]01>2484F>G Hf!) s& 5 6"8< ,&& L%('<<s /?D88EcVt|tstd|dt|tjrtd|dt j |d|gt|z5|}t|D]}|j|||}|cdddS#1swYyxYw)a(Creates a `RaggedTensor` from a nested list of `row_splits` tensors. Equivalent to: ```python result = flat_values for row_splits in reversed(nested_row_splits): result = from_row_splits(result, row_splits) ``` Args: flat_values: A potentially ragged tensor. nested_row_splits: A list of 1-D integer tensors. The `i`th tensor is used as the `row_splits` for the `i`th ragged dimension. name: A name prefix for the RaggedTensor (optional). validate: If true, then use assertions to check that the arguments form a valid `RaggedTensor`. Note: these assertions incur a runtime cost, since they must be checked for each tensor value. Returns: A `RaggedTensor` (or `flat_values` if `nested_row_splits` is empty). r:r(zAArgument `nested_row_splits` must be a list of Tensors. Received RaggedFromNestedRowSplitsrTN) r,r>r-rtrur rUrwrxr\)rJrznested_row_splitsrXrKr}splitss r6from_nested_row_splitsz#RaggedTensor.from_nested_row_splitss: h % ""*1. //#Z%6%67 ++<*=Q@ AA 9$ ->(?? Af./H&$$VVh$GH   -(BB(cVt|tstd|dt|tjrtd|dt j |d|gt|z5|}t|D]}|j|||}|cdddS#1swYyxYw)a0Creates a `RaggedTensor` from a nested list of `row_lengths` tensors. Equivalent to: ```python result = flat_values for row_lengths in reversed(nested_row_lengths): result = from_row_lengths(result, row_lengths) ``` Args: flat_values: A potentially ragged tensor. nested_row_lengths: A list of 1-D integer tensors. The `i`th tensor is used as the `row_lengths` for the `i`th ragged dimension. name: A name prefix for the RaggedTensor (optional). validate: If true, then use assertions to check that the arguments form a valid `RaggedTensor`. Note: these assertions incur a runtime cost, since they must be checked for each tensor value. Returns: A `RaggedTensor` (or `flat_values` if `nested_row_lengths` is empty). r:r(zBArgument `nested_row_lengths` must be a list of Tensors. Received RaggedFromNestedRowlengthsrTN) r,r>r-rtrur rUrwrxra)rJrznested_row_lengthsrXrKr}lengthss r6from_nested_row_lengthsz$RaggedTensor.from_nested_row_lengthss: h % ""*1. //$j&7&78 ++=*>aA BB :$ -?(@@ Bf01J'%%fg%IJ  rct|tstd|dt|trtd|dt|tj rtd|dt j|d|gt|z5|}t|D]}|j|||}|cdddS#1swYyxYw)a2Creates a `RaggedTensor` from a nested list of row partitions. Equivalent to: ```python result = flat_values for row_partition in reversed(nested_row_partitions): result = _from_row_partition(result, row_partition) ``` Args: flat_values: A potentially ragged tensor. nested_row_partitions: A list of row partitions. The `i`th element is used as the row partition for the `i`th ragged dimension. name: A name prefix for the RaggedTensor (optional). validate: If true, then use assertions to check that the arguments form a valid `RaggedTensor`. Note: these assertions incur a runtime cost, since they must be checked for each tensor value. Returns: A `RaggedTensor` (or `flat_values` if `nested_row_lengths` is empty). r:r(zKArgument `nested_row_partitions` must be a list of RowPartitions. Received RaggedFromNestedRowPartitionsrTN) r,r>r-rrtrur rUrwrxrO)rJrznested_row_partitionsrXrKr}r;s r6_from_nested_row_partitionsz(RaggedTensor._from_nested_row_partitions/s8 h % ""*1. //'6 11F0GqJ KK'):):; 11F0GqJ KK =$ -B(CC Ef 56O)((X(NO  s (B>>Cc t|tstd|dt|tr|jj |j k7rbt js3td|d|j d|jj d|j|j }||fSt|}||fS)aConverts `values` and `partition` to Tensors. If `values` is a `RaggedTensor`, then converts `values` and `partition` to have compatible row-partitioning dtypes. In particular, if any of the row partitioning tensors are `int64`, then all of the other row partitioning tensors wil be cast to `int64` (if auto_cast_partition_dtype() is true) or an error will be raised (if auto_cast_partition_dtype() is false). Args: values: The `values` for the `RaggedTensor` being constructed. row_partition: A RowPartition object for the `RaggedTensor` being constructed. name: The name of the RowPartition object. Returns: A tuple (values, partition). r'r(z4Argument `row_partition` of RaggedTensor with name: z- must have same dtype as Argument `values`. ( vs. ).) r,rr-r$r1r0rauto_cast_partition_dtyper*with_row_splits_dtypere)rJr4r5rXs r6r?z*RaggedTensor._convert_values_and_partition[s( m\ 2 ""/3 44&,'    $ $ (;(; ;668DTFK%%&eF,A,A,G,G+HLM M--m.A.AB M ""07f M ""r8c.|jjS)z%The `DType` of values in this tensor.)r2r0r3s r6r0zRaggedTensor.dtypes <<  r8c|jj}|jj}|jjdd}t j ||gj|S)aThe statically known shape of this ragged tensor. Returns: A `TensorShape` containing the statically known shape of this ragged tensor. Ragged dimensions have a size of `None`. Examples: >>> tf.ragged.constant([[0], [1, 2]]).shape TensorShape([2, None]) >>> tf.ragged.constant([[[0, 1]], [[1, 2], [3, 4]]], ragged_rank=1).shape TensorShape([2, None, 2]) r)N)r1 static_nrowsstatic_uniform_row_lengthr2r.r TensorShape concatenate)r3rRncols value_shapes r6r.zRaggedTensor.shapes\"    , ,E    9 9E,,$$QR(K  # #UEN 3 ? ? LLr8returnc|jS)aThe statically known shape of this ragged tensor. Returns: A `TensorShape` containing the statically known shape of this ragged tensor. Ragged dimensions have a size of `None`. Alias for `shape` property. Examples: >>> tf.ragged.constant([[0], [1, 2]]).get_shape() TensorShape([2, None]) >>> tf.ragged.constant( ... [[[0, 1]], [[1, 2], [3, 4]]], ragged_rank=1).get_shape() TensorShape([2, None, 2]) )r.rs r6 get_shapezRaggedTensor.get_shapes& ::r8cpt|jt}|r|jjdzSdS)aThe number of times the RaggedTensor's flat_values is partitioned. Examples: >>> values = tf.ragged.constant([[1, 2, 3], [4], [5, 6], [7, 8, 9, 10]]) >>> values.ragged_rank 1 >>> rt = tf.RaggedTensor.from_uniform_row_length(values, 2) >>> rt.ragged_rank 2 Returns: A Python `int` indicating the number of times the underlying `flat_values` Tensor has been partitioned to add a new dimension. I.e., `tf.rank(rt) = tf.rank(rt.flat_values) + rt.ragged_rank`. r))r,r2r$ ragged_rank)r3values_is_raggeds r6rzRaggedTensor.ragged_ranks1&"$,, =+;4<< # #a 'BBr8c|jS)aGThe concatenated rows for this ragged tensor. `rt.values` is a potentially ragged tensor formed by flattening the two outermost dimensions of `rt` into a single dimension. `rt.values.shape = [nvals] + rt.shape[2:]` (where `nvals` is the number of items in the outer two dimensions of `rt`). `rt.ragged_rank = self.ragged_rank - 1` Returns: A potentially ragged tensor. #### Example: >>> rt = tf.ragged.constant([[3, 1, 4, 1], [], [5, 9, 2], [6], []]) >>> print(rt.values) tf.Tensor([3 1 4 1 5 9 2 6], shape=(8,), dtype=int32) )r2rs r6r4zRaggedTensor.valuess, <<r8c|jg}|j}t|tr8|j |j|j}t|tr8t |S)z3Returns the row partitions for this `RaggedTensor`.)r1r4r,r$rGtuple)r3 partitions rt_valuess r6_nested_row_partitionsz#RaggedTensor._nested_row_partitionss]%%&J I Y - 001""i Y -  r8c6|jjS)aThe row-split indices for this ragged tensor's `values`. `rt.row_splits` specifies where the values for each row begin and end in `rt.values`. In particular, the values for row `rt[i]` are stored in the slice `rt.values[rt.row_splits[i]:rt.row_splits[i+1]]`. Returns: A 1-D integer `Tensor` with shape `[self.nrows+1]`. The returned tensor is non-empty, and is sorted in ascending order. `self.row_splits[0]` is zero, and `self.row_splits[-1]` is equal to `self.values.shape[0]`. #### Example: >>> rt = tf.ragged.constant([[3, 1, 4, 1], [], [5, 9, 2], [6], []]) >>> print(rt.row_splits) # indices of row splits in rt.values tf.Tensor([0 4 4 7 8 8], shape=(6,), dtype=int64) )r1r[rs r6r[zRaggedTensor.row_splitss*    ) ) ++r8c6|jjS)aThe length of each row in this ragged tensor, or None if rows are ragged. >>> rt1 = tf.ragged.constant([[1, 2, 3], [4], [5, 6], [7, 8, 9, 10]]) >>> print(rt1.uniform_row_length) # rows are ragged. None >>> rt2 = tf.RaggedTensor.from_uniform_row_length( ... values=rt1, uniform_row_length=2) >>> print(rt2) >>> print(rt2.uniform_row_length) # rows are not ragged (all have size 2). tf.Tensor(2, shape=(), dtype=int64) A RaggedTensor's rows are only considered to be uniform (i.e. non-ragged) if it can be determined statically (at graph construction time) that the rows all have the same length. Returns: A scalar integer `Tensor`, specifying the length of every row in this ragged tensor (for ragged tensors whose rows are uniform); or `None` (for ragged tensors whose rows are ragged). )r1rnrs r6rnzRaggedTensor.uniform_row_lengths0    1 1 33r8cx|j}t|tr|j}t|tr|S)aThe innermost `values` tensor for this ragged tensor. Concretely, if `rt.values` is a `Tensor`, then `rt.flat_values` is `rt.values`; otherwise, `rt.flat_values` is `rt.values.flat_values`. Conceptually, `flat_values` is the tensor formed by flattening the outermost dimension and all of the ragged dimensions into a single dimension. `rt.flat_values.shape = [nvals] + rt.shape[rt.ragged_rank + 1:]` (where `nvals` is the number of items in the flattened dimensions). Returns: A `Tensor`. #### Example: >>> rt = tf.ragged.constant([[[3, 1, 4, 1], [], [5, 9, 2]], [], [[6], []]]) >>> print(rt.flat_values) tf.Tensor([3 1 4 1 5 9 2 6], shape=(8,), dtype=int32) r4r,r$)r3rs r6rzzRaggedTensor.flat_values s40 I Y -""i Y - r8c|jg}|j}t|tr8|j |j|j}t|tr8t |S)aJA tuple containing the row_splits for all ragged dimensions. `rt.nested_row_splits` is a tuple containing the `row_splits` tensors for all ragged dimensions in `rt`, ordered from outermost to innermost. In particular, `rt.nested_row_splits = (rt.row_splits,) + value_splits` where: * `value_splits = ()` if `rt.values` is a `Tensor`. * `value_splits = rt.values.nested_row_splits` otherwise. Returns: A `tuple` of 1-D integer `Tensor`s. #### Example: >>> rt = tf.ragged.constant( ... [[[[3, 1, 4, 1], [], [5, 9, 2]], [], [[6], []]]]) >>> for i, splits in enumerate(rt.nested_row_splits): ... print('Splits for dimension %d: %s' % (i+1, splits.numpy())) Splits for dimension 1: [0 3] Splits for dimension 2: [0 3 3 5] Splits for dimension 3: [0 4 4 7 8 8] )r[r4r,r$rGr)r3rt_nested_splitsrs r6rzRaggedTensor.nested_row_splits=s]2( I Y -i223""i Y - ! ""r8ctj|d|g5|jjcdddS#1swYyxYw)aXReturns the row indices for the `values` in this ragged tensor. `rt.value_rowids()` corresponds one-to-one with the outermost dimension of `rt.values`, and specifies the row containing each value. In particular, the row `rt[row]` consists of the values `rt.values[j]` where `rt.value_rowids()[j] == row`. Args: name: A name prefix for the returned tensor (optional). Returns: A 1-D integer `Tensor` with shape `self.values.shape[:1]`. The returned tensor is nonnegative, and is sorted in ascending order. #### Example: >>> rt = tf.ragged.constant([[3, 1, 4, 1], [], [5, 9, 2], [6], []]) >>> print(rt.values) tf.Tensor([3 1 4 1 5 9 2 6], shape=(8,), dtype=int32) >>> print(rt.value_rowids()) # corresponds 1:1 with rt.values tf.Tensor([0 0 0 0 2 2 2 3], shape=(8,), dtype=int64) RaggedValueRowIdsN)r rUr1rAr3rXs r6rAzRaggedTensor.value_rowids]s>0 1D6 :0  - - /000 =AcFtj|d|g5|jg}|j}t |t r<|j |j|j}t |t r>> rt = tf.ragged.constant( ... [[[[3, 1, 4, 1], [], [5, 9, 2]], [], [[6], []]]]) >>> for i, ids in enumerate(rt.nested_value_rowids()): ... print('row ids for dimension %d: %s' % (i+1, ids.numpy())) row ids for dimension 1: [0 0 0] row ids for dimension 2: [0 0 0 2 2] row ids for dimension 3: [0 0 0 0 2 2 2 3] RaggedNestedValueRowIdsN)r rUrAr4r,r$rGr)r3rX rt_nested_idsrs r6r{z RaggedTensor.nested_value_rowidsxs: 7$ @"((*+m++i y, /Y3356$$  y, /= ! """sA)B BB ctj|d|g5|#|jjcdddSt j |jj|cdddS#1swYyxYw)a Returns the number of rows in this ragged tensor. I.e., the size of the outermost dimension of the tensor. Args: out_type: `dtype` for the returned tensor. Defaults to `self.row_splits.dtype`. name: A name prefix for the returned tensor (optional). Returns: A scalar `Tensor` with dtype `out_type`. #### Example: >>> rt = tf.ragged.constant([[3, 1, 4, 1], [], [5, 9, 2], [6], []]) >>> print(rt.nrows()) # rt has 5 rows. tf.Tensor(5, shape=(), dtype=int64) RaggedNRowsNr0)r rUr1rRrrE)r3out_typerXs r6rRzRaggedTensor.nrowssr( mdV 4J  ""((*JJ}}T00668I JJJsA7.A77Bctj|d|g5|jjcdddS#1swYyxYw)aReturns the start indices for rows in this ragged tensor. These indices specify where the values for each row begin in `self.values`. `rt.row_starts()` is equal to `rt.row_splits[:-1]`. Args: name: A name prefix for the returned tensor (optional). Returns: A 1-D integer Tensor with shape `[nrows]`. The returned tensor is nonnegative, and is sorted in ascending order. #### Example: >>> rt = tf.ragged.constant([[3, 1, 4, 1], [], [5, 9, 2], [6], []]) >>> print(rt.values) tf.Tensor([3 1 4 1 5 9 2 6], shape=(8,), dtype=int32) >>> print(rt.row_starts()) # indices of row starts in rt.values tf.Tensor([0 4 4 7 8], shape=(5,), dtype=int64) RaggedRowStartsN)r rUr1rdrs r6rdzRaggedTensor.row_starts>, /$ 8.  + + -...rctj|d|g5|jjcdddS#1swYyxYw)aReturns the limit indices for rows in this ragged tensor. These indices specify where the values for each row end in `self.values`. `rt.row_limits(self)` is equal to `rt.row_splits[:-1]`. Args: name: A name prefix for the returned tensor (optional). Returns: A 1-D integer Tensor with shape `[nrows]`. The returned tensor is nonnegative, and is sorted in ascending order. #### Example: >>> rt = tf.ragged.constant([[3, 1, 4, 1], [], [5, 9, 2], [6], []]) >>> print(rt.values) tf.Tensor([3 1 4 1 5 9 2 6], shape=(8,), dtype=int32) >>> print(rt.row_limits()) # indices of row limits in rt.values tf.Tensor([4 4 7 8 8], shape=(5,), dtype=int64) RaggedRowLimitsN)r rUr1rirs r6rizRaggedTensor.row_limitsrrr)c R|dk(r|jjS|dk(r|jjStj|d|g5t j ||jjd}|dk(r|jcdddS|dk(r |j}|dd|ddz cdddSt|jtr6|j|jj|dz cdddSt j|j|jj}|jt j|d|dz |jj||dz zcdddS#1swYyxYw) a'Returns the lengths of the rows in this ragged tensor. `rt.row_lengths()[i]` indicates the number of values in the `i`th row of `rt`. Args: axis: An integer constant indicating the axis whose row lengths should be returned. name: A name prefix for the returned tensor (optional). Returns: A potentially ragged integer Tensor with shape `self.shape[:axis]`. Raises: ValueError: If `axis` is out of bounds. #### Example: >>> rt = tf.ragged.constant( ... [[[3, 1, 4], [1]], [], [[5, 9], [2]], [[6]], []]) >>> print(rt.row_lengths()) # lengths of rows in rt tf.Tensor([2 0 2 1 0], shape=(5,), dtype=int64) >>> print(rt.row_lengths(axis=2)) # lengths of axis=2 rows. rr)RaggedRowLengths rank(self)) ndims_nameNr)r1rRr`r rUrget_positive_axisr.rankr[r,r4r$ with_valuesr0ones)r3axisrXrr.s r6r`zRaggedTensor.row_lengthssm6 qy  & & (( qy  , , .. 04& 9  ( (  L:d zz|  19abzF3BK' dkk< 0 7 7q AB d6I6I6O6OP NN5$(+T-@-@-F-F G $(O s!AF"FAFA* A>>Bc z||jj}ntj|}t j |d||g5|j }|j}t|tr|dk(r)tj|d|ddz cdddS|dk(rqtjtj|jd}||jjk7rtj ||}|cdddStj|j"|}tj||}|ddz g|D cgc]4} tjtj| dd| ddz d6c} z} |dd} ||jjk7r#| D cgc]} tj | |} } tj$t'j(| | gd} || ntj*| |cdddScc} wcc} w#1swYyxYw)aReturns the tight bounding box shape for this `RaggedTensor`. Args: axis: An integer scalar or vector indicating which axes to return the bounding box for. If not specified, then the full bounding box is returned. name: A name prefix for the returned tensor (optional). out_type: `dtype` for the returned tensor. Defaults to `self.row_splits.dtype`. Returns: An integer `Tensor` (`dtype=self.row_splits.dtype`). If `axis` is not specified, then `output` is a vector with `output.shape=[self.shape.ndims]`. If `axis` is a scalar, then the `output` is a scalar. If `axis` is a vector, then `output` is a vector, where `output[i]` is the bounding size for dimension `axis[i]`. #### Example: >>> rt = tf.ragged.constant([[1, 2, 3, 4], [5], [], [6, 7, 8, 9], [10]]) >>> rt.bounding_shape().numpy() array([5, 4]) NRaggedBoundingBoxrrr)rr)r1r0r as_dtyper rUrrzr,intrr.rmaximum reduce_maxr`rEr[concatrstackgather)r3rrXr nested_splitsrt_flat_valuesr} splits_shapeflat_values_shaperragged_dimensionsinner_dimensionsdbboxs r6bounding_shapezRaggedTensor.bounding_shape/s!2$$**h*h 1D$< @D,,m''n D#  19q!1HEaH1LDDQY##H$7$78H8H8J$KQO& ,,22 2]]684FDD__T__xHl#//.8L'?Q./%3   8..vabzF3BK/GH! L3+12. T((.. .0A +,HMM!X &      !2 35E FQPd\Ty'7'7d'C;DD"3 1DDs:A H1A,H1AH19H''H19H,AH1' H11H:c:t|}|jjd|jjddj |jddt |t r|jj|jjk7rltjs td|jtj}|jtjj!|St ||jdS)aReturns a copy of `self` with `values` replaced by `new_value`. Preserves cached row-partitioning tensors such as `self.cached_nrows` and `self.cached_value_rowids` if they have values. Args: new_values: Potentially ragged tensor to use as the `values` for the returned `RaggedTensor`. Must have `rank > 0`, and must have the same number of rows as `self.values`. Returns: A `RaggedTensor`. `result.rank = 1 + new_values.rank`. `result.ragged_rank = 1 + new_values.ragged_rank` r)Nzself and new_values have mismatched row_splits dtypes; use RaggedTensor.with_row_splits_dtype() to convert them to compatible dtypes.Tr4r5r )rer.r/r4rBr,r$r1r0r[rrr*rr int64rr3 new_valuess r6rzRaggedTensor.with_valuesos2*=J''*KKbq33J4D4DRa4HI:|, !!Z%:%:%@%@@  4 4 6>? ?33FLLAj  ' ' 5 A A* MM )<) 0`, and must have the same number of rows as `self.flat_values`. Returns: A `RaggedTensor`. `result.rank = self.ragged_rank + new_values.rank`. `result.ragged_rank = self.ragged_rank + new_values.ragged_rank`. )r,r2r$rr4with_flat_valuesrers r6rzRaggedTensor.with_flat_valuessM $,, -   dkk:::F GG3J?j   J ''r8ctj|}|tjtjfvrt d|d|j j |k(r|S|j}t|tr6t|j||j j|dSt||j j|dS)aReturns a copy of this RaggedTensor with the given `row_splits` dtype. For RaggedTensors with multiple ragged dimensions, the `row_splits` for all nested `RaggedTensor` objects are cast to the given dtype. Args: dtype: The dtype for `row_splits`. One of `tf.int32` or `tf.int64`. Returns: A copy of this RaggedTensor, with the `row_splits` cast to the given type. z=Argument `row_splits` dtype must be int32 or int64. Received r(Tr) r rint32rr*r1r0r2r,r$r with_dtype)r3r0current_valuess r6rz"RaggedTensor.with_row_splits_dtypes OOE "E V\\6<<00 ##(', --   E) k\\N.,/ 55e<++66u= ++66u=r8ctj||jjdd}tj||jjdd}||kst d|d|dt |||S)aMerges outer_axis...inner_axis into a single dimension. Returns a copy of this RaggedTensor with the specified range of dimensions flattened into a single dimension, with elements in row-major order. #### Examples: >>> rt = tf.ragged.constant([[[1, 2], [3]], [[4, 5, 6]]]) >>> print(rt.merge_dims(0, 1)) >>> print(rt.merge_dims(1, 2)) >>> print(rt.merge_dims(0, 2)) tf.Tensor([1 2 3 4 5 6], shape=(6,), dtype=int32) To mimic the behavior of `np.flatten` (which flattens all dimensions), use `rt.merge_dims(0, -1)`. To mimic the behavior of `tf.layers.Flatten` (which flattens all dimensions except the outermost batch dimension), use `rt.merge_dims(1, -1)`. Args: outer_axis: `int`: The first dimension in the range of dimensions to merge. May be negative if `self.shape.rank` is statically known. inner_axis: `int`: The last dimension in the range of dimensions to merge. May be negative if `self.shape.rank` is statically known. Returns: A copy of this tensor, with the specified dimensions merged into a single dimension. The shape of the returned tensor will be `self.shape[:outer_axis] + [N] + self.shape[inner_axis + 1:]`, where `N` is the total number of slices in the merged dimensions. outer_axisr) axis_namer inner_axiszExpected outer_axis (z*) to be less than or equal to inner_axis (r)rrr.rr* merge_dims)r3rrs r6rzRaggedTensor.merge_dimssB,,  !J ,,  !J  # .zl;//9l"> ?? dJ 33r8c 2tj|}|jy|j}|d+|jj j |ddz|jj}t|jD]\}}||dz}||j?tj|j}|||k(rDtd|dzd|d|dtj|||_ |j t#j$|j |dz |_t'|j(d rEtjdg||j*dzdz}|j(j |yy) aUpdates the static shape of `self` to be `shape`. * If a dimension of `shape` has known rank, and is encoded via partitioning, then this will update the corresponding partition to define `_uniform_row_length` and `nrows`. * If a dimension of `shape` has a known rank, and is encoded as one of the `flat_values` dimensions, then `flat_values.set_shape()` will be used to update its shape. Warning: Using this method to assert an incorrect shape for a RaggedTensor (i.e., one that's not consistent with its actual shape) can cause segmentation faults and very difficult-to-diagnose behavior. Only use this method if you are certain that the shape is correct. Args: shape: `tf.TensorShape` specifying the shape for this `RaggedTensor`. Nrr)zInconsistent size for axis : rr(r set_shape)ras_shaperas_listr1 _row_splitsrr0 enumerater_uniform_row_lengthrconstant_valuer*r convert_to_tensorrCrsizehasattrrzr)r3r.r0ir;rold_row_length flat_shapes r6 _set_shapezRaggedTensor._set_shapes,  ! !% (E zz MMOE Qx %%//a1 =    % %E!$"="=>9 9 1q5\d   ( ( 4&55++-.  '~%!d |!tUjT|d kD !fd fd}|jWtAjXddg|jW|jdd t#jZ|}t#j\||}|tj^t#j>d |d zdz}t#j`|d }|tcjd|d|}|jjgd t#jh||}t#jj|d}t#jl|}tj8tjnd g||gd}tjp||}tj.||}|js||dcdddStj8|d|d zg|d dgd}tj:||}tAjB|jdjD}tAjB|jd jD}|tGjH||}n|d}|tGjH||}n|d }tJjM|||dcdddS#1swYyxYw)a Converts a `tf.Tensor` into a `RaggedTensor`. The set of absent/default values may be specified using a vector of lengths or a padding value (but not both). If `lengths` is specified, then the output tensor will satisfy `output[row] = tensor[row][:lengths[row]]`. If 'lengths' is a list of lists or tuple of lists, those lists will be used as nested row lengths. If `padding` is specified, then any row *suffix* consisting entirely of `padding` will be excluded from the returned `RaggedTensor`. If neither `lengths` nor `padding` is specified, then the returned `RaggedTensor` will have no absent/default values. Examples: >>> dt = tf.constant([[5, 7, 0], [0, 3, 0], [6, 0, 0]]) >>> tf.RaggedTensor.from_tensor(dt) >>> tf.RaggedTensor.from_tensor(dt, lengths=[1, 0, 3]) >>> tf.RaggedTensor.from_tensor(dt, padding=0) >>> dt = tf.constant([[[5, 0], [7, 0], [0, 0]], ... [[0, 0], [3, 0], [0, 0]], ... [[6, 0], [0, 0], [0, 0]]]) >>> tf.RaggedTensor.from_tensor(dt, lengths=([2, 0, 3], [1, 1, 2, 0, 1])) Args: tensor: The `Tensor` to convert. Must have rank `ragged_rank + 1` or higher. lengths: An optional set of row lengths, specified using a 1-D integer `Tensor` whose length is equal to `tensor.shape[0]` (the number of rows in `tensor`). If specified, then `output[row]` will contain `tensor[row][:lengths[row]]`. Negative lengths are treated as zero. You may optionally pass a list or tuple of lengths to this argument, which will be used as nested row lengths to construct a ragged tensor with multiple ragged dimensions. padding: An optional padding value. If specified, then any row suffix consisting entirely of `padding` will be excluded from the returned RaggedTensor. `padding` is a `Tensor` with the same dtype as `tensor` and with `shape=tensor.shape[ragged_rank + 1:]`. ragged_rank: Integer specifying the ragged rank for the returned `RaggedTensor`. Must be greater than zero. name: A name prefix for the returned tensors (optional). row_splits_dtype: `dtype` for the returned `RaggedTensor`'s `row_splits` tensor. One of `tf.int32` or `tf.int64`. Returns: A `RaggedTensor` with the specified `ragged_rank`. The shape of the returned ragged tensor is compatible with the shape of `tensor`. Raises: ValueError: If both `lengths` and `padding` are specified. ValueError: If the rank of `tensor` is 0 or 1. Nz6Specify argument `lengths` or `padding`, but not both.0Argument `ragged_rank` must be an int. Received r(rz8Argument `ragged_rank` must be greater than 0. Received RaggedFromTensorr rXzThe rank of a RaggedTensor must be greater than 1, i.e., a list of scalars won't have ragged dimensions. Received argument `tensor` with rank r)rz^If Argument `lengths` is a tuple of row_lengths, argument `ragged_rank` must be len(lengths): z. Received ragged_rank: rrFrT) default_valuer)row_splits_dtype)r4rnrRrKpaddingrXr0c2tjS)Nr)r reduce_all)has_default_value reduce_axissr6z*RaggedTensor.from_tensor..sH''(9 Lr8cSN)r sr6r z*RaggedTensor.from_tensor..s%r8r)maxlen):r rr*r,rr-r rUrr.rr/rrwrrvfloatr reduce_sumrr>r to_tensor boolean_maskis_fully_definedrnpcumprodrreshape from_tensorrangerdimension_at_indexvaluerconstantr$rqr0rBequalrrr logical_notrE expand_dimsrrconvert_to_int_tensorassert_has_rankminimumrcumsumzeros sequence_maskr\)"rJr rrrrXr input_shaper num_tokens ones_mask ragged_maskdense_ragged_mask masked_datadim_size new_shape flattenedr}rdim_len tensor_rank has_defaulthas_nondefaultlength_for_nondefault_valuelimitsrmaskr4 values_shape const_nrows const_ncolsrRr r s" @@r6rzRaggedTensor.from_tensor.sB'78w2 O PP k3 ' ""-a1 22a ##.-q2 33 067G2L MKP$$V(;f    &6<<+<+:++6-q:; ;  ''G q(89((5 NNJ B C $%%&AJ  33GY4DF %%a(""7E2""7A.)!! __aS"2 3V <1F&&wu=''5""66E"BqKPKPz%% N[^ + ,k!"o >QHl  6f 33FLL!DJJk 33FLL!DJJk  $$[2BCA  $$[2BCA  1 1E2PUKPKPKPs"4G[?D?[I[C$[[ctj|d|||g5|"tj|d|j}t |}|Dcgc]}|d }}|Dcgc]}|d }}| t j d|j}t|ttfr9td|Dr'td |Drtj|}t||dj}tj ||j"||| } |j$} | j&t|t(j*sst-j.|}|j&| } n>t1|j3| j3D cgc] \} } | | n|  } } } | j5| | cdddScc}wcc}wcc} } w#1swYyxYw) aConverts this `RaggedTensor` into a `tf.Tensor`. If `shape` is specified, then the result is padded and/or truncated to the specified shape. Examples: >>> rt = tf.ragged.constant([[9, 8, 7], [], [6, 5], [4]]) >>> print(rt.to_tensor()) tf.Tensor( [[9 8 7] [0 0 0] [6 5 0] [4 0 0]], shape=(4, 3), dtype=int32) >>> print(rt.to_tensor(shape=[5, 2])) tf.Tensor( [[9 8] [0 0] [6 5] [4 0] [0 0]], shape=(5, 2), dtype=int32) Args: default_value: Value to set for indices not specified in `self`. Defaults to zero. `default_value` must be broadcastable to `self.shape[self.ragged_rank + 1:]`. name: A name prefix for the returned tensors (optional). shape: The shape of the resulting dense tensor. In particular, `result.shape[i]` is `shape[i]` (if `shape[i]` is not None), or `self.bounding_shape(i)` (otherwise).`shape.rank` must be `None` or equal to `self.rank`. Returns: A `Tensor` with shape `ragged.bounding_shape(self)` and the values specified by the non-empty values in `self`. Empty values are assigned `default_value`. RaggedToTensorNrrrr)rc3PK|]}t|tj ywr)r,rtru.0vs r6 z)RaggedTensor.to_tensor..0s >1jJ--. >$&c3\K|]$}t|ttjf&ywr)r,rrtrur>s r6rAz)RaggedTensor.to_tensor..1s" E!jS*"3"345 Es*,)r.r4rrow_partition_typesrow_partition_tensors)r rUrr0$_get_row_partition_type_tensor_pairsrr&r,rwranyallrr_shape_as_tensorrragged_tensor_to_tensorrzr.rrtrurrryrr)r3rrXr.type_tensor_pairsxrDrE shape_tensorr ragged_shape output_shapes1s2s r6rzRaggedTensor.to_tensors> .}e0L M+  "-- tzzC >tD+<=aQqT==->?qt??  !DJJ7 UT5M * > > > Eu E E%%e,%e-B1-E-K-KLl(@@!!%1 5 fZZl    &z ""0 %%e, :: %, "%--/<3G3G3IJ 2rNb* ,  & W++ >?> K++s64G G G! G-D1GG.G GG&ctj|}tj|s"t dt |j dtj|d|g5tj|d}|jjjd}n-|jjjdj}|jjjd}n-|jjjdj}|d k7r|d k7r t!d tj"t%|j5t'j(|jdddf|}t'j(|jd|}|j+|j,||d cdddcdddS#1swYnxYw dddy#1swYyxYw) aConverts a 2D `tf.sparse.SparseTensor` to a `RaggedTensor`. Each row of the `output` `RaggedTensor` will contain the explicit values from the same row in `st_input`. `st_input` must be ragged-right. If not it is not ragged-right, then an error will be generated. Example: >>> indices = [[0, 0], [0, 1], [0, 2], [1, 0], [3, 0]] >>> st = tf.sparse.SparseTensor(indices=indices, ... values=[1, 2, 3, 4, 5], ... dense_shape=[4, 3]) >>> tf.RaggedTensor.from_sparse(st).to_list() [[1, 2, 3], [4], [], [5]] Currently, only two-dimensional `SparseTensors` are supported. Args: st_input: The sparse tensor to convert. Must have rank 2. name: A name prefix for the returned tensors (optional). row_splits_dtype: `dtype` for the returned `RaggedTensor`'s `row_splits` tensor. One of `tf.int32` or `tf.int64`. Returns: A `RaggedTensor` with the same values as `st_input`. `output.ragged_rank = rank(st_input) - 1`. `output.shape = [st_input.dense_shape[0], None]`. Raises: ValueError: If the number of dimensions in `st_input` is not known statically, or is not two. zAArgument `st_input` must be of type SparseTensor, but is of type r(RaggedFromSparsest_inputrNrr)rzrank(st_input) must be 2.FrT)r rr is_sparser-type__name__r rU"convert_to_tensor_or_sparse_tensor dense_shaper.ndimsdimsrindicesr*control_dependencies'_assert_sparse_indices_are_ragged_rightrrErVr4)rJrTrXrstatic_rank_from_dense_shapestatic_rank_from_indices segment_ids num_segmentss r6 from_sparsezRaggedTensor.from_sparseRsD'78  " "8 , $$(N$;$;##>q#A#G#G % */G1/L455  # # 1(2B2B C EH mmH$4$4QT$:NO $$ OO[,%HHH#HH"HHH#HHHs&%C5G,A)G G,G G,,G5c tj|d|g5tj|j|j |}t j|j|j|jcdddS#1swYyxYw)a^Converts this `RaggedTensor` into a `tf.sparse.SparseTensor`. Example: >>> rt = tf.ragged.constant([[1, 2, 3], [4], [], [5, 6]]) >>> print(rt.to_sparse()) SparseTensor(indices=tf.Tensor( [[0 0] [0 1] [0 2] [1 0] [3 0] [3 1]], shape=(6, 2), dtype=int64), values=tf.Tensor([1 2 3 4 5 6], shape=(6,), dtype=int32), dense_shape=tf.Tensor([4 3], shape=(2,), dtype=int64)) Args: name: A name prefix for the returned tensors (optional). Returns: A SparseTensor with the same values as `self`. RaggedToSparserN) r rUrragged_tensor_to_sparserrzr SparseTensorsparse_indices sparse_valuessparse_dense_shape)r3rXr}s r6 to_sparsezRaggedTensor.to_sparses|& . 7C(@@  $"2"2?f  ' '(=(=(.(<(<(.(A(ACCCCs A!BB c tj|dtj}|jj G|E|||jj zk7r)t d|d|d|jj d|dn|}tj|d ||||g5tj||t|d |||}|j|j|jd cdddS#1swYyxYw) aConverts a `variant` Tensor into a `RaggedTensor`. The input `variant` could be a scalar, meaning it encodes a single `RaggedTensor` with ragged_rank `output_ragged_rank`. Alternatively it could have an arbitrary rank, in which case each element is decoded into a `RaggedTensor` with ragged_rank `input_ragged_rank` and these are then stacked according to the input shape to output a single `RaggedTensor` with ragged_rank `output_ragged_rank`. If `input_ragged_rank` is not provided, it is inferred dynamically as `output_ragged_rank` - `rank(variant)`. If `input_ragged_rank` is provided, the following must be true: `output_ragged_rank` = `input_ragged_rank` + `rank(variant)`. Example: >>> rt = tf.ragged.constant([[0], [1, 2]]) >>> et = rt._to_variant() >>> stacked_et = tf.stack([et, et]) >>> tf.RaggedTensor._from_variant( # scalar input. ... et, dtype=tf.int32, output_ragged_rank=1).to_list() [[0], [1, 2]] >>> tf.RaggedTensor._from_variant( # batched input. ... stacked_et, dtype=tf.int32, output_ragged_rank=2).to_list() [[[0], [1, 2]], [[0], [1, 2]]] Args: variant: A `variant` Tensor representing an encoded (possibly nested-batched) `RaggedTensor`. dtype: The dtype of the encoded `RaggedTensor`. output_ragged_rank: The expected ragged rank of the output `RaggedTensor`. input_ragged_rank: The ragged rank of each encoded `RaggedTensor`. This is optional and inferred dynamically if not provided. row_splits_dtype: `dtype` for the RaggedTensor's `row_splits` tensor. One of `tf.int32` or `tf.int64`. name: A name prefix for the returned tensors (optional). Returns: A `RaggedTensor` of dtype `dtype` and ragged rank `output_ragged_rank`. Raises: ValueError: If the input rank is known, `input_ragged_rank` is provided and `output_ragged_rank` = `input_ragged_rank` + `rank(variant)` does not hold. variantrNzArgument `output_ragged_rank` (z@) must be equal to `input_ragged_rank` + `variant.shape.ndims` (z + rrRaggedFromVariantrFrT)r rr rmr.rZr*rUrragged_tensor_from_variantmaxroutput_dense_valuesoutput_nested_splits)rJrmr0output_ragged_rankinput_ragged_rankrrXr}s r6 _from_variantzRaggedTensor._from_variants!f##iv~~7G ',=,I/'--2E2EEE  +,>+?@ GMM$7$7#8 < ==07=N  ! %*,>? A )CC $c*>> rt = tf.ragged.constant([[[0]], [[1]], [[2]]]) >>> rt._to_variant().shape.as_list() [] >>> rt._to_variant(batched_input=True).shape.as_list() [3] Args: batched_input: If `True`, the `RaggedTensor` is unbatched and converted to a `variant` vector. Set to `False` by default. name: A name prefix for the returned tensors (optional). Returns: A `variant` Tensor that encodes this `RaggedTensor`. RaggedToVariantN)r rUrragged_tensor_to_variantrrz)r3 batched_inputrXs r6 _to_variantzRaggedTensor._to_variantsW0 /$ 1F GI & ? ?  $"2"2M4IIIIs ,AAc|jrDtjdti5t|j }dddddSd|j d|j dS#1swY,xYw)NrH) formatterzztf.RaggedTensor(values=z , row_splits=)) _is_eagerr printoptions _formatternumpyr4r[)r3 value_texts r6__repr__zRaggedTensor.__repr__su ~~ ??eZ%8 9. - .  A ..=AKK<@OOMM ..s A22A;c|js td|jj}|jj}t t |dz Dcgc]}|||||dzc}s1tjd|jddz|jStfdD}|rtjnd}tj|Scc}w)aReturns a numpy `array` with the values for this `RaggedTensor`. Requires that this `RaggedTensor` was constructed in eager execution mode. Ragged dimensions are encoded using numpy `arrays` with `dtype=object` and `rank=1`, where each element is a single row. #### Examples In the following example, the value returned by `RaggedTensor.numpy()` contains three numpy `array` objects: one for each row (with `rank=1` and `dtype=int64`), and one to combine them (with `rank=1` and `dtype=object`): >>> tf.ragged.constant([[1, 2, 3], [4, 5]], dtype=tf.int64).numpy() array([array([1, 2, 3]), array([4, 5])], dtype=object) Uniform dimensions are encoded using multidimensional numpy `array`s. In the following example, the value returned by `RaggedTensor.numpy()` contains a single numpy `array` object, with `rank=2` and `dtype=int64`: >>> tf.ragged.constant([[1, 2, 3], [4, 5, 6]], dtype=tf.int64).numpy() array([[1, 2, 3], [4, 5, 6]]) Returns: A numpy `array`. z5RaggedTensor.numpy() is only supported in eager mode.r))rrNrc3RK|]}t|tdk7 yw)rN)rv)r?rowrowss r6rAz%RaggedTensor.numpy..Qs""L3s8s47|#;"Ls$')rr*r4rr[rrvrr&r.r0rGobject_array)r3r4rrhas_variable_length_rowsr0rs @r6rzRaggedTensor.numpy)s6 >>  N OO [[   F __ " " $F5:3v;?5K LF6!9VAE] + LD  XXfv||AB//v|| DD #"Lt"LL2BJJE 88D && Ms*C4ct|jtjs t d|jj j }|j}t|tr?tt|dz Dcgc]}|||||dzj c}St|dr|j j }n(t|dr|j}n t dtt|dz Dcgc]}|||||dzc}Scc}wcc}w)zReturns a nested Python `list` with the values for this `RaggedTensor`. Requires that `rt` was constructed in eager execution mode. Returns: A nested Python `list`. z'to_list can only be used in eager mode.r)rto_listz$values must be convertible to a list) r,r[r EagerTensorr*rtolistr4r$rrvrr)r3r[r4rvalues_as_lists r6rzRaggedTensor.to_listUs# doos 7 @ AA&&(//1J [[F&,'Z1,- Az!a%0 1 9 9 ;  !..0 69 %)?@@Z1,- Az!a%'8 9s #D7D<c|jj}t|jD]&}t j ||j}(|S)zFReturns a RaggedTensorValue for self. Requires self._is_eager()=true.)rzrrxrrRaggedTensorValue)r3rr[s r6 _eager_valuezRaggedTensor._eager_valueusR    " " $Et556O !33E:;K;K;MNeO Lr8c|}t|trBt|jtjsy|j }t|trBt|tjS)zCReturns True if values & row_splits Tensors are all `EagerTensor`s.F)r,r$r[r rr4)r3rs r6rzRaggedTensor._is_eager|sN B R &  s 7 99b R & b#// **r8cfd}|S)Nc&~~tdd)NzTYou must import 'tensorflow.python.ops.ragged.ragged_ops' before using RaggedTensor.r()r*)argskwargsrXs r6stubz/RaggedTensor._overloaded_operator..stubs&   ''+fA / 00r8r)rXrs` r6_overloaded_operatorz!RaggedTensor._overloaded_operators0 Kr8 __getitem____ge____gt____le____lt____and____rand__ __invert____ror____or____xor____rxor____abs____add____radd____div____rdiv__ __floordiv__ __rfloordiv____mod____rmod____mul____rmul____neg____pow____rpow____sub____rsub__ __truediv__ __rtruediv__cx|j}t|tr|j}t|tr|S)z"Convert `self` to a graph element.r)r3r4s r6_as_graph_elementzRaggedTensor._as_graph_elements/ [[F V\ *}}f V\ * Mr8c,tj|Sr)RaggedTensorSpec from_valuers r6 _type_speczRaggedTensor._type_specs  & &t ,,r8cnt||j|j|jjSr)rr0rr[)r3r.s r6_shape_invariant_to_type_specz*RaggedTensor._shape_invariant_to_type_specs, E4::t/?/? OO11 33r8c"|jSr) _consumersrs r6 consumerszRaggedTensor.consumerss ?? r8)F)T)NNT)NT)NTNr)NN)r)NNNN)FN)crW __module__ __qualname____doc__r#do_not_generate_docsr7 classmethodrOr!add_dispatch_supportrVr\rarfrjrqr~rrrr?propertyr0r.rrrrr4rr[rnrzrrAr{rRrdrir`rrrrrrrr rrrrcrkrurzrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr!WithValuesCompositeTensorGradient__composite_gradient__rr8r6r$r$As) z~$$-(%-(f+J+JZ   #!!% 8O!8Ot   ,O!,O\   (O!(OT   %O!%ON   #O!#OJ   %)'+#' HO!HOT   -1$((, ;!;z   #'&* &!&P   $('+ &!&P(,+/ ))V$#$#T   M M*11* C C*  .   , ,, 4 42  8 # #>06#"JJ4.2.2/b**:D@M:(,>.4`7-z   #)<< TP!TPlJX   &*V\\=H!=H~C4 '+%+\\ DDLI>M,*'X@+%]3+  )&  )&  )&  )&  +' !* -(#L1*  +'  )&  +' !* -(  +'  +' !* -(  +' !* -(%n5,&7-  +' !* -(  +' !* -(  +'  +' !* -(  +' !* -($]3+%n5, - -3BAACr8cBt|ttjfS)zBReturns true if `value` is a ragged tensor or ragged tensor value.)r,r$rrrs r6 is_raggedrs" E!#6#H#HI KKr8c|jdd}|rtd|dd}d}|D]?}t|ts|jj t jk(rd}>d}A|rD|rBtjs tdt j}td|D}n#|rt j}nt j}|r||fS|S)aReturn a copy of `tensors` with row_splits all having the same dtype. Args: *tensors: A list of Tensors or RaggedTensors. **kwargs: If 'return_dtype=True', then return a tuple (dtype, tensors), where `dtype` is the data type used by row-splits, and `tensors` is the converted list of `Tensors` and `RaggedTensors`. Returns: The converted list of `Tensors` and `RaggedTensors`. return_dtypeFzUnexpected keyword args r(TzInput RaggedTensors have mismatched row_splits dtypes; use RaggedTensor.with_row_splits_dtype() to convert them to compatible dtypes.c3~K|]5}t|tr|jtjn|7ywr)r,r$rr rr?ts r6rAz*match_row_splits_dtypes..s= 2>> rt = tf.ragged.constant([["a"], ["b", "c"]], dtype=tf.string) >>> tf.type_spec_from_value(rt).dtype tf.string Returns: A `tf.dtypes.DType` of the values in the RaggedTensor. rrs r6r0zRaggedTensorSpec.dtype s ;;r8c|jS)aThe statically known shape of the RaggedTensor. Examples: >>> rt = tf.ragged.constant([[0], [1, 2]]) >>> tf.type_spec_from_value(rt).shape TensorShape([2, None]) >>> rt = tf.ragged.constant([[[0, 1]], [[1, 2], [3, 4]]], ragged_rank=1) >>> tf.type_spec_from_value(rt).shape TensorShape([2, None, 2]) Returns: A `tf.TensorShape` containing the statically known shape of the RaggedTensor. Ragged dimensions have a size of `None`. rrs r6r.zRaggedTensorSpec.shape) s$ ;;r8c|jS)aZThe number of times the RaggedTensor's flat_values is partitioned. Defaults to `shape.ndims - 1`. Examples: >>> values = tf.ragged.constant([[1, 2, 3], [4], [5, 6], [7, 8, 9, 10]]) >>> tf.type_spec_from_value(values).ragged_rank 1 >>> rt1 = tf.RaggedTensor.from_uniform_row_length(values, 2) >>> tf.type_spec_from_value(rt1).ragged_rank 2 Returns: A Python `int` indicating the number of times the underlying `flat_values` Tensor has been partitioned to add a new dimension. I.e., `tf.rank(rt) = tf.rank(rt.flat_values) + rt.ragged_rank`. rrs r6rzRaggedTensorSpec.ragged_rank= s*   r8c|jS)aXThe `tf.dtypes.DType` of the RaggedTensor's `row_splits`. Examples: >>> rt = tf.ragged.constant([[1, 2, 3], [4]], row_splits_dtype=tf.int64) >>> tf.type_spec_from_value(rt).row_splits_dtype tf.int64 Returns: A `tf.dtypes.DType` for the RaggedTensor's `row_splits` tensor. One of `tf.int32` or `tf.int64`. rrs r6rz!RaggedTensorSpec.row_splits_dtypeT s  ! !!r8c|jS)zThe `TypeSpec` of the flat_values of RaggedTensor. Returns: - The TypeSpec of flat_values. - None when the flat_values is a Tensor. )rrs r6flat_values_specz!RaggedTensorSpec.flat_values_specd s  ! !!r8cL|jdkDrtStjSNr)rr$rtrurs r6 value_typezRaggedTensorSpec.value_typen s ,,q0<Gj6G6GGr8Nctj||_tj||_|)| |j }n'||j k7rtd| tdtj||_||_ |jj}|| td|dz }||_ t|jtstd|d|||k\rtd|d |d yy) aOConstructs a type specification for a `tf.RaggedTensor`. Args: shape: The shape of the RaggedTensor, or `None` to allow any shape. If a shape is specified, then all ragged dimensions must have size `None`. dtype: `tf.DType` of values in the RaggedTensor. ragged_rank: Python integer, the number of times the RaggedTensor's flat_values is partitioned. Defaults to `shape.ndims - 1`. row_splits_dtype: `dtype` for the RaggedTensor's `row_splits` tensor. One of `tf.int32` or `tf.int64`. flat_values_spec: TypeSpec for flat_value of the RaggedTensor. It shall be provided when the flat_values is a CompositeTensor rather then Tensor. If both `dtype` and `flat_values_spec` and are provided, `dtype` must be the same as `flat_values_spec.dtype`. (experimental) Nz0dtype must be the same as flat_values_spec.dtypez:At least one of dtype or flat_values_spec must be providedz6Must specify ragged_rank or a shape with a known rank.r)rr(zArgument `ragged_rank` (z) must be less than rank (r)rrrr rrr0r*rrrZrr,rr-)r3r.r0rrrrs r6r7zRaggedTensorSpec.__init__r s3*''.DK#__-=>D#  && $** *KLL   F HH//%(DK-D ;;  D 67 71Hk#D d'' - ""-a1 22   3K=A''+fB01 1 r8c|jdk(r|jct|tjtj frjtj |j |jj|St|ttfs|jj|Stt|#|Sr) rrr,rtru TensorSpecrris_compatible_withr$rsuper)r3 spec_or_value __class__s r6rz#RaggedTensorSpec.is_compatible_with s A   '  J--z/D/DE G&&kk4;;((:(:=(I J-,8H)IJ%%88GG !4 ;M JJr8c|j.|j|j|j|jfS|j|j|j|j|jfSr)rrrrrrs r6 _serializezRaggedTensorSpec._serialize sh %kk4;;(9(9$$&&kk4;;(9(9$$d&<&<>>r8c|jdkrD|j |jgStj|j|j gS|j}|_t jdgj|j|jdzd}tj||j }t j|jd}|dn|dzg}tjdg|j}|tj||jgt|jdz Dcgc]}|c}z}|Scc}w)Nrr)) rrrtrrrrrrrrr)r3rr outer_dimouter_splits_shapeinner_splits_spec_specss r6_component_specsz!RaggedTensorSpec._component_specs sG A   +&&''%%dkk4;;?@@--&22D6:FF ++d''!+, -/#../@$++N// Q?I"+"3$QG"--tfd6L6LM 0$2H2HI%*$*;*;a*?$@Aq ABE L Bs Ecdt|r#|jgt|jzS|gSr)rrzrwrr3rs r6_to_componentszRaggedTensorSpec._to_components s0 4(?(?#@ @@Wnr8c@|d}td|Dr>tjs*t|ddD]}t j ||}nzt |dtjr'|Dcgc]}tj|}}|d}t|ddD]%}t|tj|dd}'|jjt |trn|j!|j|j"Et%|j&dr/|j&j)|j"j*|St |t,j.r|j)|j|Scc}w) Nrc3PK|]}t|tj ywr)r,rndarrayrs r6rAz4RaggedTensorSpec._from_components.. s ;!Jq"** % ;rBr)FrTTrr)rHrenabledrxrrr,rrr rr$rr\rrZrrrrzrr.rtru)r3 tensor_listr}r[rs r6_from_componentsz!RaggedTensorSpec._from_components sh ^F ;{ ;; KKM QR1K*$66vzJK KNBJJ /9DEAs,,Q/E EQ QR1*   ( (e D  {{$ FL )$++&  ,9K9K9D2F    & &t'<'<'B'B C M fj// 0% M!Fs7FcLtjdtjgSr)rtrr rmrs r6_flat_tensor_specsz#RaggedTensorSpec._flat_tensor_specs s  ! !$ 7 88r8c|j tdt|trR|j|j k7r&td|jd|j d|j dgS|j dkDr/td|j d t|jdtjd |dgS) N'Customized value_type is not supported.Ragged rank of value $ does not match ragged rank of type r(Fryr'Expected a RaggedTensor if ragged rank= but got r rr*r,r$rrrzrVrWrrxrs r6_to_tensor_listz RaggedTensorSpec._to_tensor_list s ) @ AA%&   d// /#E$5$5#67##'#4#4"5Q 89 9e4 55  Q 5d6G6G5HU ,,-Q 0  $ < <%u .r8c|j tdt|trR|j|j k7r&td|jd|j d|j dgS|j dkDr/td|j d t|jdtjd |d gS) Nr r r r(Tr rrrr)rrt_dense_valuesryrrs r6_to_batched_tensor_listz(RaggedTensorSpec._to_batched_tensor_list s ) @ AA%&   d// /#E$5$5#67##'#4#4"5Q 89 9d3 44  Q 5d6G6G5HU ,,-Q 0  $ < <!5 Nr8c|j tdtj|d|j|j |j }|jjt|trn|j|j|jEt|jdr/|jj|jj|S|j|j|S)Nr r)r0rrsr)rr*r$rurrrrrZr,rrrrzrr.)r3rr}s r6_from_compatible_tensor_listz-RaggedTensorSpec._from_compatible_tensor_list& s ) @ AA  ' 'Akk//,, (.F  {{$ FL )$++&  ,9I9I9D2F    & &t'<'<'B'B C M % Mr8c|j tdttj|gj |j |j|jdz|jSNr r)) rr*rrrrrrrr)r3 batch_sizes r6_batchzRaggedTensorSpec._batch9 sd ) @ AA   *.::4;;G T&&*D,B,B DDr8c|j tdt|jdd|j|j dz |j Sr)rr*rrrrrrs r6_unbatchzRaggedTensorSpec._unbatch@ sQ ) @ AA DKKOT[[$:K:Ka:O 22 44r8c|jSrrrs r6_to_legacy_output_typesz(RaggedTensorSpec._to_legacy_output_typesJ ;;r8c|jSrrrs r6_to_legacy_output_shapesz)RaggedTensorSpec._to_legacy_output_shapesM rr8c|Srrrs r6_to_legacy_output_classesz*RaggedTensorSpec._to_legacy_output_classesP s Kr8ct|tjs$t|jtj rH||j |jj|j|jjStj|j}|jjd}||j |jj|j|jj|S)N)r.r0rr)r.r0rrr)r,rrrzrtrur.r4r0rr[rtype_spec_from_valuerr)rJrrs r6rzRaggedTensorSpec.from_valueS s5-??@5$$j&7&78   ""'' ++11 33 #778I8IJ)224;;DA   ""'' ++11+ --r8)!rWrrr __slots__rr0r.rrrrr float32rr7rrrrrrrrrrrrr!r#rr __classcell__)rs@r6rr s"4)       &  ,  "  " " " H H^^ & $ 01d K>  , : 9 9.*&D4--r8c t|trR|rN|j|js3t d|j d|jj d|d|St|t jrftj|dg5tj|j||d}tj||jd cd d d Stj||||S#1swYy xYw) aMConverts value to a `RaggedTensor` or `Tensor`. * If `value` is a `RaggedTensor`, then return it as-is. * If `value` is a `RaggedTensorValue`, return a corresponding constant `RaggedTensor`. * Otherwise, use `convert_to_tensor` to convert `value` to a `Tensor`. Args: value: A `RaggedTensor`, a `RaggedTensorValue`, or an object whose type has a registered `Tensor` conversion function. dtype: Optional element type for the returned tensor. If missing the type is inferred from the type of `value`. preferred_dtype: Optional element type for the returned tensor, used when dtype is None. This argument has no effect if `value` is already a tensor, or when conversion is not possible. name: Optional name to use if a new `Tensor` is created. Returns: A `Tensor` or `RaggedTensor`. z"Tensor conversion requested dtype z for RaggedTensor with dtype rr(ConvertToTensorOrRaggedTensorrz)rr0rSrXFrTN)r,r$rr0r*rXrrr rUrrzrrr "convert_to_tensor_v2_with_dispatch)rr0preferred_dtyperXrzs r6"convert_to_tensor_or_ragged_tensorr-v s 0|$ U--ekk: ;EJJ<H227++2B2B1C2eWAO PP L%,>>? =r B@))!!$ k  0 0 u..1@ @@  ? ?5_4 @@s AC<<Dc6t|r|St|dS)aConverts value to supported RaggedTensor value. * If `value` is an object of supported value type, then return it as-is. * Otherwise convert it to Tensor or RaggedTensor. Args: value: An object of `Tensor`, `RaggedTensor` or registerred RaggedTensor value types, or an object whose type has a registered `Tensor` conversion function. Returns: An object of `Tensor`, `RaggedTensor` or registerred RaggedTensor value types r4r) _is_supported_ragged_values_typer-rs r6rere s&e, L -e( CCr8ct|}|j}|r'tj||j}|r'|Sr)rwrrr) componentsrs r6$_ragged_tensor_value_from_componentsr2 s>J* .. %  1 1%9I JE  ,r8cF|j|jfz}|tfSr)rrzr2)rr1s r6_ragged_tensor_session_fetchr4 s%##r~~&77* : ;;r8c|j|jfz}|j|jfz}t||Sr)rrzry)feed_keyfeed_valkey_componentsval_componentss r6_ragged_tensor_session_feedr: sA--1E1E0GG.--1E1E0GG. ^^ ,,r8c6|j|jfzSr)rrz)r6s r6+_ragged_tensor_session_feed_for_partial_runr< s  # #x';';&= ==r8cjeZdZdZej fdZedZedZ edZ dZ y)RaggedTensorTypezEncoding of a static type for a `RaggedTensor`. Use this type to express/declare that an output must have the type of `RaggedTensor`. cXtj|}||_||_||_y)a#Initializes a RaggedTensorType object. Args: dtype: data type of the `RaggedTensor`'s inner values. ragged_rank: ragged_rank of the declared `RaggedTensor`. row_splits_dtype: data type for the `RaggedTensor`'s row splits. One of: `tf.int32` or `tf.int64`. N)r rrrr)r3r0rrs r6r7zRaggedTensorType.__init__ s+'78DK#D-Dr8c|jSrrrs r6r zRaggedTensorType. s  r8c|jSrrrs r6r zRaggedTensorType. s d&7&7r8c|jSrrrs r6r zRaggedTensorType. s 4+A+Ar8cVd|jd|jd|jdS)NzRaggedTensorType(, r~)r0rrrs r6rzRaggedTensorType.__repr__ s&-1ZZ9I9I-1-B-B DDr8N) rWrrrr rr7rr0rrrrr8r6r>r> s@ ;A,, . + ,%78+ABDr8r>c |ddddf}|dddf}tjtj|dd|ddd}tj|tj |dddtj |dd|dddz}tj tjtj |dddtj|}dd|g}tj||gS)a/Checks that the given SparseTensor.indices tensor is ragged-right. Example: `indices = [[0, 0], [0, 1], [2, 0], [3, 1]]` is not ragged right because the entry `[3, 1]` skips a cell. Args: indices: The SparseTensor indices to check. Returns: A list of control dependency op tensors. Nrr)rrz SparseTensor is not right-raggedzSparseTensor.indices =) r reduce_any not_equalrwherer logical_andr rAssert)r\ index_prefix index_suffixindex_prefix_changedindex_oksparse_indices_are_ragged_rightr=s r6r^r^ s CRC,B, ",,ab)<+<=AG__HNN<+;Q?nn\!"%|CR'81'<=?( %-$8$8(..bq)91=>(#%%! )*BG '  $ $%Dg N OOr8RaggedTensorToSparsec|jdd}|jd}dgt|z}tj|}tj||}||gzS)z"Gradient for RaggedTensorToSparse.Nr)inputsrvrr.r) opunused_sparse_indices_gradsparse_values_gradunused_sparse_shape_gradop_inputs_nested_row_splitsop_inputs_flat_valuesnested_row_splits_gradientrflat_values_gradients r6!_ragged_tensor_to_sparse_gradientr[# sq !# #2))B-!%v,G(HH oo&;<"**+=+<> $';&< <t|tr.|j}|dz}|dk(r|S|dk(rt|tr.t|ts|jj r-|jj }|d|dgz||dzdz}n9t j|}t j|d|dg||dzdgd}t j||S|dkDr,|jt|j|dz |dz S|j}|j}t||D]}t|tr-t j|j|}|j}A||z dz}|jj r2|jj }dg||dz}t|d|} nht j|}t jdg||dgd}tj tj"|d||j$} t j||}|| z}ntj'||S)atMerges value[outer_axis...inner_axis] into a single dimension. See `RaggedTensor.merge_dims()` for more details. This helper differs from `RaggedTensor.merge_dims()` in that `value` may be a dense or ragged tensor. Args: value: A `RaggedTensor` or `Tensor` outer_axis: `int` inner_axis: `int` Returns: A flattened `RaggedTensor` or `Tensor`. rr)Nrr)r,r$r4r.rrrrrrrr[rr_prodrrE reduce_prodr0r\) rrr old_shaper/r new_splitsr shape_split flat_sizes r6rrH sn: L aJul; LLE!OJQ l aJul; E< ( {{##%++%%'iKZ(B4/)JNO2LLi//%(i"" [j !B4:>?)C D1Ni   UI ..!^   5<<aa@ BB ||**J + d*l+##J$9$9:Fj$$j%)k    * * ,$$,,. D9[\22 )Ak23 OOJ/ $$rdIkl,C%D1M MM  1[!9 :JB CC:a=335 6 7$ >> PsA=c|tjk7r"|tjk7rtd|dt |t j rR|jtjk7r3|jtjk7rtj||S|Stj|}|stjd|S|jDcgc]}|dn| }}tj||Scc}w)aTakes shape and coerces it to a shape as a tensor. If the object is already a tensor, simply passes it on (result is guaranteed to be int64 or int32, but not necessarily dtype). If not, creates a tensor of type dtype. Result is either a scalar equal to -1 if the shape is unknown_rank. Otherwise, it is a vector, where unknown dimensions are represented with a value of -1. In C++, see TensorShapeFromTensor for parsing shapes in kernels, and InferenceContext::MakeShapeFromShapeTensorTreatScalarAsUnknownShape, for use in the shape inference function. Args: shape: input to coerce from TensorShape, Tensor, None, List[Optional[Int]], Tuple[Optional[Int]]. dtype: tf.int64 or tf.int32 Returns: a scalar or vector tensor of dtype tf.int32 or tf.int64. z'Expected int64 or int32 for dtype: got r(rr)r rrr*r,rtrur0rrErrrrr)r.r0rLs r6rIrI s. fllu 4 >ugQG HHz(() {{fll"u{{fll'B ]]5% (( L    &%    % 00-2]]_ =!)B " =% =   e5 11 >s" DcFtj|jdj}|"t j ||j }|St|tr|j|j }|Stj||j d}|S)z> g  = >>C6)r8c"t|tSr)r,rrs r6r/r/+ s E8 99r8cvt|s.djdtD}td|d|dy)NrDc34K|]}|jywr)rW)r?rJs r6rAz:_assert_is_supported_ragged_values_type..1 sO#Osztype(values) must be one of: z, got r()r/joinrr-)rok_typess r6r+r+/ s? )% 0yyO1NOOH 3H:VE7!L MM 1r8ct|tjr?|jdk7rtj|dSt |j St|S)z)Separate Numpy array elements with comma.rrD) separator)r,rrr array2stringreprrstr)rLs r6rr5 sH2::vv{ __Q$ //!((*  q6Mr8) ragged_opsrr)frrkrmtypingrrtensorflow.core.protobufrtensorflow.pythonrtensorflow.python.clientrtensorflow.python.frameworkrrrr r r r rtr rrrrtensorflow.python.opsrrrrrrrtensorflow.python.ops.raggedrrr*tensorflow.python.ops.ragged.row_partitionrtensorflow.python.saved_modelrtensorflow.python.typesr core_typesr internal_typestensorflow.python.utilr! tensorflow.python.util.tf_exportr"tensorflow.tools.docsr#ror NativeObjectr$rrregisterBatchableTypeSpecrregister_codecBuiltInTypeSpecCodec TypeSpecProtoRAGGED_TENSOR_SPEC'register_type_spec_from_value_converterrrr-rer2r4r:r<)register_session_run_conversion_functionsr>r^RegisterGradientr[r_rarrCrrdrsrFrIrprWrurrr/r+rUnionRagged TensorLike RaggedOrDenserrr8r6rs; /!,8A3.+5<9431:+1+&5;*6<4C@6>+6.&<< >R"E$$R"ER"EjDK ,d 23T- !@!@T-4T-n &%%///*22EE2 11))+;+F+FH.27;,0)XD0< - >211.0K/1DDB&PR,-=.=&1 L #LL9@>F0 2$?.$2N ",!2!2L A /*d:N  l$7$I$II J  VZ%:%::; 4r8