AVhdZddlmZddlmZddlmZddlmZddlmZddlmZddlm Z dd lm Z dd l m Z dd l m Z dd l mZdd l mZddl mZddl mZddl mZddl mZddl mZddlmZddlmZddlmZdZdZ d'dZedgej< d(dZedgej<d)dZ edgej< d*dZ!edgej< d+d Z"ed!gej< d,d"Z#ed!gej< d-d#Z$d$Z%d%Z&d&Z'y).zOperations for embeddings.)compat)composite_tensor) constant_op)dtypes)indexed_slices)ops) sparse_tensor) tensor_shape) array_ops)array_ops_stack)clip_ops)data_flow_grad) data_flow_ops)math_ops)resource_variable_ops) sparse_ops) variables)core)dispatch) tf_exportc d}||S||\}}||\}}tj|||r|rtt||St j||S)aHelper function for _embedding_lookup_and_transform. This function optionally clips embeddings to an l2-norm of max_norm. Args: params: A `Tensor` of embeddings retrieved by `gather`. ids: The `ids` argument that was passed to `gather`. max_norm: If not `None`, each embedding is clipped if its l2-norm is larger than this value. Returns: A `Tensor` with the same type as `params`. ctj|jj}|r|dfSt j |dfS)a/Helper function to retrieve the rank of a tensor. Args: x: Something convertible to `Tensor`. Returns: Either a pair `(rank, True)` where `rank` is an integer or a pair `(rank, False)` where `rank` is an integer `Tensor`. In either case, `rank` is the rank of `x`. TF)rconvert_to_tensor get_shapendimsr rank)xrs S/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/ops/embedding_ops.py_rankz_clip.._rank7sD   # - - / 5 5D 4Z ^^A  %%)axes)r clip_by_normlistranger)paramsidsmax_normrids_rank ids_static params_rank params_statics r_clipr,(sv&" Ms(J$V}+}    2<Dx- . 99+6  99r ctjrLt|dr@tjtjj |j Stj|S)Nhandle)rinside_functionhasattr colocate_withget_default_graphcapturer.)params r_colocate_withr5SsSwuh7   S224<| j:|j@}ng}t-|D]} t5j8|| jd }||jC|@t|| 5|jCt#jD|| d dddt+jFt+jHtKjL||j@}||z}||z}t+jN||d zz||z |z} t#jP| |k||d zz||z |z} ntd|dt+jH| t0j2} tSjT| | |}tSjT| | |}g}t-|D]{} || }t jVd5t|| 5t#j$|| |}|r|t!|||}dddddd|jC}tSjX|||}|C|d jd d}|d dD]$} |j[| jd d}&n|jd d}|j]r|}nO|5t|d 5t#jD|d }dddd d}nt#jD|d d}t#j(|t#j^t#jD||gd }|ja|jjc||s t!|||}|cdddS#1swYxYw#1swYxxYw#1swYxYw#1swYxYw#1swYxYw#1swYyxYw)aHelper function for embedding_lookup and _compute_sampled_logits. This function is a generalization of embedding_lookup that optionally applies a caller-specified transformation to each embedding. This is done through the `transform_fn` argument. If provided, the function is applied to each partitioned tensor of retrieved embeddings, colocated with the embeddings. This function will be called with a single `Tensor` argument of the same type as the `params` tensor and should return a `Tensor`. The shape of the argument will be the same as `params` except for the size of the first dimension. The first dimension of the result's shape must be the same size as the argument's. Args: params: See embedding_lookup. ids: See embedding_lookup. partition_strategy: See embedding_lookup. name: See embedding_lookup. max_norm: See embedding_lookup. transform_fn: An optional function to apply to each retrieved embedding. If max_norm is provided, transform_fn is applied to the norm-limited embeddings. Returns: See embedding_lookup for details. Raises: ValueError: If `params` is empty. Nzparams must be specifiedz9Length of params is currently 0. Need at least one param.embedding_lookupc3PK|]}t|tj yw)N) isinstancerBaseResourceVariable).0ps r z2_embedding_lookup_and_transform..s&  1+@@As$&r%namer&r)out_typemoddivz!Unrecognized partition strategy: z&.Must be one of either `mod` or `div`.)2 ValueErrorr9r#tuplerPartitionedVariabler name_scopelenanyr%convert_n_to_tensor_or_indexed_slicesrrrr5r,r gatheridentityreshaperr$sizerint32r Dimensiondimension_valuevaluerconstantdtypeappendshape reduce_sumcastr stackmaximumwhererdynamic_partition device_v2parallel_dynamic_stitch merge_withis_fully_definedconcat set_shape concatenate)r%r&partition_strategyr?r' transform_fnnpresultflat_idsoriginal_indices p_assignmentsnew_ids dim_0_sizer< num_total_ids dim_0_sizes param_p_dimids_per_partitionextras gather_idspindicespartitioned_resultpidsretelement_shape_selement_shape_d params_shapes r_embedding_lookup_and_transformr{]s B ^ / 00u &v 0 11 556 &\F FD !XF ~~d.#?@4 VB  CC x!f  % 0C Qw  (=(=(B &) $(   VAY$ 7hH '& (   '%@@2""3-h! ..FLL 9 u $ 2 b.  &"++  ( ()<)<)>q)A BD q" FA  ..**6!9+>+>+@+CDF F* F   %..z/?/?P-+9 Ba&66vay7J7J7LQ7OPK&  -!&),B""9??6!9#=a#@ABB  B#--mmO11+>OQ-)R/# ((6G!6K)L*2V*;):*;< //-&"8"*.?!.C"D#+f#48I"IK/0B/CD4 45 5 mmM6<<@m 227M2Nj001A1>DhRy *!!} ]]4  CfQi( C%%fQi6F$E&$$ABf  C C !!&) *  1 1 &T 3c   )--/3 JA+66q{{}QR7HI/ J--/!"-  ) ) +)  F1I & 4"3, 4&qr*#//#.qr2    y!5 GK Mc  mmCMMO//@A Ch' A@@((TBBB C C C C6 4 4_@@sA>X01W03X0FX0+W=D.X06X/X 4X 1`, each element `id` of `ids` is partitioned between the elements of `params` according to the `partition_strategy`. In all strategies, if the id space does not evenly divide the number of partitions, each of the first `(max_id + 1) % len(params)` partitions will be assigned one more id. If `partition_strategy` is `"mod"`, we assign each id to partition `p = id % len(params)`. For instance, 13 ids are split across 5 partitions as: `[[0, 5, 10], [1, 6, 11], [2, 7, 12], [3, 8], [4, 9]]` If `partition_strategy` is `"div"`, we assign ids to partitions in a contiguous manner. In this case, 13 ids are split across 5 partitions as: `[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 10], [11, 12]]` If the input ids are ragged tensors, partition variables are not supported and the partition strategy and the max_norm are ignored. The results of the lookup are concatenated into a dense tensor. The returned tensor has shape `shape(ids) + shape(params)[1:]`. Args: params: A single tensor representing the complete embedding tensor, or a list of P tensors all of same shape except for the first dimension, representing sharded embedding tensors. Alternatively, a `PartitionedVariable`, created by partitioning along dimension 0. Each element must be appropriately sized for the given `partition_strategy`. ids: A `Tensor` or a 'RaggedTensor' with type `int32` or `int64` containing the ids to be looked up in `params`. Caution: Out-of-bounds indices will result in undefined behavior, which will differ between devices and backends. partition_strategy: A string specifying the partitioning strategy, relevant if `len(params) > 1`. Currently `"div"` and `"mod"` are supported. Default is `"mod"`. name: A name for the operation (optional). validate_indices: DEPRECATED. If this operation is assigned to CPU, values in `indices` are always validated to be within range. If assigned to GPU, out-of-bound indices result in safe but unspecified behavior, which may include raising an error. max_norm: If not `None`, each embedding is clipped if its l2-norm is larger than this value. Returns: A `Tensor` or a 'RaggedTensor', depending on the input, with the same type as the tensors in `params`. Raises: ValueError: If `params` is empty. N)r%r&rer?r'rf)r{)r%r&rer?validate_indicesr's rr7r7 s*DB )  +   r c"t||d||S)axLooks up embeddings for the given `ids` from a list of tensors. This function is used to perform parallel lookups on the list of tensors in `params`. It is a generalization of `tf.gather`, where `params` is interpreted as a partitioning of a large embedding tensor. If `len(params) > 1`, each element `id` of `ids` is partitioned between the elements of `params` according to the "div" partition strategy, which means we assign ids to partitions in a contiguous manner. For instance, 13 ids are split across 5 partitions as: `[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 10], [11, 12]]`. If the id space does not evenly divide the number of partitions, each of the first `(max_id + 1) % len(params)` partitions will be assigned one more id. The results of the lookup are concatenated into a dense tensor. The returned tensor has shape `shape(ids) + shape(params)[1:]`. Args: params: A single tensor representing the complete embedding tensor, or a list of tensors all of same shape except for the first dimension, representing sharded embedding tensors following "div" partition strategy. ids: A `Tensor` with type `int32` or `int64` containing the ids to be looked up in `params`. max_norm: If not `None`, each embedding is clipped if its l2-norm is larger than this value. name: A name for the operation (optional). Returns: A `Tensor` with the same type as the tensors in `params`. For instance, if `params` is a 5x2 matrix: ```python [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10]] ``` or a list of matrices: ```python params[0]: [[1, 2], [3, 4]] params[1]: [[5, 6], [7, 8]] params[2]: [[9, 10]] ``` and `ids` is: ```python [0, 3, 4] ``` The output will be a 3x2 matrix: ```python [[1, 2], [7, 8], [9, 10]] ``` Raises: ValueError: If `params` is empty. rD)r')r7)r%r&r'r?s rembedding_lookup_v2rws~ &#udX FFr znn.embedding_lookup_sparsec |d}|dvrtd|t|tjr t |}t|ts|g}t|t j stdt||du}|st|t j stdt||jjj|jj|jjj|jj|jjj|jjtj|d||gz5}|jdddf} |j} t!|| || |||||| cdddS#1swYyxYw) aZLooks up embeddings for the given ids and weights from a list of tensors. This op assumes that there is at least one id for each row in the dense tensor represented by sp_ids (i.e. there are no rows with empty features), and that all the indices of sp_ids are in canonical row-major order. `sp_ids` and `sp_weights` (if not None) are `SparseTensor`s or `RaggedTensor`s with rank of 2. For `SpareTensor`s with left-aligned non-zero entries which can be described as `RaggedTensor`s, use of `RaggedTensor`s can yield higher performance. It also assumes that all id values lie in the range [0, p0), where p0 is the sum of the size of params along dimension 0. Args: params: A single tensor representing the complete embedding tensor, or a list tensors all of same shape except for the first dimension, representing sharded embedding tensors. Alternatively, a `PartitionedVariable`, created by partitioning along dimension 0. Each element must be appropriately sized for the given `partition_strategy`. sp_ids: N x M `SparseTensor` of int64 ids where N is typically batch size and M is arbitrary or a `RaggedTensor` with rank 2. sparse_weights: `SparseTensor` or `RaggedTensor` of same type and shape as `sparse_ids`, containing float / double weights corresponding to `sparse_ids`, or `None` if all weights are assumed to be 1.0. partition_strategy: A string specifying the partitioning strategy, relevant if `len(params) > 1`. Currently `"div"` and `"mod"` are supported. Default is `"mod"`. See `tf.nn.embedding_lookup` for more details. name: Optional name for the op. combiner: A string specifying the reduction op. Currently "mean", "sqrtn" and "sum" are supported. "sum" computes the weighted sum of the embedding results for each row. "mean" is the weighted sum divided by the total weight. "sqrtn" is the weighted sum divided by the square root of the sum of the squares of the weights. Defaults to `mean`. max_norm: If not `None`, each embedding is clipped if its l2-norm is larger than this value, before combining. allow_fast_lookup: An optional boolean specifying whether to allow simplified embedding lookups when `params` is a single tensor and `max_norm` is `None`. Setting this flag to `True` during training can cause the use of dense gradients with increased memory footprint. Returns: A dense tensor representing the combined embeddings for the sparse ids. For each row in the dense tensor represented by `sp_ids`, the op looks up the embeddings for all ids in that row, multiplies them by the corresponding weight, and combines these embeddings as specified. In other words, if `shape(combined params) = [p0, p1, ..., pm]` and `shape(sp_ids) = shape(sp_weights) = [d0, d1]` then `shape(output) = [d0, p1, ..., pm]`. For instance, if params is a 10x20 matrix, and sp_ids / sp_weights are ```python [0, 0]: id 1, weight 2.0 [0, 1]: id 3, weight 0.5 [1, 0]: id 0, weight 1.0 [2, 3]: id 1, weight 3.0 ``` with `combiner`="mean", then the output will be a 3x20 matrix where ```python output[0, :] = (params[1, :] * 2.0 + params[3, :] * 0.5) / (2.0 + 0.5) output[1, :] = (params[0, :] * 1.0) / 1.0 output[2, :] = (params[1, :] * 3.0) / 3.0 ``` Raises: TypeError: If `sp_ids` is not a `SparseTensor` or `RaggedTensor`, or if `sp_weights` is neither `None` nor of the same type as `sp_ids`. ValueError: If `combiner` is not one of {"mean", "sqrtn", "sum"}. Nmean)rsqrtnsumz6combiner must be one of 'mean', 'sqrtn' or 'sum', got z!sp_ids must be SparseTensor, got z3sp_weights must be either None or SparseTensor,got embedding_lookup_sparser)rEr9rrGr#r SparseTensor TypeErrortypevaluesrassert_is_compatible_withindices dense_shaperrHembedding_lookup_sparse_impl) r%sp_ids sp_weightsrer?combinerr'allow_fast_lookupignore_weights segment_idsr&s rrrszH --  @ K MM 556 &\F FD !XF FM66 7 7V ~F GG%.  j-"<"< = !*-.0 11 MM77##%' NN88$$&(   "<<((*,  ~~d5x'),0..A&K --C '    s 3GGc &t|||d||||S)a&Looks up embeddings for the given ids and weights from a list of tensors. `params` is a dense tensor or a list of dense tensors, and `sp_ids` is a 2D `tf.SparseTensor` or `tf.RaggedTensor` indicating the indices of `params` to gather. This op is best described with an example. Suppose `params` is an embedding table of size `(4, 2)` and `sp_ids` has 3 rows. Since `sp_ids` is sparse or ragged, not every row has the same number of elements. The output has shape (3, 2). Each row of `sp_ids` is a list of indices, where each index selects a row of `params`. For a given row of `sp_ids`, the rows of `params` are gathered based on the indices in `sp_ids`, then combined by taking their sum or mean. >>> params = tf.constant([[1, 2], [3, 4], [5, 6], [7, 8]], dtype=tf.float32) >>> sp_ids = tf.SparseTensor(indices=[[0, 0], [0, 1], [1, 0], [2, 0]], ... values=[0, 1, 3, 2], dense_shape=(3, 2)) >>> tf.nn.embedding_lookup_sparse(params, sp_ids, sp_weights=None, ... combiner='sum').numpy() array([[4., 6.], [7., 8.], [5., 6.]], dtype=float32) In this example, `sp_ids` has 3 rows, so the output has 3 rows. Row 0 of `sp_ids` has values 0 and 1, so it selects rows 0 and 1 from `params`, which are `[1, 2]` and `[3, 4]`. The rows are summed since `combiner='sum'`, resulting in the output row of `[4, 6]`. Since row 1 and 2 of `sp_ids` only have one value each, they simply select the corresponding row from `params` as the output row. Row 1 has value `3` so it selects the `params` elements `[7, 8]` and row 2 has the value 2 so it selects the `params` elements `[5, 6]`. If `sparse_weights` is specified, it must have the same shape as `sp_ids`. `sparse_weights` is used to assign a weight to each slice of `params`. For example: >>> params = tf.constant([[1, 2], [3, 4], [5, 6], [7, 8]], dtype=tf.float32) >>> sp_ids = tf.SparseTensor(indices=[[0, 0], [0, 1], [1, 0], [2, 0]], ... values=[0, 1, 3, 2], dense_shape=(3, 2)) >>> sparse_weights = tf.SparseTensor(indices=[[0, 0], [0, 1], [1, 0], [2, 0]], ... values=[0.1, 1.0, 0.5, 2.0], ... dense_shape=(3, 2)) >>> tf.nn.embedding_lookup_sparse(params, sp_ids, sp_weights=sparse_weights, ... combiner='sum').numpy() array([[3.1, 4.2], [3.5, 4.], [10., 12.]], dtype=float32) In general, `params` can have shape `(p0, ..., pn)` and `sp_ids` can have `M` rows, where each row can have any number of elements. The output has shape `(M, p1, ..., pn)`. Each slice of the output `output[i, ...]` is obtained as follows: The `combiner` argument is used to combine the values `params[sp_ids[i, j], ...] * sparse_weights[i, j]` for each `j` in `range(0, len(sp_ids[i]))`, e.g. by taking the sum or mean of the values. This op assumes that there is at least one id for each row in the dense tensor represented by sp_ids (i.e. there are no rows with empty features), and that all the indices of sp_ids are in canonical row-major order. `sp_ids` and `sp_weights` (if not None) are `SparseTensor`s or `RaggedTensor`s with rank of 2. For `SpareTensor`s with left-aligned non-zero entries which can be described as `RaggedTensor`s, use of `RaggedTensor`s can yield higher performance. This op assumes that all id values lie in the range [0, p0), where p0 is `params.shape[0]`. If you want a version of this op that prunes id values less than 0, see `tf.nn.safe_embedding_lookup_sparse` If `len(params) > 1`, each element of `sp_ids` is partitioned between the elements of `params` according to the "div" partition strategy, which means we assign ids to partitions in a contiguous manner. For instance, 13 ids are split across 5 partitions as: `[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 10], [11, 12]]`. If the id space does not evenly divide the number of partitions, each of the first `(max_id + 1) % len(params)` partitions will be assigned one more id. Args: params: A single tensor representing the complete embedding tensor, or a list of tensors all of same shape except for the first dimension, representing sharded embedding tensors following "div" partition strategy. sp_ids: N x M `SparseTensor` of int64 ids where N is typically batch size and M is arbitrary or a `RaggedTensor` with rank 2. sparse_weights: `SparseTensor` or `RaggedTensor` of same type and shape as `sparse_ids`, containing float / double weights corresponding to `sparse_ids`, or `None` if all weights are assumed to be 1.0. combiner: A string specifying the reduction op. Currently "mean", "sqrtn" and "sum" are supported. "sum" computes the weighted sum of the embedding results for each row. "mean" is the weighted sum divided by the total weight. "sqrtn" is the weighted sum divided by the square root of the sum of the squares of the weights. Defaults to `mean`. max_norm: If not `None`, each embedding is clipped if its l2-norm is larger than this value, before combining. name: Optional name for the op. allow_fast_lookup: An optional boolean specifying whether to allow simplified embedding lookups when `params` is a single tensor and `max_norm` is `None`. Setting this flag to `True` during training can cause the use of dense gradients with increased memory footprint. Returns: A dense tensor representing the combined embeddings for the sparse ids. For each row in the dense tensor represented by `sp_ids`, the op looks up the embeddings for all ids in that row, multiplies them by the corresponding weight, and combines these embeddings as specified. In other words, if `shape(combined params) = [p0, p1, ..., pm]` and `shape(sp_ids) = shape(sp_weights) = [d0, d1]` then `shape(output) = [d0, p1, ..., pm]`. For instance, if params is a 10x20 matrix, and sp_ids / sp_weights are ```python [0, 0]: id 1, weight 2.0 [0, 1]: id 3, weight 0.5 [1, 0]: id 0, weight 1.0 [2, 3]: id 1, weight 3.0 ``` with `combiner`="mean", then the output will be a 3x20 matrix where ```python output[0, :] = (params[1, :] * 2.0 + params[3, :] * 0.5) / (2.0 + 0.5) output[1, :] = (params[0, :] * 1.0) / 1.0 output[2, :] = (params[1, :] * 3.0) / 3.0 ``` Raises: TypeError: If `sp_ids` is not a `SparseTensor`, or if `sp_weights` is neither `None` nor `SparseTensor`. ValueError: If `combiner` is not one of {"mean", "sqrtn", "sum"}. rD)r)r%rrrr'r?rs rembedding_lookup_sparse_v2rCs*f !      r znn.safe_embedding_lookup_sparsec *t||||||d|| S)a]Lookup embedding results, accounting for invalid IDs and empty features. The partitioned embedding in `embedding_weights` must all be the same shape except for the first dimension. The first dimension is allowed to vary as the vocabulary size is not necessarily a multiple of num of shards. This is similar to `tf.nn.embedding_lookup_sparse`, except invalid IDs (< 0) are pruned from input IDs and weights, as well as any IDs with non-positive weight. For an entry with no features, the embedding vector for `default_id` is returned, or the 0-vector if `default_id` is not supplied. See `tf.nn.embedding_lookup_sparse` for more information on how sparse embedding lookups work in general. The ids and weights may be multi-dimensional `SparseTensor`s or `RaggedTensor`s with rank of 2. For `SpareTensor`s with left-aligned non-zero entries which can be described as `RaggedTensor`s, use of `RaggedTensor`s can yield higher performance. If `len(embedding_weights) > 1`, each element `id` of `ids` is partitioned between the elements of `embedding_weights` according to the "div" partition strategy, which means we assign ids to partitions in a contiguous manner. For instance, 13 ids are split across 5 partitions as: `[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 10], [11, 12]]`. If the id space does not evenly divide the number of partitions, each of the first `(max_id + 1) % len(embedding_weights)` partitions will be assigned one more id. Args: embedding_weights: A single tensor representing the complete embedding tensor, or a list of tensors all of same shape except for the first dimension, representing sharded embedding tensors following "div" partition strategy. sparse_ids: `SparseTensor` of shape `[d_0, d_1, ..., d_n]` containing the ids, where `d_0` is typically batch size, or a `RaggedTensor` with rank 2. sparse_weights: `SparseTensor` or `RaggedTensor` of same type and shape as `sparse_ids`, containing float weights corresponding to `sparse_ids`, or `None` if all weights are assumed to be 1.0. combiner: A string specifying how to combine embedding results for each entry. Currently "mean", "sqrtn" and "sum" are supported, with "mean" the default. default_id: The id to use for an entry with no features. Defaults to 0-vector. max_norm: If not `None`, all embeddings are l2-normalized to max_norm before combining. name: A name for this operation (optional). allow_fast_lookup: An optional boolean specifying whether to allow simplified embedding lookups when `params` is a single tensor and `max_norm` is `None`. Setting this flag to `True` during training can cause the use of dense gradients with increased memory footprint. Returns: A dense tensor representing the combined embeddings for the sparse ids. For each row in the dense tensor represented by `sparse_ids`, the op looks up the embeddings for all ids in that row, multiplies them by the corresponding weight, and combines these embeddings as specified. In other words, if `shape(combined embedding_weights) = [p0, p1, ..., pm]` and `shape(sparse_ids) = shape(sparse_weights) = [d0, d1, ..., dn]` then `shape(output) = [d0, d1, ... dn-1, p1, ..., pm]`. For instance, if params is a 10x20 matrix, and sp_ids / sp_weights are ```python [0, 0]: id 1, weight 2.0 [0, 1]: id 3, weight 0.5 [1, 0]: id -1, weight 1.0 [2, 3]: id 1, weight 3.0 ``` `default_id` is 0. with `combiner`="mean", then the output will be a 3x20 matrix where ```python output[0, :] = (params[1, :] * 2.0 + params[3, :] * 0.5) / (2.0 + 0.5) output[1, :] = (params[0, :] * 1.0) / 1.0 output[2, :] = (params[1, :] * 3.0) / 3.0 ``` Raises: ValueError: if `embedding_weights` is empty. rD)sparse_weightsr default_idr?rer'r)safe_embedding_lookup_sparse)embedding_weights sparse_idsrrrr'r?rs rsafe_embedding_lookup_sparse_v2rs-N &# )  r c |td|dt|tjr t |}t|ts|g}t |dkrtd|d| |j nd} |D cgc]@} tj| r| d| j fvr| ntj| | B}} tj|d|||gz5} |j} tj|jjd} | t!j"| n| }t%j&|t)j*t!j,| dg|dz gt!j.| |dz g}|5t1j2|j4|j6|j}t9||\}}|dk7rt;||\}}t%j<||xsd\}}|t%j<|d \}}t?||||||dn| || }|t!j@t!jB|d dgtEjFdt!jH|dg}t!jJ|t!jL||| }t!jB|t!jNt!j,t)jP| tRjTdg|dz gt!j,t!jH|dgd ggd}|jWtjXtjZ| dz j\j_|jdd|cdddScc} w#1swYyxYw) aMLookup embedding results, accounting for invalid IDs and empty features. The partitioned embedding in `embedding_weights` must all be the same shape except for the first dimension. The first dimension is allowed to vary as the vocabulary size is not necessarily a multiple of `P`. `embedding_weights` may be a `PartitionedVariable` as returned by using `tf.compat.v1.get_variable()` with a partitioner. Invalid IDs (< 0) are pruned from input IDs and weights, as well as any IDs with non-positive weight. For an entry with no features, the embedding vector for `default_id` is returned, or the 0-vector if `default_id` is not supplied. The ids and weights may be multi-dimensional `SparseTensor`s or `RaggedTensor`s with rank of 2. For `SpareTensor`s with left-aligned non-zero entries which can be described as `RaggedTensor`s, use of `RaggedTensor`s can yield higher performance. Embeddings are always aggregated along the last dimension. Args: embedding_weights: A single tensor representing the complete embedding tensor, or a list tensors all of same shape except for the first dimension, representing sharded embedding tensors. Alternatively, a `PartitionedVariable`, created by partitioning along dimension 0. Each element must be appropriately sized for the given `partition_strategy`. sparse_ids: `SparseTensor` of shape `[d_0, d_1, ..., d_n]` containing the ids, where `d_0` is typically batch size, or a `RaggedTensor` with rank 2. sparse_weights: `SparseTensor` or `RaggedTensor` of same type and shape as `sparse_ids`, containing float weights corresponding to `sparse_ids`, or `None` if all weights are assumed to be 1.0. combiner: A string specifying how to combine embedding results for each entry. Currently "mean", "sqrtn" and "sum" are supported, with "mean" the default. default_id: The id to use for an entry with no features. name: A name for this operation (optional). partition_strategy: A string specifying the partitioning strategy. Currently `"div"` and `"mod"` are supported. Default is `"div"`. max_norm: If not `None`, all embeddings are l2-normalized to max_norm before combining. allow_fast_lookup: An optional boolean specifying whether to allow simplified embedding lookups when `params` is a single tensor and `max_norm` is `None`. Setting this flag to `True` during training can cause the use of dense gradients with increased memory footprint. Returns: A dense tensor representing the combined embeddings for the sparse ids. For each row in the dense tensor represented by `sp_ids`, the op looks up the embeddings for all ids in that row, multiplies them by the corresponding weight, and combines these embeddings as specified. In other words, if `shape(combined embedding_weights) = [p0, p1, ..., pm]` and `shape(sparse_ids) = shape(sparse_weights) = [d0, d1, ..., dn]` then `shape(output) = [d0, d1, ... dn-1, p1, ..., pm]`. For instance, if params is a 10x20 matrix, and sp_ids / sp_weights are ```python [0, 0]: id 1, weight 2.0 [0, 1]: id 3, weight 0.5 [1, 0]: id -1, weight 1.0 [2, 3]: id 1, weight 3.0 ``` `default_id` is 0. with `combiner`="mean", then the output will be a 3x20 matrix where ```python output[0, :] = (params[1, :] * 2.0 + params[3, :] * 0.5) / (2.0 + 0.5) output[1, :] = (params[0, :] * 1.0) / 1.0 output[2, :] = (params[1, :] * 3.0) / 3.0 ``` Raises: ValueError: if `embedding_weights` is empty. NzMissing embedding_weights .r@rUr7rrg?)rrer?r'rrAr>)0rEr9rrGr#rIrUris_resource_variablerrrHrr rRrr rOrsparse_reshaper reduce_prodslicerLr rrr_prune_invalid_ids_prune_invalid_weightssparse_fill_empty_rowsrtilerNr rZrWr\ zeros_likerbrYrrPrc unknown_shaperQrSrd)rrrrrr?rer'rrUwscopeoriginal_shapeoriginal_rank_dim original_rank is_row_empty_rh final_results rrrVsB 12C1DAF GG!9#@#@A./ %t ,*+ a 12C1DAF GG"0"<.  $% !   " 6 6q 9$(  % 01 ~~d.0A!>2134B7<++N$44((*1-/  $ ~&*;**: OONQC-!2C1D E G):;8J !$11*2D2D2@2G2G2<2H2HJn "4J!OJ5#9 n$& j. *@@JO! %J !$;;NCPna $-'TU+ F^^   L2q' 2   IOOF$;A$> ? @Bl  ,,V4f5Jf$$ OO nfll;aS"# % OOIOOF3aS2$ ?   L""  # #$5 6 : A A +f&&(, - EBBBBs;AN?KOO c  d} tj|d}t|dk(rHt|dtj t jfstj|ddg}t|dk(rt|dt jso|dj|jk(rS|Q|s|rMtjddd r6|} |d} t| tjr| j} |s*d } n'tj|\}} t!|||| } |s|j"t$j&k7r$t)j*|t$j&}|j,} tj.| | } | j"}| j"t$j0t$j2fvr$t)j*| t$j4} | j"| j"k7r t)j*| | j"} tj6tj8| dz d}tj:| }tj<||j" }tj>||gd}| jA}tjB| |} | jAjDR| jG|jItK| jAjDdz Dcgc]}dc}| | z} |d k(rt)jL| || } n|dk(rEt)jL| |} t)jL| |}t)jN| || } n||dk(rpt)jL| |} t)jP| d}t)jL||}t)jR|}t)jN| || } nJd| j"|k7rt)j*| |} | S|j"t$j&t$jTfvr$t)j*|t$j&}| J|d k(rt)jV| | || | } | S|dk(rt)jX| | || | } | S|dk(rt)jZ| | || | } | SJdcc}w)z/Implementation of sparse embedding aggregation.Frr>r@rr%i T)rer'rrrrzUnrecognized combiner)r?sparse_gradient).rrrIr9rTensorrCompositeTensordevicerforward_compatiblerr:read_value_no_copyr uniquer7rUrrPrrYrrLfloat16bfloat16float32 expand_dimsrrWonesrbrrNrrcrdr$ segment_sum div_no_nanpowsqrtint64sparse_segment_sumsparse_segment_meansparse_segment_sqrt_n)r%rrr&rrr'rrer?need_sparse_segment_gradientidx embeddingsweightsoriginal_dtype ones_shape weights_shaperbcast_weights_shapeorig_weights_shaper weight_sumweights_squaredweight_sum_sqrts rrr s{"'%%k F+[Aj Qi$++/??@##F1IH= >F &kQ$4$D$DE )  k00 0   !:!:4B!G CJ*3HHI002j %)"$HC!(:XJ FLL(MM+v||>*M,?,? @D#**M4+@!D **,)<=G##/   ( ( 4 4 6 < )==^NON ^ ##r )rCNNN)rCNTN)NN)rCNNNF)NNNF)NrNNNF)NrNNrDNF)(__doc__tensorflow.python.compatrtensorflow.python.frameworkrrrrrr r tensorflow.python.opsr r r rrrrrrtensorflow.python.typesrtensorflow.python.utilr tensorflow.python.util.tf_exportrr,r5r{add_dispatch_supportr7rrrrrrrrr rrs!+83.6+54+1*0/*7,+(+6(9V$8=)--115 k\ $%&    g'gT R( =G)=G@ +,-    E.EP 'B/    Z0Zz ,4    o5od 012    r3rj@F $$r