AVh` Z d Z ddlZddlmZ ddlmZ ddlmZ ddlmZ ddlmZ ddl m
Z
dd l m Z dd
l m Z dd l m
Z
dd l dd
lmZ ddlmZ ddlmZ ddlmZ ed ej, ej. dBd Ze j0 j e_ edddg ej, ej. ej2 d dCd Z ed ej. dDd ZdEdZ dFdZ d Z ed d g ej. ej> dd d ej2 d dGd Z ed g ej. dHd! Z!e!j e _ ed"g ej, ej. dId# Z" ed"g ej, ej. dJd$ Z#e jD j e#_ ed%g ej, ej. ejH dd& dId' Z%e jL j e%_ ed(g ej, ej. dId) Z&e jL j e&_ ed(g ej, ej. dJd* Z'e jL j e'_ ejP d+ ejP d, ejP d- ejP d. ejP d/ ejP d0 ejP d1 ejP d2 ejP d3 ejP d4 ed5g ej, ej. ejR dfd6 Z* ed5d7g ej. dejR ddfd8 Z+e jT j e+_ ed9g ej, ej. dBd: Z, ed9d;g ej, ej. dKd< Z-e jX j e-_ ed=d=d>g ej. ej2 d> dLd? Z. ed@ ej. dLdA Z/ ej, e j` ej, e jb ej, e jd ej, e jf ej, e jh ej, e jj ej, e jl ej, e jn ej, e jp ej, e jr y)MzOOperations for working with string Tensors.
API docstring: tensorflow.strings
N) constant_op)dtypes)ops)
sparse_tensor) tensor_util) array_ops)gen_parsing_ops)gen_string_ops)math_ops)*)compat) deprecation)dispatch) tf_exportzstrings.regex_full_matchc t |t j rt j | || S t j
| || S )am Match elements of `input` with regex `pattern`.
Args:
input: string `Tensor`, the source strings to process.
pattern: string or scalar string `Tensor`, regular expression to use,
see more details at https://github.com/google/re2/wiki/Syntax
name: Name of the op.
Returns:
bool `Tensor` of the same shape as `input` with match results.
inputpatternname)
isinstance util_compatbytes_or_text_typesr
static_regex_full_matchregex_full_matchr s P/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/ops/string_ops.pyr r / sJ 889 1 1W4 1 1 ( ( 7
/ / zstrings.regex_replace
regex_replace)v1c t |t j r4t |t j rt j | |||| S t j
| |||| S )a Replace elements of `input` matching regex `pattern` with `rewrite`.
>>> tf.strings.regex_replace("Text with tags.
contains html",
... "<[^>]+>", " ")
Args:
input: string `Tensor`, the source strings to process.
pattern: string or scalar string `Tensor`, regular expression to use,
see more details at https://github.com/google/re2/wiki/Syntax
rewrite: string or scalar string `Tensor`, value to use in match
replacement, supports backslash-escaped digits (\1 to \9) can be to insert
text matching corresponding parenthesized group.
replace_global: `bool`, if `True` replace all non-overlapping matches,
else replace only the first match.
name: A name for the operation (optional).
Returns:
string `Tensor` of the same shape as `input` with specified replacements.
r r rewritereplace_globalr )r r r r
static_regex_replacer r s r r r J sh 4 +99:+99: . .W
% % 7n
r zstrings.formatc t j | r|g}| j | t | k7 r*t d| j | dt | d t j || ||| S )a Formats a string template using a list of tensors.
Formats a string template using a list of tensors, abbreviating tensors by
only printing the first and last `summarize` elements of each dimension
(recursively). If formatting only one tensor into a template, the tensor does
not have to be wrapped in a list.
Example:
Formatting a single-tensor template:
>>> tensor = tf.range(5)
>>> tf.strings.format("tensor: {}, suffix", tensor)
Formatting a multi-tensor template:
>>> tensor_a = tf.range(2)
>>> tensor_b = tf.range(1, 4, 2)
>>> tf.strings.format("a: {}, b: {}, suffix", (tensor_a, tensor_b))
Args:
template: A string template to format tensor values into.
inputs: A list of `Tensor` objects, or a single Tensor.
The list of tensors to format into the template string. If a solitary
tensor is passed in, the input tensor will automatically be wrapped as a
list.
placeholder: An optional `string`. Defaults to `{}`.
At each placeholder occurring in the template, a subsequent tensor
will be inserted.
summarize: An optional `int`. Defaults to `3`.
When formatting the tensors, show the first and last `summarize`
entries of each tensor dimension (recursively). If set to -1, all
elements of the tensor will be shown.
name: A name for the operation (optional).
Returns:
A scalar `Tensor` of type `string`.
Raises:
ValueError: if the number of placeholders does not match the number of
inputs.
zThe template expects z" tensors, but the inputs only has zM. Please ensure the number of placeholders in template matches inputs length.)template placeholder summarizer )r
is_tf_typecountlen
ValueErrorr
string_format)r% inputsr& r' r s r r, r, s s b F#XF
^^K CK/
,X^^K-H,I J992<><>3"` and
sep of `"<>"` returns `["1", "2", "", "3"]`. If `sep` is None or an empty
string, consecutive whitespace are regarded as a single separator, and the
result will contain no empty strings at the start or end if the string has
leading or trailing whitespace.
Note that the above mentioned behavior matches python's str.split.
Args:
source: `1-D` string `Tensor`, the strings to split.
sep: `0-D` string `Tensor`, the delimiter character.
maxsplit: An `int`. If `maxsplit > 0`, limit of the split of the result.
Raises:
ValueError: If sep is not a string.
Returns:
A `SparseTensor` of rank `2`, the strings split according to the delimiter.
The first column of the indices corresponds to the row in `source` and the
second column corresponds to the index of the split component in this row.
N r2 )r/ maxsplitr5 ) r r7 r r8 r
string_split_v2r: r r; )r< r/ rB r= r> r? s r rC rC s P [
C c 7# v}} =&)99 # *'65 T1I D6//1# # #GVU ;;r c B ||S | j j Ut j t j
| j j dz
dd t j S t j t j | dz
dd S )zEReturns range(rank(x) - 1, 0, -1) if axis is None; or axis otherwise. r2 ) get_shapendimsr constantnparanger int32r ranger rank)xaxiss r _reduce_join_reduction_dimsrQ # s K {{}&
!
!
))AKKM''!+R
4FLLJ J >>)..+a/R 88r zstrings.reduce_join reduce_joinz-keep_dims is deprecated, use keepdims instead keep_dimsc t j d|d| }|d}t j d|d| }t | |||| S )NkeepdimsrS FrP reduction_indices)r- rP rU separatorr )r r6 reduce_join_v2)r- rP rS rW r rV rU s r rR rR 1 sb
3
3J4?L(I / /0C0A
C$
r c t j dd| |g 5 t j | }t || }t j
||||| cddd S # 1 sw Y yxY w)a Joins all strings into a single string, or joins along an axis.
This is the reduction operation for the elementwise `tf.strings.join` op.
>>> tf.strings.reduce_join([['abc','123'],
... ['def','456']]).numpy()
b'abc123def456'
>>> tf.strings.reduce_join([['abc','123'],
... ['def','456']], axis=-1).numpy()
array([b'abc123', b'def456'], dtype=object)
>>> tf.strings.reduce_join([['abc','123'],
... ['def','456']],
... axis=-1,
... separator=" ").numpy()
array([b'abc 123', b'def 456'], dtype=object)
Args:
inputs: A `tf.string` tensor.
axis: Which axis to join along. The default behavior is to join all
elements, producing a scalar.
keepdims: If true, retains reduced dimensions with length 1.
separator: a string added between each string being joined.
name: A name for the operation (optional).
Returns:
A `tf.string` tensor.
N
ReduceJoin)r- rV rS rW r )r
name_scoper7 rQ r
rR )r- rP rU rW r inputs_ts r rX rX L sg F ~~dL64.9 $$V,H &x 6D % %
s ;A A(zstrings.lengthc 2 t j | || S )u Computes the length of each string given in the input tensor.
>>> strings = tf.constant(['Hello','TensorFlow', '🙂'])
>>> tf.strings.length(strings).numpy() # default counts bytes
array([ 5, 10, 4], dtype=int32)
>>> tf.strings.length(strings, unit="UTF8_CHAR").numpy()
array([ 5, 10, 1], dtype=int32)
Args:
input: A `Tensor` of type `string`. The strings for which to compute the
length for each element.
name: A name for the operation (optional).
unit: An optional `string` from: `"BYTE", "UTF8_CHAR"`. Defaults to
`"BYTE"`. The unit that is counted to compute string length. One of:
`"BYTE"` (for the number of bytes in each string) or `"UTF8_CHAR"` (for
the number of UTF-8 encoded Unicode code points in each string). Results
are undefined if `unit=UTF8_CHAR` and the `input` strings do not contain
structurally valid UTF-8.
Returns:
A `Tensor` of type `int32`, containing the length of the input string in
the same element of the input tensor.
unitr r
string_length)r r r_ s r ra ra ~ s 6
% %e$T BBr c 2 t j | || S Nr^ r` )r r_ r s r string_length_v2rd s
% %e$T BBr substrz/Use `tf.strings.substr` instead of `tf.substr`.c " t | |||| S )N)r r_ )re r posr* r r_ s r substr_deprecatedri s
sCd 66r zstrings.substrc 6 t j | |||| S rc r
re rg s r re re
uc3T EEr c 6 t j | |||| S rc rk )r rh r* r_ r s r substr_v2rn rl r RegexReplaceStringToHashBucketStringToHashBucketFastStringToHashBucketStrongrZ
StringJoin StringSplitAsString EncodeBase64 DecodeBase64zstrings.to_numberc 0 t j | || S )a Converts each string in the input Tensor to the specified numeric type.
(Note that int32 overflow results in an error while float overflow
results in a rounded value.)
Examples:
>>> tf.strings.to_number("1.55")
>>> tf.strings.to_number("3", tf.int32)
Args:
input: A `Tensor` of type `string`.
out_type: An optional `tf.DType` from: `tf.float32, tf.float64, tf.int32,
tf.int64, tf.uint32, tf.uint64`. Defaults to `tf.float32`.
The numeric type to interpret each string in `string_tensor` as.
name: A name for the operation (optional).
Returns:
A `Tensor` of type `out_type`.
)r string_to_number)r out_typer s r ry ry s 4
) )%4 @@r ry c ` t j d|d| } t j | || S Nr
string_tensor)r r6 r ry )r} rz r r s r string_to_number_v1r~ s2 88
uo}6- ) )-4 HHr zstrings.to_hash_bucketc 0 t j | || S )a Converts each string in the input Tensor to its hash mod by a number of buckets.
The hash function is deterministic on the content of the string within the
process.
Note that the hash function may change from time to time.
This functionality will be deprecated and it's recommended to use
`tf.strings.to_hash_bucket_fast()` or `tf.strings.to_hash_bucket_strong()`.
Examples:
>>> tf.strings.to_hash_bucket(["Hello", "TensorFlow", "2.x"], 3)
Args:
input: A `Tensor` of type `string`.
num_buckets: An `int` that is `>= 1`. The number of buckets.
name: A name for the operation (optional).
Returns:
A `Tensor` of type `int64`.
)r
string_to_hash_bucket)r num_bucketsr s r r r s 8
- -e[$ GGr r c ` t j d|d| } t j | || S r| )r r6 r
r )r} r r r s r string_to_hash_bucket_v1r s2 88
uo}6- - -m[$ OOr z strings.join string_joinc 2 t j | || S )a Perform element-wise concatenation of a list of string tensors.
Given a list of string tensors of same shape, performs element-wise
concatenation of the strings of the same index in all tensors.
>>> tf.strings.join(['abc','def']).numpy()
b'abcdef'
>>> tf.strings.join([['abc','123'],
... ['def','456'],
... ['ghi','789']]).numpy()
array([b'abcdefghi', b'123456789'], dtype=object)
>>> tf.strings.join([['abc','123'],
... ['def','456']],
... separator=" ").numpy()
array([b'abc def', b'123 456'], dtype=object)
The reduction version of this elementwise operation is
`tf.strings.reduce_join`
Args:
inputs: A list of `tf.Tensor` objects of same size and `tf.string` dtype.
separator: A string added between each string being joined.
name: A name for the operation (optional).
Returns:
A `tf.string` tensor.
rW r )r
r )r- rW r s r r r ' s @
# #Fid KKr z strings.unsorted_segment_joinc 6 t j | |||| S )aI Joins the elements of `inputs` based on `segment_ids`.
Computes the string join along segments of a tensor.
Given `segment_ids` with rank `N` and `data` with rank `N+M`:
```
output[i, k1...kM] = strings.join([data[j1...jN, k1...kM])
```
where the join is over all `[j1...jN]` such that `segment_ids[j1...jN] = i`.
Strings are joined in row-major order.
For example:
>>> inputs = ['this', 'a', 'test', 'is']
>>> segment_ids = [0, 1, 1, 0]
>>> num_segments = 2
>>> separator = ' '
>>> tf.strings.unsorted_segment_join(inputs, segment_ids, num_segments,
... separator).numpy()
array([b'this is', b'a test'], dtype=object)
>>> inputs = [['Y', 'q', 'c'], ['Y', '6', '6'], ['p', 'G', 'a']]
>>> segment_ids = [1, 0, 1]
>>> num_segments = 2
>>> tf.strings.unsorted_segment_join(inputs, segment_ids, num_segments,
... separator=':').numpy()
array([[b'Y', b'6', b'6'],
[b'Y:p', b'q:G', b'c:a']], dtype=object)
Args:
inputs: A list of `tf.Tensor` objects of type `tf.string`.
segment_ids: A tensor whose shape is a prefix of `inputs.shape` and whose
type must be `tf.int32` or `tf.int64`. Negative segment ids are not
supported.
num_segments: A scalar of type `tf.int32` or `tf.int64`. Must be
non-negative and larger than any segment id.
separator: The separator to use when joining. Defaults to `""`.
name: A name for the operation (optional).
Returns:
A `tf.string` tensor representing the concatenated values, using the given
separator.
r )r
unsorted_segment_join)r- segment_ids num_segmentsrW r s r r r J s% j
- - k<94
I Ir )N)TN)z{} N)NTN)NrF )NNrA NNN)NFrA N)NBYTE)r N)NNNN)rA N):__doc__numpyrJ tensorflow.python.frameworkr r r r r tensorflow.python.opsr r r
r $tensorflow.python.ops.gen_string_opstensorflow.python.utilr
r r r tensorflow.python.util.tf_exportr register_unary_elementwise_apiadd_dispatch_supportr deprecated_endpointsr r, r9 rC rQ deprecated_argsrR rX ra rd
deprecatedri re rn NotDifferentiablefloat32ry r~ r r r r as_string
decode_base64
encode_base64 string_lower string_upperunicode_transcode string_stripstring_to_hash_bucket_fast string_to_hash_bucket_strongunicode_script r r r s "
3 . + 5 3 + 1 0 *
3 8 . + 6 %& (( / ) '/* *::BB !8/ JL (( !!!/2! 3 )L
!H ;1 ;1@4 > F F #+ (( H ) ,H8 ')@AB ((
P ) CP $2#G#G#O#O >~} => !!!-0 L 1 ? L@ *+ %' # 4I ,4Ip ( ' '(@(@ A ' ' '(D(D E ' ' '(D(D E ' ' '(C(C D ' ' '(C(C D ' ' '(H(H I ' ' '(C(C D ' ' '--/ ' ' '//1 ' ' '(E(E Fr