AVhw zdZddlZddlZddlZddlZddlZddlZddlm Z ddl m Z ddl mZddlmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddlm Z ddlm!Z!ddlm"Z"ddlm#Z#ddlm$Z$ddlm%Z%ddlm Z ddlm&Z&ddlm'Z'ddlm(Z(ddlm)Z)ddlm*Z*ddlm+Z+ddl,m-Z-ddl,m.Z/ddl0m1Z1ddl0m2Z3ddl0m4Z4dd l5m6Z6dd!l7m8Z8dd"l7m9Z9dd#l7m:Z:dd$l;mZ>dd&l?m@Z@dZAd'ZBd(ZCd)ZDGd*d+eEZFe>d,g-Gd.d/eFZGd0ZHe@jeCe>d1g-e8jdeDd2ZKe@jeCe>d3e8jdeD dd4ZLe@jeCe>d5g-e8jdeD dd6ZMe@jeCe>d7g-e8jdeD dd8ZNe@jeCe>d9e8jdeDd:dejdfd;ZPe@jeCe>d<e8jdeDd=ZQe@jeCe>d>e8jdeDejfd?ZSe@jeCe>d@g-e8jdeDdddejfdAZTe@jeCe>d@g-e8jdeDdejdddfdBZUe@jeCe>dCe8jdeD ddDZVe@jeCe>dEe8jdeDddFZWe@jeCe>dGe8jdeDdHZXe@jeCe>dIe8jdeDejfdJZYe@jeCe>dKe8jddLddMZZe>dNg-GdOdPejZ\dQZ]dRZ^dSZ_GdTdUejZ`dVZae>dWg-ejGdXdYejZce>dZg-Gd[d\eEZddd]Zed^ZfejGd_d`e\ejejd`daZiejGdbdce\e`ejejejdcddZkejGdedfe\ecejejejdfdgZmdhZnGdidje1jZpejGdkdle\ecejejejdldmZqdnZrejGdodpe`ejejdpdqZsejGdrdse`ejejdsdtZtejGdudve`ejejdvdwZuejGdxdye`ejejdydzZvejGd{d|e`ejejd|d}ZwejGd~de`ejejddZxdZydZzdZ{ejGdde\ecejejejddZ|dZ}ejGdde`ejejddZdZdZdZy)awThis API defines FeatureColumn abstraction. FeatureColumns provide a high level abstraction for ingesting and representing features. FeatureColumns can also be transformed into a generic input layer for custom models using `input_layer`. NOTE: Functions prefixed with "_" indicate experimental or private parts of the API subject to change, and should not be relied upon! N) lookup_ops)readers)context)feature_column)feature_column_v2_types) serialization)utils)dtypes)ops) sparse_tensor)tensor) tensor_shape) array_ops)array_ops_stack) check_ops)cond) embedding_ops)init_ops)math_ops) parsing_ops) sparse_ops) string_ops)variable_scope) variables)gfile) tf_logging) autotrackable)base)data_structures)checkpoint_utils) deprecation)nest) tf_inspect)collections_abc) tf_export) doc_controlsz`The old _FeatureColumn APIs are being deprecated. Please use the new FeatureColumn APIs instead.a Warning: tf.feature_column is not recommended for new code. Instead, feature preprocessing can be done directly using either [Keras preprocessing layers](https://www.tensorflow.org/guide/migrate/migrating_feature_columns) or through the one-stop utility [`tf.keras.utils.FeatureSpace`](https://www.tensorflow.org/api_docs/python/tf/keras/utils/FeatureSpace) built on top of them. See the [migration guide](https://tensorflow.org/guide/migrate) for details. zUse Keras preprocessing layers instead, either directly or via the `tf.keras.utils.FeatureSpace` utility. Each of `tf.feature_column.*` has a functional equivalent in `tf.keras.layers` for feature preprocessing when training a Keras model.c>eZdZdZ d dZdZdZdZdZdZ y) StateManageraManages the state associated with FeatureColumns. Some `FeatureColumn`s create variables or resources to assist their computation. The `StateManager` is responsible for creating and storing these objects since `FeatureColumn`s are supposed to be stateless configuration only. Nc&~~~~~~~td)Creates a new variable. Args: feature_column: A `FeatureColumn` object this variable corresponds to. name: variable name. shape: variable shape. dtype: The type of the variable. Defaults to `self.dtype` or `float32`. trainable: Whether this variable is trainable or not. use_resource: If true, we use resource variables. Otherwise we use RefVariable. initializer: initializer instance (callable). Returns: The created variable. zStateManager.create_variableNotImplementedError)selfrnameshapedtype trainable use_resource initializers b/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/feature_column/feature_column_v2.pycreate_variablezStateManager.create_variablees. eUI|[ < ==c~~td)zAdds an existing variable to the state. Args: feature_column: A `FeatureColumn` object to associate this variable with. var: The variable. zStateManager.add_variabler+)r-rvars r4 add_variablezStateManager.add_variables  9 ::r6c~~td)Returns an existing variable. Args: feature_column: A `FeatureColumn` object this variable corresponds to. name: variable name. zStateManager.get_varr+r-rr.s r4 get_variablezStateManager.get_variables  4 55r6c~~~td)a'Creates a new resource. Resources can be things such as tables, variables, trackables, etc. Args: feature_column: A `FeatureColumn` object this resource corresponds to. name: Name of the resource. resource: The resource. Returns: The created resource. zStateManager.add_resourcer+)r-rr.resources r4 add_resourcezStateManager.add_resources h 9 ::r6c~~td)zReturns true iff a resource with same name exists. Resources can be things such as tables, variables, trackables, etc. Args: feature_column: A `FeatureColumn` object this variable corresponds to. name: Name of the resource. zStateManager.has_resourcer+r<s r4 has_resourcezStateManager.has_resource  9 ::r6c~~td)zReturns an already created resource. Resources can be things such as tables, variables, trackables, etc. Args: feature_column: A `FeatureColumn` object this variable corresponds to. name: Name of the resource. zStateManager.get_resourcer+r<s r4 get_resourcezStateManager.get_resourcerCr6NTTN) __name__ __module__ __qualname____doc__r5r9r=r@rBrEr6r4r(r(\s3! $#'"&>4;6; ; ;r6r(z(__internal__.feature_column.StateManager)v1c>eZdZdZdZ d dZdZdZdZdZ y) _StateManagerImpla!Manages the state of DenseFeatures and LinearLayer. Some `FeatureColumn`s create variables or resources to assist their computation. The `StateManager` is responsible for creating and storing these objects since `FeatureColumn`s are supposed to be stateless configuration only. c||_||_|j9t|jds#tj|j_t jd|_t jd|_ y)zCreates an _StateManagerImpl object. Args: layer: The input layer this state manager is associated with. trainable: Whether by default, variables created are trainable or not. N _resourcesciSNrKrKr6r4z,_StateManagerImpl.__init__..sRr6ciSrRrKrKr6r4rSz,_StateManagerImpl.__init__..s"r6) _trainable_layerhasattrrMappingrP collections defaultdict_cols_to_vars_map_cols_to_resources_map)r-layerr1s r4__init__z_StateManagerImpl.__init__sd DODK {{wt{{L'I.668dkk(44Z@D"-"9"9*"EDr6Nc ||j|vr tdtj|j5|jj |||||j xr||tj}dddttjr`|D]Z} |dzt| jjdz} |jj| |j dz| z\nFt|tj"r,|jj||j dz|z||j||<|S#1swYxYw)r*zVariable already exists.)r.r/r0r3r1r2getterN/r)r[ ValueError trackable#no_manual_dependency_tracking_scoperV add_weightrUrr= isinstancerPartitionedVariablestr_get_save_slice_info var_offset_track_trackabler. Trackable) r-rr.r/r0r1r2r3r8v part_names r4r5z!_StateManagerImpl.create_variablesI. t%%n55 1 22  6 6t{{ C . KK " "!OO1 # ,, # .c .#y445O!3JQ%;%;%=%H%H%K!LL  $$Q(;(;c(AI(MNO C,, - $$S.*=*=*Cd*JK36D>*40 J+ . .s ?EE c^||j|vr|j||Std)r;zVariable does not exist.)r[rbr<s r4r=z_StateManagerImpl.get_variables9 t%%n55  # #N 3D 99 / 00r6c||j||<|jt|tjr|j |jj vr5tj|jj |j <||jj |j vr'||jj |j |<yyyy)a0Creates a new resource. Resources can be things such as tables, variables, trackables, etc. Args: feature_column: A `FeatureColumn` object this resource corresponds to. resource_name: Name of the resource. resource: The resource. Returns: The created resource. N) r\rVrfrcrlr.rPrrX)r-r resource_namer?s r4r@z_StateManagerImpl.add_resourcesBJD/ > {{:h 8K8K#L   DKK$:$: :6E6M6M6O ~223 dkk44^5H5HI IEM ~223MB J $Mr6c$||j|vS)aReturns true iff a resource with same name exists. Resources can be things such as tables, variables, trackables, etc. Args: feature_column: A `FeatureColumn` object this variable corresponds to. resource_name: Name of the resource. )r\r-rrqs r4rBz_StateManagerImpl.has_resource)s D77G GGr6cz||jvs||j|vr td|j||S)zReturns an already created resource. Resources can be things such as tables, variables, trackables, etc. Args: feature_column: A `FeatureColumn` object this variable corresponds to. resource_name: Name of the resource. zResource does not exist.)r\rbrss r4rEz_StateManagerImpl.get_resource4sG d999T88HH 1 22  & &~ 6} EEr6rF) rGrHrIrJr^r5r=r@rBrErKr6r4rNrNs8 F$! $#'"&1f 1N. H Fr6rNcft|}i}tjdd|j5t |}|D]J}tjdt |j 5|j||||<dddL ddd|S#1swYaxYw#1swY|SxYw)aReturns transformed features based on features columns passed in. Please note that most probably you would not need to use this function. Please check `input_layer` and `linear_model` to see whether they will satisfy your use case or not. Example: ```python # Define features and transformations crosses_a_x_b = crossed_column( columns=["sparse_feature_a", "sparse_feature_b"], hash_bucket_size=10000) price_buckets = bucketized_column( source_column=numeric_column("price"), boundaries=[...]) columns = [crosses_a_x_b, price_buckets] features = tf.io.parse_example(..., features=make_parse_example_spec(columns)) transformed = transform_features(features=features, feature_columns=columns) assertCountEqual(columns, transformed.keys()) ``` Args: features: A mapping from key to tensors. `FeatureColumn`s look up via these keys. For example `numeric_column('price')` will look at 'price' key in this dict. Values can be a `SparseTensor` or a `Tensor` depends on corresponding `FeatureColumn`. feature_columns: An iterable containing all the `FeatureColumn`s. state_manager: A StateManager object that holds the FeatureColumn state. Returns: A `dict` mapping `FeatureColumn` to `Tensor` and `SparseTensor` values. Ntransform_features) default_namevaluesrw)_normalize_feature_columnsr name_scoperxFeatureTransformationCache(_sanitize_column_name_for_variable_scoper.get)featuresfeature_columns state_manageroutputstransformation_cachecolumns r4_transform_features_v2rCsD/?/ ' ~~ -hoo6GIJ5h?!J >> ? L NJ/226=IJJJJ . JJ J .s#;B&/B B&B# B&&B0z&feature_column.make_parse_example_specc Li}|D]}t|tjstdj ||j }t j|D]1\}}||vs |||k7stdj |||||j||S)aCreates parsing spec dictionary from input feature_columns. The returned dictionary can be used as arg 'features' in `tf.io.parse_example`. Typical usage example: ```python # Define features and transformations feature_a = tf.feature_column.categorical_column_with_vocabulary_file(...) feature_b = tf.feature_column.numeric_column(...) feature_c_bucketized = tf.feature_column.bucketized_column( tf.feature_column.numeric_column("feature_c"), ...) feature_a_x_feature_c = tf.feature_column.crossed_column( columns=["feature_a", feature_c_bucketized], ...) feature_columns = set( [feature_b, feature_c_bucketized, feature_a_x_feature_c]) features = tf.io.parse_example( serialized=serialized_examples, features=tf.feature_column.make_parse_example_spec(feature_columns)) ``` For the above example, make_parse_example_spec would return the dict: ```python { "feature_a": parsing_ops.VarLenFeature(tf.string), "feature_b": parsing_ops.FixedLenFeature([1], dtype=tf.float32), "feature_c": parsing_ops.FixedLenFeature([1], dtype=tf.float32) } ``` Args: feature_columns: An iterable containing all feature columns. All items should be instances of classes derived from `FeatureColumn`. Returns: A dict mapping each feature key to a `FixedLenFeature` or `VarLenFeature` value. Raises: ValueError: If any of the given `feature_columns` is not a `FeatureColumn` instance. z>All feature_columns must be FeatureColumn instances. Given: {}zHfeature_columns contain different parse_spec for key {}. Given {} and {}) rffc_types FeatureColumnrbformatparse_example_specsix iteritemsupdate)rresultrconfigkeyvalues r4make_parse_example_spec_v2rrsf & f fh44 5 ##)6&> 33  & &FmmF+P U 5F3K///5vc5&+/NP PP MM&  -r6zfeature_column.embedding_columnc T||dkrtdj||du|duk7r td|/t|s$tdj|j|-t j ddt j|z }t||||||||| S) a `DenseColumn` that converts from sparse, categorical input. Use this when your inputs are sparse, but you want to convert them to a dense representation (e.g., to feed to a DNN). Args: categorical_column: A `CategoricalColumn` created by a `categorical_column_with_*` function. This column produces the sparse IDs that are inputs to the embedding lookup. dimension: An integer specifying dimension of the embedding, must be > 0. combiner: A string specifying how to reduce if there are multiple entries in a single row. Currently 'mean', 'sqrtn' and 'sum' are supported, with 'mean' the default. 'sqrtn' often achieves good accuracy, in particular with bag-of-words columns. Each of this can be thought as example level normalizations on the column. For more information, see `tf.embedding_lookup_sparse`. initializer: A variable initializer function to be used in embedding variable initialization. If not specified, defaults to `truncated_normal_initializer` with mean `0.0` and standard deviation `1/sqrt(dimension)`. ckpt_to_load_from: String representing checkpoint name/pattern from which to restore column weights. Required if `tensor_name_in_ckpt` is not `None`. tensor_name_in_ckpt: Name of the `Tensor` in `ckpt_to_load_from` from which to restore the column weights. Required if `ckpt_to_load_from` is not `None`. max_norm: If not `None`, embedding values are l2-normalized to this value. trainable: Whether or not the embedding is trainable. Default is True. use_safe_embedding_lookup: If true, uses safe_embedding_lookup_sparse instead of embedding_lookup_sparse. safe_embedding_lookup_sparse ensures there are no empty rows and all weights and ids are positive at the expense of extra compute cost. This only applies to rank 2 (NxM) shaped input tensors. Defaults to true, consider turning off if the above checks are not needed. Note that having empty rows will not trigger any error though the output result might be 0 or omitted. Returns: `DenseColumn` that converts from sparse input. Raises: ValueError: if `dimension` not > 0. ValueError: if exactly one of `ckpt_to_load_from` and `tensor_name_in_ckpt` is specified. ValueError: if `initializer` is specified and is not callable. RuntimeError: If eager execution is enabled. NInvalid dimension {}.PMust specify both `ckpt_to_load_from` and `tensor_name_in_ckpt` or none of them.zGinitializer must be callable if specified. Embedding of column_name: {}meanstddev categorical_column dimensioncombinerr3ckpt_to_load_fromtensor_name_in_ckptmax_normr1use_safe_embedding_lookup) rbrcallabler.rtruncated_normal_initializermathsqrtEmbeddingColumnrs r4embedding_columnrsrY] ,33I> ??4%8D%@A > ??(= 44:F+0052 3377 TYYy113K +)- 9 ; ;r6z'feature_column.shared_embedding_columnsc ~tjr td||dkrtdj ||du|duk7r td|t |s td|-t jddtj|z }t|d } | d } | j} t| tjs$td j | t| t| tj t"tj$t&frA| j(} t| tj t"tj$t&frA| ddD]} t| tj t"tj$t&frA| j(} t| tj t"tj$t&frAt| t| s/tdj | t| | t| | | jk7stdj | | | | j|sdj+d| D}|dz }g}|D]0}|j-tj.||||||||||  2|S)aList of dense columns that convert from sparse, categorical input. This is similar to `embedding_column`, except that it produces a list of embedding columns that share the same embedding weights. Use this when your inputs are sparse and of the same type (e.g. watched and impression video IDs that share the same vocabulary), and you want to convert them to a dense representation (e.g., to feed to a DNN). Inputs must be a list of categorical columns created by any of the `categorical_column_*` function. They must all be of the same type and have the same arguments except `key`. E.g. they can be categorical_column_with_vocabulary_file with the same vocabulary_file. Some or all columns could also be weighted_categorical_column. Args: categorical_columns: List of categorical columns created by a `categorical_column_with_*` function. These columns produce the sparse IDs that are inputs to the embedding lookup. All columns must be of the same type and have the same arguments except `key`. E.g. they can be categorical_column_with_vocabulary_file with the same vocabulary_file. Some or all columns could also be weighted_categorical_column. dimension: An integer specifying dimension of the embedding, must be > 0. combiner: A string specifying how to reduce if there are multiple entries in a single row. Currently 'mean', 'sqrtn' and 'sum' are supported, with 'mean' the default. 'sqrtn' often achieves good accuracy, in particular with bag-of-words columns. Each of this can be thought as example level normalizations on the column. For more information, see `tf.embedding_lookup_sparse`. initializer: A variable initializer function to be used in embedding variable initialization. If not specified, defaults to `truncated_normal_initializer` with mean `0.0` and standard deviation `1/sqrt(dimension)`. shared_embedding_collection_name: Optional name of the collection where shared embedding weights are added. If not given, a reasonable name will be chosen based on the names of `categorical_columns`. This is also used in `variable_scope` when creating shared embedding weights. ckpt_to_load_from: String representing checkpoint name/pattern from which to restore column weights. Required if `tensor_name_in_ckpt` is not `None`. tensor_name_in_ckpt: Name of the `Tensor` in `ckpt_to_load_from` from which to restore the column weights. Required if `ckpt_to_load_from` is not `None`. max_norm: If not `None`, each embedding is clipped if its l2-norm is larger than this value, before combining. trainable: Whether or not the embedding is trainable. Default is True. use_safe_embedding_lookup: If true, uses safe_embedding_lookup_sparse instead of embedding_lookup_sparse. safe_embedding_lookup_sparse ensures there are no empty rows and all weights and ids are positive at the expense of extra compute cost. This only applies to rank 2 (NxM) shaped input tensors. Defaults to true, consider turning off if the above checks are not needed. Note that having empty rows will not trigger any error though the output result might be 0 or omitted. Returns: A list of dense columns that converts from sparse input. The order of results follows the ordering of `categorical_columns`. Raises: ValueError: if `dimension` not > 0. ValueError: if any of the given `categorical_columns` is of different type or has different arguments than the others. ValueError: if exactly one of `ckpt_to_load_from` and `tensor_name_in_ckpt` is specified. ValueError: if `initializer` is specified and is not callable. RuntimeError: if eager execution is enabled. Kshared_embedding_columns are not supported when eager execution is enabled.Nrrr*initializer must be callable if specified.r?rc|jSrRr.xs r4rSz*shared_embedding_columns..g QVVr6rrzXAll categorical_columns must be subclasses of _CategoricalColumn. Given: {}, of type: {}To use shared_embedding_column, all categorical_columns must have the same type, or be weighted_categorical_column or sequence column of the same type. Given column: {} of type: {} does not match given column: {} of type: {}zTo use shared_embedding_column, all categorical_columns must have the same number of buckets. ven column: {} with buckets: {} does not match column: {} with buckets: {}_c34K|]}|jywrRr.0cs r4 z+shared_embedding_columns../O1/O_shared_embedding) rr3rr shared_embedding_collection_namerrrr1r)rexecuting_eagerly RuntimeErrorrbrrrrrrsorted _num_bucketsrffc_old_CategoricalColumntype_WeightedCategoricalColumnWeightedCategoricalColumn_SequenceCategoricalColumnSequenceCategoricalColumnrjoinappend_SharedEmbeddingColumn)categorical_columnsrrr3rrrrr1rsorted_columnsc0 num_bucketsrrrs r4shared_embedding_columnsrs^  / 00Y] ,33I> ??4%8D%@A > ??(= A BB77 dii 224K -3CD.a"+ B11 2  !!'DH!5 77   + + #  + + # % &  B   + + #  + + # % & !" 3a   - - %  - - %  ' (  a   - - %  - - %  ' ( ab "  #$*6"d2h47#C  EE ann$  228&+q!..32 33!3, *'*xx/O/O'O$$(;;$ &# Bf MM%%%#-M/ 3&? A B B -r6z feature_column.shared_embeddingsc vtjr td||dkrtdj ||du|duk7r td|t |s td|-t jddtj|z }t|d } | d } | j} t| ts$td j | t| t| tt fr#| j"} t| tt fr#| ddD]} t| tt fr#| j"} t| tt fr#t| t| s/tdj | t| | t| | | jk7stdj | | | | j|sdj%d| D}|dz }t'||||| ||| }g}|D]}|j)|||||S)ayList of dense columns that convert from sparse, categorical input. This is similar to `embedding_column`, except that it produces a list of embedding columns that share the same embedding weights. Use this when your inputs are sparse and of the same type (e.g. watched and impression video IDs that share the same vocabulary), and you want to convert them to a dense representation (e.g., to feed to a DNN). Inputs must be a list of categorical columns created by any of the `categorical_column_*` function. They must all be of the same type and have the same arguments except `key`. E.g. they can be categorical_column_with_vocabulary_file with the same vocabulary_file. Some or all columns could also be weighted_categorical_column. Args: categorical_columns: List of categorical columns created by a `categorical_column_with_*` function. These columns produce the sparse IDs that are inputs to the embedding lookup. All columns must be of the same type and have the same arguments except `key`. E.g. they can be categorical_column_with_vocabulary_file with the same vocabulary_file. Some or all columns could also be weighted_categorical_column. dimension: An integer specifying dimension of the embedding, must be > 0. combiner: A string specifying how to reduce if there are multiple entries in a single row. Currently 'mean', 'sqrtn' and 'sum' are supported, with 'mean' the default. 'sqrtn' often achieves good accuracy, in particular with bag-of-words columns. Each of this can be thought as example level normalizations on the column. For more information, see `tf.embedding_lookup_sparse`. initializer: A variable initializer function to be used in embedding variable initialization. If not specified, defaults to `truncated_normal_initializer` with mean `0.0` and standard deviation `1/sqrt(dimension)`. shared_embedding_collection_name: Optional collective name of these columns. If not given, a reasonable name will be chosen based on the names of `categorical_columns`. ckpt_to_load_from: String representing checkpoint name/pattern from which to restore column weights. Required if `tensor_name_in_ckpt` is not `None`. tensor_name_in_ckpt: Name of the `Tensor` in `ckpt_to_load_from` from which to restore the column weights. Required if `ckpt_to_load_from` is not `None`. max_norm: If not `None`, each embedding is clipped if its l2-norm is larger than this value, before combining. trainable: Whether or not the embedding is trainable. Default is True. use_safe_embedding_lookup: If true, uses safe_embedding_lookup_sparse instead of embedding_lookup_sparse. safe_embedding_lookup_sparse ensures there are no empty rows and all weights and ids are positive at the expense of extra compute cost. This only applies to rank 2 (NxM) shaped input tensors. Defaults to true, consider turning off if the above checks are not needed. Note that having empty rows will not trigger any error though the output result might be 0 or omitted. Returns: A list of dense columns that converts from sparse input. The order of results follows the ordering of `categorical_columns`. Raises: ValueError: if `dimension` not > 0. ValueError: if any of the given `categorical_columns` is of different type or has different arguments than the others. ValueError: if exactly one of `ckpt_to_load_from` and `tensor_name_in_ckpt` is specified. ValueError: if `initializer` is specified and is not callable. RuntimeError: if eager execution is enabled. rNrrrrrrrc|jSrRrrs r4rSz-shared_embedding_columns_v2..rr6rrzWAll categorical_columns must be subclasses of CategoricalColumn. Given: {}, of type: {}rzTo use shared_embedding_column, all categorical_columns must have the same number of buckets. Given column: {} with buckets: {} does not match column: {} with buckets: {}rc34K|]}|jywrRrrs r4rz.shared_embedding_columns_v2.. rrr)rrr)rrrrbrrrrrrrrrfCategoricalColumnrrrrrSharedEmbeddingColumnCreatorr)rrrr3rrrrr1rrrrrcolumn_creatorrrs r4shared_embedding_columns_v2rsl`  / 00Y] ,33I> ??4%8D%@A > ??(= A BB77 dii 224K -3CD.a"+ B) *  !!'DH!5 77 213LMN  B 213LMN !" 2a Q24MN O  a Q24MN O ab "  #$*6"d2h47#C  EE amm#  228&+q!--31 222 *'*xx/O/O'O$$(;;$//1D9>!. &#Nf MM%8 MNN -r6zfeature_column.numeric_column)rcHt||}|js'|jstdj ||t j ||||}|%t|stdj |t j|t|||||S)a Represents real valued or numerical features. Example: Assume we have data with two features `a` and `b`. >>> data = {'a': [15, 9, 17, 19, 21, 18, 25, 30], ... 'b': [5.0, 6.4, 10.5, 13.6, 15.7, 19.9, 20.3 , 0.0]} Let us represent the features `a` and `b` as numerical features. >>> a = tf.feature_column.numeric_column('a') >>> b = tf.feature_column.numeric_column('b') Feature column describe a set of transformations to the inputs. For example, to "bucketize" feature `a`, wrap the `a` column in a `feature_column.bucketized_column`. Providing `5` bucket boundaries, the bucketized_column api will bucket this feature in total of `6` buckets. >>> a_buckets = tf.feature_column.bucketized_column(a, ... boundaries=[10, 15, 20, 25, 30]) Create a `DenseFeatures` layer which will apply the transformations described by the set of `tf.feature_column` objects: >>> feature_layer = tf.keras.layers.DenseFeatures([a_buckets, b]) >>> print(feature_layer(data)) tf.Tensor( [[ 0. 0. 1. 0. 0. 0. 5. ] [ 1. 0. 0. 0. 0. 0. 6.4] [ 0. 0. 1. 0. 0. 0. 10.5] [ 0. 0. 1. 0. 0. 0. 13.6] [ 0. 0. 0. 1. 0. 0. 15.7] [ 0. 0. 1. 0. 0. 0. 19.9] [ 0. 0. 0. 0. 1. 0. 20.3] [ 0. 0. 0. 0. 0. 1. 0. ]], shape=(8, 7), dtype=float32) Args: key: A unique string identifying the input feature. It is used as the column name and the dictionary key for feature parsing configs, feature `Tensor` objects, and feature columns. shape: An iterable of integers specifies the shape of the `Tensor`. An integer can be given which means a single dimension `Tensor` with given width. The `Tensor` representing the column will have the shape of [batch_size] + `shape`. default_value: A single value compatible with `dtype` or an iterable of values compatible with `dtype` which the column takes on during `tf.Example` parsing if data is missing. A default value of `None` will cause `tf.io.parse_example` to fail if an example does not contain this column. If a single value is provided, the same value will be applied as the default value for every item. If an iterable of values is provided, the shape of the `default_value` should be equal to the given `shape`. dtype: defines the type of values. Default value is `tf.float32`. Must be a non-quantized, real integer or floating point type. normalizer_fn: If not `None`, a function that can be used to normalize the value of the tensor after `default_value` is applied for parsing. Normalizer function takes the input `Tensor` as its argument, and returns the output `Tensor`. (e.g. lambda x: (x - 3.0) / 4.2). Please note that even though the most common use case of this function is normalization, it can be used for any kind of Tensorflow transformations. Returns: A `NumericColumn`. Raises: TypeError: if any dimension in shape is not an int ValueError: if any dimension in shape is not a positive integer TypeError: if `default_value` is an iterable but not compatible with `shape` TypeError: if `default_value` is not compatible with `dtype`. ValueError: if `dtype` is not convertible to `tf.float32`. z6dtype must be convertible to float. dtype: {}, key: {}z+normalizer_fn must be a callable. Given: {})r/ default_valuer0 normalizer_fn) _check_shape is_integer is_floatingrbrfc_utilscheck_default_valuer TypeErrorassert_key_is_string NumericColumnrr/rr0rs r4numeric_columnr1sb uc "%   e// **0&*< >>..umUCP-x '> 5<<]K MM $  ! !  ##r6z feature_column.bucketized_columnct|ttjfst dj |t |jdkDrt dj ||s t dt|tst|ts t dtt |dz D]}||||dzk\st dt|t|S)aRepresents discretized dense input bucketed by `boundaries`. Buckets include the left boundary, and exclude the right boundary. Namely, `boundaries=[0., 1., 2.]` generates buckets `(-inf, 0.)`, `[0., 1.)`, `[1., 2.)`, and `[2., +inf)`. For example, if the inputs are ```python boundaries = [0, 10, 100] input tensor = [[-5, 10000] [150, 10] [5, 100]] ``` then the output will be ```python output = [[0, 3] [3, 2] [1, 3]] ``` Example: ```python price = tf.feature_column.numeric_column('price') bucketized_price = tf.feature_column.bucketized_column( price, boundaries=[...]) columns = [bucketized_price, ...] features = tf.io.parse_example( ..., features=tf.feature_column.make_parse_example_spec(columns)) dense_tensor = tf.keras.layers.DenseFeatures(columns)(features) ``` A `bucketized_column` can also be crossed with another categorical column using `crossed_column`: ```python price = tf.feature_column.numeric_column('price') # bucketized_column converts numerical feature to a categorical one. bucketized_price = tf.feature_column.bucketized_column( price, boundaries=[...]) # 'keywords' is a string feature. price_x_keywords = tf.feature_column.crossed_column( [bucketized_price, 'keywords'], 50K) columns = [price_x_keywords, ...] features = tf.io.parse_example( ..., features=tf.feature_column.make_parse_example_spec(columns)) dense_tensor = tf.keras.layers.DenseFeatures(columns)(features) linear_model = tf.keras.experimental.LinearModel(units=...)(dense_tensor) ``` Args: source_column: A one-dimensional dense column which is generated with `numeric_column`. boundaries: A sorted list or tuple of floats specifying the boundaries. Returns: A `BucketizedColumn`. Raises: ValueError: If `source_column` is not a numeric column, or if it is not one-dimensional. ValueError: If `boundaries` is not a sorted list or tuple. zIsource_column must be a column generated with numeric_column(). Given: {}rz7source_column must be one-dimensional column. Given: {}zboundaries must not be empty.z!boundaries must be a sorted list.) rfrr_NumericColumnrbrlenr/listtuplerangeBucketizedColumn) source_column boundariesis r4bucketized_columnrsL MM63H3H#I J  F=) ++   ! !!' !6 88  4 55 Z &*Z*G 8 99 Z1$ % 1. The number of buckets. dtype: The type of features. Only string and integer types are supported. Returns: A `HashedCategoricalColumn`. Raises: ValueError: `hash_bucket_size` is not greater than 1. ValueError: `dtype` is neither string nor integer. z%hash_bucket_size must be set. key: {}rzBhash_bucket_size must be at least 1. hash_bucket_size: {}, key: {}column_name: {}prefix)rbrrrassert_string_or_intHashedCategoricalColumnrhash_bucket_sizer0s r4#categorical_column_with_hash_bucketrst ?FFsK LL 55;V)360 11 $ .?.F.Fs.KL &6 >>r6z6feature_column.categorical_column_with_vocabulary_filec"t||||||S)a4A `CategoricalColumn` with a vocabulary file. Use this when your inputs are in string or integer format, and you have a vocabulary file that maps each value to an integer ID. By default, out-of-vocabulary values are ignored. Use either (but not both) of `num_oov_buckets` and `default_value` to specify how to include out-of-vocabulary values. For input dictionary `features`, `features[key]` is either `Tensor` or `SparseTensor`. If `Tensor`, missing values can be represented by `-1` for int and `''` for string, which will be dropped by this feature column. Example with `num_oov_buckets`: File '/us/states.txt' contains 50 lines, each with a 2-character U.S. state abbreviation. All inputs with values in that file are assigned an ID 0-49, corresponding to its line number. All other values are hashed and assigned an ID 50-54. ```python import tensorflow as tf states = tf.feature_column.categorical_column_with_vocabulary_file( key='states', vocabulary_file='states.txt', vocabulary_size=5, num_oov_buckets=1) columns = [states] features = {'states':tf.constant([['california', 'georgia', 'michigan', 'texas', 'new york'], ['new york', 'georgia', 'california', 'michigan', 'texas']])} linear_prediction = tf.compat.v1.feature_column.linear_model(features, columns) ``` Example with `default_value`: File '/us/states.txt' contains 51 lines - the first line is 'XX', and the other 50 each have a 2-character U.S. state abbreviation. Both a literal 'XX' in input, and other values missing from the file, will be assigned ID 0. All others are assigned the corresponding line number 1-50. ```python import tensorflow as tf states = tf.feature_column.categorical_column_with_vocabulary_file( key='states', vocabulary_file='states.txt', vocabulary_size=6, default_value=0) columns = [states] features = {'states':tf.constant([['california', 'georgia', 'michigan', 'texas', 'new york'], ['new york', 'georgia', 'california', 'michigan', 'texas']])} linear_prediction = tf.compat.v1.feature_column.linear_model(features, columns) ``` And to make an embedding with either: ```python import tensorflow as tf states = tf.feature_column.categorical_column_with_vocabulary_file( key='states', vocabulary_file='states.txt', vocabulary_size=5, num_oov_buckets=1) columns = [tf.feature_column.embedding_column(states, 3)] features = {'states':tf.constant([['california', 'georgia', 'michigan', 'texas', 'new york'], ['new york', 'georgia', 'california', 'michigan', 'texas']])} input_layer = tf.keras.layers.DenseFeatures(columns) dense_tensor = input_layer(features) ``` Args: key: A unique string identifying the input feature. It is used as the column name and the dictionary key for feature parsing configs, feature `Tensor` objects, and feature columns. vocabulary_file: The vocabulary file name. vocabulary_size: Number of the elements in the vocabulary. This must be no greater than length of `vocabulary_file`, if less than length, later values are ignored. If None, it is set to the length of `vocabulary_file`. num_oov_buckets: Non-negative integer, the number of out-of-vocabulary buckets. All out-of-vocabulary inputs will be assigned IDs in the range `[vocabulary_size, vocabulary_size+num_oov_buckets)` based on a hash of the input value. A positive `num_oov_buckets` can not be specified with `default_value`. default_value: The integer ID value to return for out-of-vocabulary feature values, defaults to `-1`. This can not be specified with a positive `num_oov_buckets`. dtype: The type of features. Only string and integer types are supported. Returns: A `CategoricalColumn` with a vocabulary file. Raises: ValueError: `vocabulary_file` is missing or cannot be opened. ValueError: `vocabulary_size` is missing or < 1. ValueError: `num_oov_buckets` is a negative integer. ValueError: `num_oov_buckets` and `default_value` are both specified. ValueError: `dtype` is neither string nor integer. )*categorical_column_with_vocabulary_file_v2)rvocabulary_filevocabulary_sizenum_oov_bucketsrr0s r4'categorical_column_with_vocabulary_filer4s#L 4C4CU4A4C EEr6c |stdj|||jdrd}|tj|stdj||dk(rNt j |d}|jdd}tjrC|j}n2tj|d 5}td |D}dddtjd |||t|t j"s|d krtdj||r<|tdj||dkrtdj||t%j&|dj|t%j(|t+||||dn||dn|||S#1swYxYw)aA `CategoricalColumn` with a vocabulary file. Use this when your inputs are in string or integer format, and you have a vocabulary file that maps each value to an integer ID. By default, out-of-vocabulary values are ignored. Use either (but not both) of `num_oov_buckets` and `default_value` to specify how to include out-of-vocabulary values. For input dictionary `features`, `features[key]` is either `Tensor` or `SparseTensor`. If `Tensor`, missing values can be represented by `-1` for int and `''` for string, which will be dropped by this feature column. Example with `num_oov_buckets`: File `'/us/states.txt'` contains 50 lines, each with a 2-character U.S. state abbreviation. All inputs with values in that file are assigned an ID 0-49, corresponding to its line number. All other values are hashed and assigned an ID 50-54. ```python states = categorical_column_with_vocabulary_file( key='states', vocabulary_file='/us/states.txt', vocabulary_size=50, num_oov_buckets=5) columns = [states, ...] features = tf.io.parse_example(..., features=make_parse_example_spec(columns)) linear_prediction = linear_model(features, columns) ``` Example with `default_value`: File `'/us/states.txt'` contains 51 lines - the first line is `'XX'`, and the other 50 each have a 2-character U.S. state abbreviation. Both a literal `'XX'` in input, and other values missing from the file, will be assigned ID 0. All others are assigned the corresponding line number 1-50. ```python states = categorical_column_with_vocabulary_file( key='states', vocabulary_file='/us/states.txt', vocabulary_size=51, default_value=0) columns = [states, ...] features = tf.io.parse_example(..., features=make_parse_example_spec(columns)) linear_prediction, _, _ = linear_model(features, columns) ``` And to make an embedding with either: ```python columns = [embedding_column(states, 3),...] features = tf.io.parse_example(..., features=make_parse_example_spec(columns)) dense_tensor = input_layer(features, columns) ``` Args: key: A unique string identifying the input feature. It is used as the column name and the dictionary key for feature parsing configs, feature `Tensor` objects, and feature columns. vocabulary_file: The vocabulary file name. vocabulary_size: Number of the elements in the vocabulary. This must be no greater than length of `vocabulary_file`, if less than length, later values are ignored. If None, it is set to the length of `vocabulary_file`. dtype: The type of features. Only string and integer types are supported. default_value: The integer ID value to return for out-of-vocabulary feature values, defaults to `-1`. This can not be specified with a positive `num_oov_buckets`. num_oov_buckets: Non-negative integer, the number of out-of-vocabulary buckets. All out-of-vocabulary inputs will be assigned IDs in the range `[vocabulary_size, vocabulary_size+num_oov_buckets)` based on a hash of the input value. A positive `num_oov_buckets` can not be specified with `default_value`. file_format: The format of the vocabulary file. The format is 'text' by default unless `vocabulary_file` is a string which ends in 'tfrecord.gz'. Accepted alternative value for `file_format` is 'tfrecord_gzip'. Returns: A `CategoricalColumn` with a vocabulary file. Raises: ValueError: `vocabulary_file` is missing or cannot be opened. ValueError: `vocabulary_size` is missing or < 1. ValueError: `num_oov_buckets` is a negative integer. ValueError: `num_oov_buckets` and `default_value` are both specified. ValueError: `dtype` is neither string nor integer. zMissing vocabulary_file in {}.Nz tfrecord.gz tfrecord_gzipz%vocabulary_file in {} does not exist.GZIPrc |dzSNrrK)rrs r4rSz. s !a%r6rb)modec3 K|]}dyw)rNrK)rrs r4rz=categorical_column_with_vocabulary_file_v2..s^Aa^s z]vocabulary_size = %d in %s is inferred from the number of elements in the vocabulary_file %s.rzInvalid vocabulary_size in {}.;Can't specify both num_oov_buckets and default_value in {}.!Invalid num_oov_buckets {} in {}.rr)rrrrrr0 file_format)rbrendswithrExistsrTFRecordDatasetreducerrnumpyGFilesumlogginginforf tensor_libTensorrrrVocabularyFileCategoricalColumn) rrrr0rrr dsfs r4rrsz  5<EEcJ KKo%  " "?F ;b !%78o  " " $)//1 ;;T 2,a^^+, LL %&5sOM OZ%6%6 7Oadtype {} and vocabulary dtype {} do not match, column_name: {}rrvocabulary_listr0rr)rrbrsetr as_dtypenparrayr0rrrrVocabularyListCategoricalColumnr)rrr0rrvocabulary_dtypes r4'categorical_column_with_vocabulary_listr"*sf3#7!#; ?FF S " ## _ #o"66 ?FF S " ##__RXXo%>%D%DE  H O O   :AA 3  !! ;BB3GI ] E +666 HOO #S * ++ .?.F.Fs.KL $ ( O, !%  ''r6z/feature_column.categorical_column_with_identityc|dkrtdj|||&|dks||k\rtdj|||tj|t |||S)a A `CategoricalColumn` that returns identity values. Use this when your inputs are integers in the range `[0, num_buckets)`, and you want to use the input value itself as the categorical ID. Values outside this range will result in `default_value` if specified, otherwise it will fail. Typically, this is used for contiguous ranges of integer indexes, but it doesn't have to be. This might be inefficient, however, if many of IDs are unused. Consider `categorical_column_with_hash_bucket` in that case. For input dictionary `features`, `features[key]` is either `Tensor` or `SparseTensor`. If `Tensor`, missing values can be represented by `-1` for int and `''` for string, which will be dropped by this feature column. In the following examples, each input in the range `[0, 1000000)` is assigned the same value. All other inputs are assigned `default_value` 0. Note that a literal 0 in inputs will result in the same default ID. Linear model: ```python import tensorflow as tf video_id = tf.feature_column.categorical_column_with_identity( key='video_id', num_buckets=1000000, default_value=0) columns = [video_id] features = {'video_id': tf.sparse.from_dense([[2, 85, 0, 0, 0], [33,78, 2, 73, 1]])} linear_prediction = tf.compat.v1.feature_column.linear_model(features, columns) ``` Embedding for a DNN model: ```python import tensorflow as tf video_id = tf.feature_column.categorical_column_with_identity( key='video_id', num_buckets=1000000, default_value=0) columns = [tf.feature_column.embedding_column(video_id, 9)] features = {'video_id': tf.sparse.from_dense([[2, 85, 0, 0, 0], [33,78, 2, 73, 1]])} input_layer = tf.keras.layers.DenseFeatures(columns) dense_tensor = input_layer(features) ``` Args: key: A unique string identifying the input feature. It is used as the column name and the dictionary key for feature parsing configs, feature `Tensor` objects, and feature columns. num_buckets: Range of inputs and outputs is `[0, num_buckets)`. default_value: If set, values outside of range `[0, num_buckets)` will be replaced with this value. If not set, values >= num_buckets will cause a failure while values < 0 will be dropped. Returns: A `CategoricalColumn` that returns identity values. Raises: ValueError: if `num_buckets` is less than one. ValueError: if `default_value` is not in range `[0, num_buckets)`. rz"num_buckets {} < 1, column_name {}rz5default_value {} not in range [0, {}), column_name {}rnumber_bucketsr)rbrrrIdentityCategoricalColumn)rrrs r4 categorical_column_with_identityr'sB1_ 9@@S }q'8'4 'C ?FF ; - .. $ " k HHr6zfeature_column.indicator_columnct|ttjfst dj |t |S)aRepresents multi-hot representation of given categorical column. - For DNN model, `indicator_column` can be used to wrap any `categorical_column_*` (e.g., to feed to DNN). Consider to Use `embedding_column` if the number of buckets/unique(values) are large. - For Wide (aka linear) model, `indicator_column` is the internal representation for categorical column when passing categorical column directly (as any element in feature_columns) to `linear_model`. See `linear_model` for details. ```python name = indicator_column(categorical_column_with_vocabulary_list( 'name', ['bob', 'george', 'wanda'])) columns = [name, ...] features = tf.io.parse_example(..., features=make_parse_example_spec(columns)) dense_tensor = input_layer(features, columns) dense_tensor == [[1, 0, 0]] # If "name" bytes_list is ["bob"] dense_tensor == [[1, 0, 1]] # If "name" bytes_list is ["bob", "wanda"] dense_tensor == [[2, 0, 0]] # If "name" bytes_list is ["bob", "bob"] ``` Args: categorical_column: A `CategoricalColumn` which is created by `categorical_column_with_*` or `crossed_column` functions. Returns: An `IndicatorColumn`. Raises: ValueError: If `categorical_column` is not CategoricalColumn type. zDUnsupported input type. Input must be a CategoricalColumn. Given: {})rfrrrrbrIndicatorColumnrs r4indicator_columnr+sJJ &&(A(AB D  F-. 00 + ,,r6z*feature_column.weighted_categorical_columnc||js&|jstdj|t |||S)aApplies weight values to a `CategoricalColumn`. Use this when each of your sparse inputs has both an ID and a value. For example, if you're representing text documents as a collection of word frequencies, you can provide 2 parallel sparse input features ('terms' and 'frequencies' below). Example: Input `tf.Example` objects: ```proto [ features { feature { key: "terms" value {bytes_list {value: "very" value: "model"}} } feature { key: "frequencies" value {float_list {value: 0.3 value: 0.1}} } }, features { feature { key: "terms" value {bytes_list {value: "when" value: "course" value: "human"}} } feature { key: "frequencies" value {float_list {value: 0.4 value: 0.1 value: 0.2}} } } ] ``` ```python categorical_column = categorical_column_with_hash_bucket( column_name='terms', hash_bucket_size=1000) weighted_column = weighted_categorical_column( categorical_column=categorical_column, weight_feature_key='frequencies') columns = [weighted_column, ...] features = tf.io.parse_example(..., features=make_parse_example_spec(columns)) linear_prediction, _, _ = linear_model(features, columns) ``` This assumes the input dictionary contains a `SparseTensor` for key 'terms', and a `SparseTensor` for key 'frequencies'. These 2 tensors must have the same indices and dense shape. Args: categorical_column: A `CategoricalColumn` created by `categorical_column_with_*` functions. weight_feature_key: String key for weight values. dtype: Type of weights, such as `tf.float32`. Only float and integer weights are supported. Returns: A `CategoricalColumn` composed of two sparse features: one represents id, the other represents weight (value) of the id feature in that example. Raises: ValueError: if `dtype` is not convertible to float. z%dtype {} is not convertible to float.rweight_feature_keyr0)rrrbrrr-s r4weighted_categorical_columnr/sGL mU--1B1B <CCEJ KK "++  r6zfeature_column.crossed_columnzUse `tf.keras.layers.experimental.preprocessing.HashedCrossing` instead for feature crossing when preprocessing data to train a Keras model.c|r|dkrtdj||rt|dkrtdj||D]}t|tj s:t|t tjfstdj|t|ttjfsxtdj|tt|||S)a Returns a column for performing crosses of categorical features. Crossed features will be hashed according to `hash_bucket_size`. Conceptually, the transformation can be thought of as: Hash(cartesian product of features) % `hash_bucket_size` For example, if the input features are: * SparseTensor referred by first key: ```python shape = [2, 2] { [0, 0]: "a" [1, 0]: "b" [1, 1]: "c" } ``` * SparseTensor referred by second key: ```python shape = [2, 1] { [0, 0]: "d" [1, 0]: "e" } ``` then crossed feature will look like: ```python shape = [2, 2] { [0, 0]: Hash64("d", Hash64("a")) % hash_bucket_size [1, 0]: Hash64("e", Hash64("b")) % hash_bucket_size [1, 1]: Hash64("e", Hash64("c")) % hash_bucket_size } ``` Here is an example to create a linear model with crosses of string features: ```python keywords_x_doc_terms = crossed_column(['keywords', 'doc_terms'], 50K) columns = [keywords_x_doc_terms, ...] features = tf.io.parse_example(..., features=make_parse_example_spec(columns)) linear_prediction = linear_model(features, columns) ``` You could also use vocabulary lookup before crossing: ```python keywords = categorical_column_with_vocabulary_file( 'keywords', '/path/to/vocabulary/file', vocabulary_size=1K) keywords_x_doc_terms = crossed_column([keywords, 'doc_terms'], 50K) columns = [keywords_x_doc_terms, ...] features = tf.io.parse_example(..., features=make_parse_example_spec(columns)) linear_prediction = linear_model(features, columns) ``` If an input feature is of numeric type, you can use `categorical_column_with_identity`, or `bucketized_column`, as in the example: ```python # vertical_id is an integer categorical feature. vertical_id = categorical_column_with_identity('vertical_id', 10K) price = numeric_column('price') # bucketized_column converts numerical feature to a categorical one. bucketized_price = bucketized_column(price, boundaries=[...]) vertical_id_x_price = crossed_column([vertical_id, bucketized_price], 50K) columns = [vertical_id_x_price, ...] features = tf.io.parse_example(..., features=make_parse_example_spec(columns)) linear_prediction = linear_model(features, columns) ``` To use crossed column in DNN model, you need to add it in an embedding column as in this example: ```python vertical_id_x_price = crossed_column([vertical_id, bucketized_price], 50K) vertical_id_x_price_embedded = embedding_column(vertical_id_x_price, 10) dense_tensor = input_layer(features, [vertical_id_x_price_embedded, ...]) ``` Args: keys: An iterable identifying the features to be crossed. Each element can be either: * string: Will use the corresponding feature which must be of string type. * `CategoricalColumn`: Will use the transformed tensor produced by this column. Does not support hashed categorical column. hash_bucket_size: An int > 1. The number of buckets. hash_key: Specify the hash_key that will be used by the `FingerprintCat64` function to combine the crosses fingerprints on SparseCrossOp (optional). Returns: A `CrossedColumn`. Raises: ValueError: If `len(keys) < 2`. ValueError: If any of the keys is neither a string nor `CategoricalColumn`. ValueError: If any of the keys is `HashedCategoricalColumn`. ValueError: If `hash_bucket_size < 1`. rz2hash_bucket_size must be > 1. hash_bucket_size: {}z.keys must be a list with length > 1. Given: {}zuUnsupported key type. All keys must be either string, or categorical column except HashedCategoricalColumn. Given: {}zcategorical_column_with_hash_bucket is not supported for crossing. Hashing before crossing will increase probability of collision. Instead, use the feature name as a string. Given: {}keysrhash_key) rbrrrfr string_typesrrrr_HashedCategoricalColumn CrossedColumnr)r3rr4rs r4crossed_columnr8js^ -1 ,,2F3C,D FF TQ 8??E GG  Nc sC,, - s.0I0IJ K  fSk ###*F,K,KLN  AAG NN N  ;)9H NNr6z'__internal__.feature_column.DenseColumncXeZdZdZej dZejdZy) DenseColumnzRepresents a column which can be represented as `Tensor`. Some examples of this type are: numeric_column, embedding_column, indicator_column. cy)z=`TensorShape` of `get_dense_tensor`, without batch dimension.NrKr-s r4variable_shapezDenseColumn.variable_shape r6cy)adReturns a `Tensor`. The output of this function will be used by model-builder-functions. For example the pseudo code of `input_layer` will be like: ```python def input_layer(features, feature_columns, ...): outputs = [fc.get_dense_tensor(...) for fc in feature_columns] return tf.concat(outputs) ``` Args: transformation_cache: A `FeatureTransformationCache` object to access features. state_manager: A `StateManager` to create / access resources such as lookup tables. Returns: `Tensor` of shape [batch_size] + `variable_shape`. NrKr-rrs r4get_dense_tensorzDenseColumn.get_dense_tensors, r6N) rGrHrIrJabcabstractpropertyr=abstractmethodrArKr6r4r:r:s;     r6r:cf|D],}t|tjsy|jr,yy)z+Returns True if all feature columns are V2.FT)rfrr _is_v2_column)rrs r4is_feature_column_v2rGs5'n nh&<&< =   ' '   r6c`t|trt|||||St||||S)zGCreates a weighted sum for a dense/categorical column for linear_model.rrrsparse_combiner weight_var)rrrrK)rfr'_create_categorical_column_weighted_sum!_create_dense_column_weighted_sumrIs r4_create_weighted_sumrN!sF)* 21#'   -1#  r6c|j||}|jj}tj|d}tj |||f}t j||dS)z9Create a weighted sum of a dense column for linear_model.r)r/ weighted_sumr)rAr= num_elementsrr/reshapermatmul)rrrrKr rQ batch_sizes r4rMrM3sh  " "#7 G&&&335,v&q)*   VJ +E F& . AAr6c~eZdZdZej ddZejdZ ejdZ y)rzyRepresents a categorical feature. A categorical feature typically handled with a `tf.sparse.SparseTensor` of IDs. IdWeightPair) id_tensor weight_tensorcy)1Returns number of buckets in this sparse feature.NrKr<s r4rzCategoricalColumn.num_bucketsGr>r6cy)Returns an IdWeightPair. `IdWeightPair` is a pair of `SparseTensor`s which represents ids and weights. `IdWeightPair.id_tensor` is typically a `batch_size` x `num_buckets` `SparseTensor` of `int64`. `IdWeightPair.weight_tensor` is either a `SparseTensor` of `float` or `None` to indicate all weights should be taken to be 1. If specified, `weight_tensor` must have exactly the same shape and indices as `sp_ids`. Expected `SparseTensor` is same as parsing output of a `VarLenFeature` which is a ragged matrix. Args: transformation_cache: A `FeatureTransformationCache` object to access features. state_manager: A `StateManager` to create / access resources such as lookup tables. NrKr@s r4get_sparse_tensorsz$CategoricalColumn.get_sparse_tensorsLs( r6N) rGrHrIrJrY namedtuplerVrBrCrrDr]rKr6r4rr=sU (''46,    r6rcV|j||}tj|jt j |jddg}|j }|.tj|t j |ddg}tj||||dS)aCreate a weighted sum of a categorical column for linear_model. Note to maintainer: As implementation details, the weighted sum is implemented via embedding_lookup_sparse toward efficiency. Mathematically, they are the same. To be specific, conceptually, categorical column can be treated as multi-hot vector. Say: ```python x = [0 0 1] # categorical column input w = [a b c] # weights ``` The weighted sum is `c` in this case, which is same as `w[2]`. Another example is ```python x = [0 1 1] # categorical column input w = [a b c] # weights ``` The weighted sum is `b + c` in this case, which is same as `w[2] + w[3]`. For both cases, we can implement weighted sum via embedding_lookup with sparse_combiner = "sum". rr rP)sparse_weightsrr.) r]rsparse_reshaperWrr/rXrsafe_embedding_lookup_sparse)rrrrJrKsparse_tensorsrWrXs r4rLrLcs>,,-A-:<.''~//03R8:)!..---  6q92>@M  3 3"   r6z/__internal__.feature_column.SequenceDenseColumncZeZdZdZej ddZejdZ y)SequenceDenseColumnzRepresents dense sequence data.TensorSequenceLengthPair dense_tensorsequence_lengthcy)zReturns a `TensorSequenceLengthPair`. Args: transformation_cache: A `FeatureTransformationCache` object to access features. state_manager: A `StateManager` to create / access resources such as lookup tables. NrKr@s r4get_sequence_dense_tensorz-SequenceDenseColumn.get_sequence_dense_tensors r6N) rGrHrIrJrYr^rfrBrDrkrKr6r4reres;(3[33 "EG    r6rez6__internal__.feature_column.FeatureTransformationCachec$eZdZdZdZddZdZy)r|aHandles caching of transformations while building the model. `FeatureColumn` specifies how to digest an input column to the network. Some feature columns require data transformations. This class caches those transformations. Some features may be used in more than one place. For example, one can use a bucketized feature by itself and a cross with it. In that case we should create only one bucketization op instead of creating ops for each feature column separately. To handle re-use of transformed columns, `FeatureTransformationCache` caches all previously transformed columns. Example: We're trying to use the following `FeatureColumn`s: ```python bucketized_age = fc.bucketized_column(fc.numeric_column("age"), ...) keywords = fc.categorical_column_with_hash_buckets("keywords", ...) age_X_keywords = fc.crossed_column([bucketized_age, "keywords"]) ... = linear_model(features, [bucketized_age, keywords, age_X_keywords] ``` If we transform each column independently, then we'll get duplication of bucketization (one for cross, one for bucketization itself). The `FeatureTransformationCache` eliminates this duplication. c<|j|_i|_y)aCreates a `FeatureTransformationCache`. Args: features: A mapping from feature column to objects that are `Tensor` or `SparseTensor`, or can be converted to same via `sparse_tensor.convert_to_tensor_or_sparse_tensor`. A `string` key signifies a base feature (not-transformed). A `FeatureColumn` key means that this `Tensor` is the output of an existing `FeatureColumn` which can be reused. N)copy _features_feature_tensors)r-rs r4r^z#FeatureTransformationCache.__init__s]]_DNDr6Ncv||jvr|j|S||jvr"|j|}||j|<|St|tj rt dj|t|tjstdj||}tjd| |j|||}|$t dj|j||j|<|S#t$r|j||}YTwxYw)aReturns a `Tensor` for the given key. A `str` key is used to access a base feature (not-transformed). When a `FeatureColumn` is passed, the transformed feature is returned if it already exists, otherwise the given `FeatureColumn` is asked to provide its transformed output, which is then cached. Args: key: a `str` or a `FeatureColumn`. state_manager: A StateManager object that holds the FeatureColumn state. training: Boolean indicating whether to the column is being used in training mode. This argument is passed to the transform_feature method of any `FeatureColumn` that takes a `training` argument. For example, if a `FeatureColumn` performed dropout, it could expose a `training` argument to control whether the dropout should be applied. Returns: The transformed `Tensor` corresponding to the `key`. Raises: ValueError: if key is not found or a transformed `Tensor` cannot be computed. z)Feature {} is not in features dictionary.z="key" must be either a "str" or "FeatureColumn". Provided: {}zTransforming feature_column %s.)trainingzColumn {} is not supported.)rpro_get_raw_feature_as_tensorrfrr5rbrrrrrdebugtransform_featurer.)r-rrrrfeature_tensorr transformeds r4r~zFeatureTransformationCache.gets=0 d###  " "3 '' dnn66s;n#1dC #s''( BII#N OO c811 2 %%+VC[ 22F MM3V< B,,  -2k 4;;FKKH II$/D&!  B,,T=AkBsDD87D8c <|j|}tj|djj}|/|dk(rt dj ||dk7rSStjtjtjdj |g5tjtjdtjfdfdcdddS#1swYyxYw) aGets the raw_feature (keyed by `key`) as `tensor`. The raw feature is converted to (sparse) tensor and maybe expand dim. For both `Tensor` and `SparseTensor`, the rank will be expanded (to 2) if the rank is 1. This supports dynamic rank also. For rank 0 raw feature, will error out as it is not supported. Args: key: A `str` key to access the raw feature. Returns: A `Tensor` or `SparseTensor`. Raises: ValueError: if the raw feature has rank 0. ct|tjr.tj|t j |ddgSt j|dS)Nrrr )rfsparse_tensor_lib SparseTensorrrarr/ expand_dims) input_tensors r4r|zJFeatureTransformationCache._get_raw_feature_as_tensor..expand_dims$sU L"3"@"@ A((*3//,*G*JA)NP P$$\266r6Nrz/Feature (key: {}) cannot have rank 0. Given: {}r)messagecSrRrK)r|rvsr4rSzGFeatureTransformationCache._get_raw_feature_as_tensor..=s +n-r6cSrRrK)rvsr4rSzGFeatureTransformationCache._get_raw_feature_as_tensor..=s~r6)rorz"convert_to_tensor_or_sparse_tensor get_shapendimsrbrr control_dependenciesrassert_positiverrankrrequal)r-r raw_featurerr|rvs @@r4rsz5FeatureTransformationCache._get_raw_feature_as_tensors $..%K&IIN7  # # % + +D   = D D^ %& & $qy^Ik..II ! !!! NN> *ELL^% &# G YY ..INN>: ; -/EG GGGs ADDrR)rGrHrIrJr^r~rsrKr6r4r|r|s8 6p/Gr6r|c tj|}t|tjr|St j dd||f5|S|j tjk(rd}n3|j jrd}n|j j}tj||j d}tjtj||d}tj|tj ||dtj"|tj$d  cdddS#1swYyxYw) a(Converts a `Tensor` to a `SparseTensor`, dropping ignore_value cells. If `input_tensor` is already a `SparseTensor`, just return it. Args: input_tensor: A string or integer `Tensor`. ignore_value: Entries in `dense_tensor` equal to this value will be absent from the resulting `SparseTensor`. If `None`, default value of `dense_tensor`'s dtype will be used ('' for `str`, -1 for `int`). Returns: A `SparseTensor` with the same shape as `input_tensor`. Raises: ValueError: when `input_tensor`'s rank is `None`. Nto_sparse_inputr  ignore_valuerindicesrx dense_shape)out_typer.rrxr)rzrrfr{r r{r0r stringras_numpy_dtypercastrwhere_v2 not_equal gather_ndr/int64)r}rrs r4'_to_sparse_input_and_drop_ignore_valuesrAs&"#EE, /<<=  ~~d-0F   v}} ,    ( ( $))88: ==l((~?L  <6YHG  ) )""<xHOO 6<.s qvvr6r) rfrrr$Iteratorrdictrbrrr.r)rrname_to_columns r4rzrzps !7!78&'O!9!9:?+O& K LLLf fh44 5 ..4fT&\6.J LLL  9 ::.)f {{n$ ()/v/=fkk/J)L  MM #)N6;;) %5 66r6cheZdZdZedZedZedZeeje e dZ dZ eje e dZdZed Zeeje e d Zd Zeje e dd ZedZdZeddZy )rzsee `numeric_column`.cyNTrKr<s r4rFzNumericColumn._is_v2_column r6c|jSSee `FeatureColumn` base class.rr<s r4r.zNumericColumn.name  88Or6c|jtj|j|j|j iSr)rrFixedLenFeaturer/r0rr<s r4rz NumericColumn.parse_example_specs8   ' ' DJJ(,(:(: < r6c|jSrRrr<s r4_parse_example_specz!NumericColumn._parse_example_spec  " ""r6ct|tjr$tdj |j |j |j |}tj|tjS)NzeThe corresponding Tensor of numerical column must be a Tensor. SparseTensor is not supported. key: {}) rfrzr{rbrrrrrr float32r-r}s r4_transform_input_tensorz%NumericColumn._transform_input_tensorsh, 1 > >?  3396$((3C EE %'' 5l ==v~~ 66r6cZ|j|j}|j|SrRr~rrr-inputsr}s r4_transform_featurez NumericColumn._transform_features'::dhh'L  ' ' 55r6c\|j|j|}|j|S)aSee `FeatureColumn` base class. In this case, we apply the `normalizer_fn` to the input tensor. Args: transformation_cache: A `FeatureTransformationCache` object to access features. state_manager: A `StateManager` to create / access resources such as lookup tables. Returns: Normalized input tensor. Raises: ValueError: If a SparseTensor is passed in. rr-rrr}s r4ruzNumericColumn.transform_features+ (++DHHmDL  ' ' 55r6c@tj|jSSee `DenseColumn` base class.)r TensorShaper/r<s r4r=zNumericColumn.variable_shapes  # #DJJ //r6c|jSrRr=r<s r4_variable_shapezNumericColumn._variable_shape   r6c&|j||S)aKReturns dense `Tensor` representing numeric feature. Args: transformation_cache: A `FeatureTransformationCache` object to access features. state_manager: A `StateManager` to create / access resources such as lookup tables. Returns: Dense `Tensor` created within `transform_feature`. r~r@s r4rAzNumericColumn.get_dense_tensors # #D- 88r6Nc(~~|j|SrRrr-rweight_collectionsr1s r4_get_dense_tensorzNumericColumn._get_dense_tensors  ::d r6c|jgSSee 'FeatureColumn` base class.rr<s r4parentszNumericColumn.parents  HH:r6ctt|j|}tj|j |d<|j j|d<|S)rrr0)rzip_fieldsr_serialize_keras_objectrr0r.r-rs r4 get_configzNumericColumn.get_config sK #dllD) *F+CC F?jjooF7O Mr6ct||jt|}tj|d||d<t j |d|d<|di|S)rr)custom_objectsr0rK)_check_config_keysr_standardize_and_copy_configr_deserialize_keras_objectr rclsrrcolumns_by_namekwargss r4 from_configzNumericColumn.from_config s`vs{{+ )& 1F+EE%'F?oofWo6F7O ==r6NN)rGrHrIrJpropertyrFr.rr! deprecated _FEATURE_COLUMN_DEPRECATION_DATE_FEATURE_COLUMN_DEPRECATIONrrrrur=rrArrr classmethodrrKr6r4rrs=       ;:57#7 #7;:576766& 0 0 ;:577 9 ;:577     r6rrceZdZdZedZedZedZeeje e dZ eje e dZ dZedZeeje e d Zd Zd Zeje e dd ZedZeeje e dZdZdZeje e  ddZedZdZeddZy )rzSee `bucketized_column`.czt|jtjxr|jjSrR)rfrrrrFr<s r4rFzBucketizedColumn._is_v2_column& s3 4%%x'='=> -    , ,r6cLdj|jjS)rz {}_bucketized)rrr.r<s r4r.zBucketizedColumn.name- s!  ! !$"4"4"9"9 ::r6c.|jjSr)rrr<s r4rz#BucketizedColumn.parse_example_spec2 s    0 00r6c.|jjSrR)rrr<s r4rz$BucketizedColumn._parse_example_spec7 s    1 11r6cz|j|j}tj||jSz6Returns bucketized categorical `source_column` tensor.)rr~rr _bucketizer)r-r source_tensors r4rz#BucketizedColumn._transform_feature= s5JJt112M   ?? $$r6c||j|j|}tj||jSrr)r-rrrs r4ruz"BucketizedColumn.transform_featureF s7(,,T-?-?OM   ?? $$r6ctjt|jjt |j dzfzS)rr)rrrrr/rrr<s r4r=zBucketizedColumn.variable_shapeM sC  # # d  &&'3t+?!+C*EE GGr6c|jSrRrr<s r4rz BucketizedColumn._variable_shapeS rr6ctjtj|tj t |jdzddS)Nrrr)rdepthon_value off_value)rone_hotrrr rrrrs r4"_get_dense_tensor_for_input_tensorz3BucketizedColumn._get_dense_tensor_for_input_tensorY s>    lFLL9$//"Q&  r6cH|j||}|j|S)z'Returns one hot encoded dense `Tensor`.r~rrs r4rAz!BucketizedColumn.get_dense_tensor` s%'++D-@L  2 2< @@r6NcJ~~|j|}|j|SrRrr-rrr1r}s r4rz"BucketizedColumn._get_dense_tensore s) ::d#L  2 2< @@r6cft|jdz|jjdzS)#See `CategoricalColumn` base class.rr)rrrr/r<s r4rzBucketizedColumn.num_bucketsm s.  1 $(:(:(@(@(C CCr6c|jSrRrr<s r4rzBucketizedColumn._num_bucketss    r6c ,tj|d}|jjd}tjtjtj t jd|dd|gd}tjt jd||g}tj|dt|jdz|zz}t jtjtj||ftj}t jtj||gtj}t!j"|||} t$j'| dS)Nrr)r r)rr/rrRtiler|rrrrr transposerstackr rrzr{rrV) r-r}rTsource_dimensioni1i2bucket_indicesrrr s r4$_get_sparse_tensors_for_input_tensorz5BucketizedColumn._get_sparse_tensors_for_input_tensory sM.q1J))//2     ! !(..J"? C  ! #$) +B q*:;j\ JB , !$'$81$<#B CmmO112r(;>r6cH|j||}|j|SzDConverts dense inputs to SparseTensor so downstream code can use it.r~rrs r4r]z#BucketizedColumn.get_sparse_tensors s%'++D-@L  4 4\ BBr6cJ~~|j|}|j|Srrrs r4_get_sparse_tensorsz$BucketizedColumn._get_sparse_tensors s) ::d#L  4 4\ BBr6c|jgSr)rr<s r4rzBucketizedColumn.parents s    r6czddlm}tt|j|}||j |d<|S)rrserialize_feature_columnr).tensorflow.python.feature_column.serializationr rrrrr-r rs r4rzBucketizedColumn.get_config s4W #dllD) *F6t7I7IJF? Mr6cddlm}t||jt |}||d|||d<|di|S)rrdeserialize_feature_columnrrKrrrrrrrrrrrs r4rzBucketizedColumn.from_config sHZvs{{+ )& 1F8BF? ==r6r)rGrHrIrJrrFr.rr!rrrrrrur=rrrArrrrr]r rrrrrKr6r4rr s!    ; ; 1 1 ;:5727 2;:57$7$$ G G  ;:577 A ;:57A7A  D D  ;:577 ?0C ;:57.2$(C7C    r6r)rrceZdZdZ dfd ZedZedZedZee je e dZ dZe je e dZed Zee je e d Zd Zd Zd ZdZdZe je e ddZdZe je e  ddZedZdZeddZxZS)rSee `embedding_column`.c >tt| ||||||||||  S)Nr)superr__new__) rrrrr3rrrr1r __class__s r4rzEmbeddingColumn.__new__ s< # . -+/"; / = =r6czt|jtjxr|jjSrRrfrrrrFr<s r4rFzEmbeddingColumn._is_v2_column 3 4**H,B,BC 2  # # 1 1r6cLdj|jjS)rz {}_embeddingrrr.r<s r4r.zEmbeddingColumn.name !  !8!8!=!= >>r6c.|jjSrrrr<s r4rz"EmbeddingColumn.parse_example_spec   " " 5 55r6c.|jjSrRrrr<s r4rz#EmbeddingColumn._parse_example_spec   " " 6 66r6c:|j|j|S)z+Transforms underlying `categorical_column`.r~rr@s r4ruz!EmbeddingColumn.transform_feature   # #D$;$;] KKr6c8|j|jSrRr(r-rs r4rz"EmbeddingColumn._transform_feature s ::d-- ..r6cBtj|jgSr)rrrr<s r4r=zEmbeddingColumn.variable_shape s  # #T^^$4 55r6c|jSrRrr<s r4rzEmbeddingColumn._variable_shape rr6c 2|jr|jjn|jj}t |jd|}||j f}|j |d|tj|jd|jy)z&Creates the embedding lookup variable.rembedding_weightsT)r.r/r0r1r2r3N) rFrrrgetattrrr5r rr1r3)r-rdefault_num_bucketsrembedding_shapes r4 create_statezEmbeddingColumn.create_state s    ++#'#:#:#G#G$11=-/K"DNN3O!! nn..$$"&r6c|j}|j}|jX|}t|tj r|j }tj|j|j|itj|jjd}tj}|j s||dkrtj"}|||||j$d|j&z|j(S)Nrr1 %s_weightsrr.r)rWrXrrfrrg_get_variable_listr init_from_checkpointrrdimension_valuerrrrbrembedding_lookup_sparse_v2rr.r)r-rcr/ sparse_idsr` to_restoresparse_id_rankembedding_lookup_sparses r4!_get_dense_tensor_internal_helperz1EmbeddingColumn._get_dense_tensor_internal_helper s))J#11N )$j J = = >224 ++  4#;#;Z"HJ"11((*1-/N+HH  * *~/I! - H H " DII %   r6cL|j|d}|j||S)z>Private method that follows the signature of get_dense_tensor.r/r)r=r?)r-rcrr/s r4_get_dense_tensor_internalz*EmbeddingColumn._get_dense_tensor_internal3 s8%22 &3(  1 1.2C EEr6cz|jj|jf}|rEtjj |vr)|j tjj tjd|tj|j|jxr||}|j||S)?Private method that follows the signature of _get_dense_tensor.r/)r.r/r0r3r1rY)rrrr GraphKeysGLOBAL_VARIABLESrrr=r rr3r1r?)r-rcrr1r2r/s r4_old_get_dense_tensor_internalz.EmbeddingColumn._old_get_dense_tensor_internal: s..;;T^^LO &&.@@ > >?&33 nn$$...Y& (  1 1.2C EEr6ct|jtrCtdj |j t |j|j|jj||}|j||S)aReturns tensor after doing the embedding lookup. Args: transformation_cache: A `FeatureTransformationCache` object to access features. state_manager: A `StateManager` to create / access resources such as lookup tables. Returns: Embedding lookup tensor. Raises: ValueError: `categorical_column` is SequenceCategoricalColumn. 5In embedding_column: {}. categorical_column must not be of type SequenceCategoricalColumn. Suggested fix A: If you wish to use DenseFeatures, use a non-sequence categorical_column_with_*. Suggested fix B: If you wish to create sequence input, use SequenceFeatures instead of DenseFeatures. Given (type {}): {}) rfrrrbrr.rr]rAr-rrrcs r4rAz EmbeddingColumn.get_dense_tensorK s$))+DE  !'tyy$t7N7N2O'+'>'>!@ AA,,??m-N  * *>= IIr6c<t|jttjfrCt dj |jt|j|j|jj|||}|j|||S)Na6In embedding_column: {}. categorical_column must not be of type _SequenceCategoricalColumn. Suggested fix A: If you wish to use DenseFeatures, use a non-sequence categorical_column_with_*. Suggested fix B: If you wish to create sequence input, use SequenceFeatures instead of DenseFeatures. Given (type {}): {}) rfrrrrrbrr.rr rFr-rrr1rcs r4rz!EmbeddingColumn._get_dense_tensori s  "F$E$EFH  !'tyy$t7N7N2O'+'>'>!@ AA,,@@"I/N  . .~/A9 NNr6ct|jtsCtdj |j t |j|j|jj||}|j||}tj|j}tj||S%See `SequenceDenseColumn` base class.In embedding_column: {}. categorical_column must be of type SequenceCategoricalColumn to use SequenceFeatures. Suggested fix: Use one of sequence_categorical_column_with_*. Given (type {}): {}rg)rfrrrbrr.rr]rAr"sequence_length_from_sparse_tensorrWrerf)r-rrrcrhris r4rkz)EmbeddingColumn.get_sequence_dense_tensor} s d--/H I  !'tyy$t7N7N2O'+'>'>!@  AA,,??m-N22>3@BLAA  "O  7 7!? 8 DDr6ct|jttjfsCt dj |jt|j|j|jj|}|j|||}tj|j}tj||S)NrO)rr1rg)rfrrrrrbrr.rr rFrrPrWrerf)r-rrr1rcrhris r4_get_sequence_dense_tensorz*EmbeddingColumn._get_sequence_dense_tensor s   "F$E$EF H  !'tyy$t7N7N2O'+'>'>!@  AA,,@@HN66-7LAA  "O  7 7!? 8 DDr6c|jgSrr*r<s r4rzEmbeddingColumn.parents   # # $$r6ctt|j|}tj|j |d<tj |j|d<|S)rrr3)rrrrr rrr3rs r4rzEmbeddingColumn.get_config sY #dllD) *F#0#I#I $!F  )AA F= Mr6c>d|vrd|d<t||jt|}tj|d|||d<t t jtt j}tj|d|||d<|di|S)rrTrr3)module_objectsrrK) rrrrrrr# getmembersrisclassr)rrrrrall_initializerss r4rzEmbeddingColumn.from_config s#&0,0f ()vs{{+ )& 1F#0#K#K#$no$GF  J11(Js r4rAz0SharedEmbeddingColumn._get_dense_tensor_internalA s 499 5"..AA  /n!++j%33n>>PP#33  * * ,Q /1n - J J,,1K A "/"J"J $   ==dii'== "!"""s CDDct|jtrCtdj |j t |j|j|j||S)z$Returns the embedding lookup result.rH)rfrrrbrr.rrAr@s r4rAz&SharedEmbeddingColumn.get_dense_tensor^ sf$))+DE  !'tyy$t7N7N2O'+'>'>!@ AA  * *+? OOr6ctSrRrxrs r4rz'SharedEmbeddingColumn._get_dense_tensorl r|r6ct|jtsCtdj |j t |j|j|j||}|jj||}tj|j}tj||SrM)rfrrrbrr.rrAr]rrPrWrerfr-rrrhrcris r4rkz/SharedEmbeddingColumn.get_sequence_dense_tensoro s d--/H I  !'tyy$t7N7N2O'+'>'>!@  AA223G3@BL,,??m-NAA  "O  7 7!? 8 DDr6ctSrRrxrs r4rRz0SharedEmbeddingColumn._get_sequence_dense_tensor s  0 11r6c|jgSrr*r<s r4rzSharedEmbeddingColumn.parents rTr6r[r)rGrHrIrJrrrFr.rrrurr=rrArArrkrRrr\r]s@r4rjrj s )- =   F F 6 6 2 2L2 : :  2 2": P2D*59+/2  % %r6rjrsc|Jtj|s|g}t|}|D]M}t|tst dj |||dks4tdj |||S)z4Returns shape if it's valid, raises error otherwise.z4shape dimensions must be integer. shape: {}, key: {}rz;shape dimensions must be greater than 0. shape: {}, key: {})r" is_nestedrrfintrrrb)r/rrs r4rr s      GE ,%@i i % ++16%+= ??1} ,,2F5#,> @@ @ ,r6cleZdZdZedZedZedZeeje e dZ dZ dZeje e dZed Zeeje e d Zd Zeje e  dd ZedZdZeddZy )rz*see `categorical_column_with_hash_bucket`.cyrrKr<s r4rFz%HashedCategoricalColumn._is_v2_column rr6c|jSrrr<s r4r.zHashedCategoricalColumn.name rr6cX|jtj|jiSrrr VarLenFeaturer0r<s r4rz*HashedCategoricalColumn.parse_example_spec " HHk// ; <@ zz 2 2 = ==  88>hh L$6$698 99  zzV]]""))m **<+>+>?m!<<t,,8=  ) ),*>*>*:*6*B*B DDr6cnt|j|j|}|j|S)rrr~rrrs r4ruz)HashedCategoricalColumn.transform_feature 1:  =9;L  ' ' 55r6clt|j|j}|j|SrRrrs r4rz*HashedCategoricalColumn._transform_feature ,;6::dhh;OPL  ' ' 55r6c|jSrZrr<s r4rz#HashedCategoricalColumn.num_buckets    r6c|jSrRrr<s r4rz$HashedCategoricalColumn._num_buckets rr6cNtj|j||dSrNrrVr~r@s r4r]z*HashedCategoricalColumn.get_sparse_tensors '  ) )  }5t ==r6NcP~~tj|j|dSrRrrs r4r z+HashedCategoricalColumn._get_sparse_tensors &   ) )&**T*:D AAr6c|jgSrrr<s r4rzHashedCategoricalColumn.parents rr6cvtt|j|}|jj|d<|Srr0rrrr0r.rs r4rz"HashedCategoricalColumn.get_config - #dllD) *FjjooF7O Mr6ct||jt|}tj|d|d<|di|Srr0rKrrrr rrs r4rz#HashedCategoricalColumn.from_config >vs{{+ )& 1FoofWo6F7O ==r6rrGrHrIrJrrFr.rr!rrrrrrurrrr]r rrrrrKr6r4rr sH 3     = = ;:57#7 #D46 ;:57676 ! ! ;:577 = ;:57.2$(B7B   r6rrceZdZdZ dfd ZedZedZedZee je e dZ dZdZdd Zd Ze je e d Zed Zee je e d ZdZe je e  ddZedZdZeddZxZS)rz.See `categorical_column_with_vocabulary_file`.c :tt| ||||||||S)Nrrrrr0rr )rrr) rrrrrr0rr rs r4rz'VocabularyFileCategoricalColumn.__new__ s7 0# > '''# ? !!r6cyrrKr<s r4rFz-VocabularyFileCategoricalColumn._is_v2_column* rr6c|jSrrr<s r4r.z$VocabularyFileCategoricalColumn.name. rr6cX|jtj|jiSrrr<s r4rz2VocabularyFileCategoricalColumn.parse_example_spec3 rr6c|jSrRrr<s r4rz3VocabularyFileCategoricalColumn._parse_example_spec8 rr6ctj|jd}fd}tj|j ||j |j|j|S)Nr)compression_typecZtjur|Stj|S)N)r)r rrstring_to_number)r key_dtypes r4 key_dtype_fnzYVocabularyFileCategoricalColumn._make_table_from_tfrecord_gzip_file..key_dtype_fnB s-.S#J4O4O 5##r6)r vocab_sizerrr.) rr rdata_lookup_opsindex_table_from_datasetmaprrr)r-rr.datasetrs ` r4#_make_table_from_tfrecord_gzip_filezCVocabularyFileCategoricalColumn._make_table_from_tfrecord_gzip_file> se%% v7G#  3 3 L!,,''((   r6c dj|j}||j||stj5|j dk(r|j ||}nCtj|j|j|j|j||}ddd||j||S|j||}|S#1swY4xYw)N {}_lookupr)rrrrrr.)rrrBr init_scoper rrindex_table_from_filerrrrr@rE)r-rrr.tables r4 _make_tablez+VocabularyFileCategoricalColumn._make_tableN s   dhh 'DM$>$>tT$J >>      .::9dK%22"22"22-- ..! %   """4u5 L((t4e L!  s A&CC&c2|jj|jjk7r:tdj|j|j|jt j |jdj|j|j}|jjr4tj}tj|tj}|j||j|S)*Creates a lookup table for the vocabulary.rrr) r0rrbrrrrr rrrrr)r-r}rrs r4rz7VocabularyFileCategoricalColumn._transform_input_tensorc s zz 2 2 = ==  88>hh L$6$698 99  !!-44TXX>@ I$$,,i]]<>l   I} 5 < <\ JJr6cpt|j|j|}|j||S)rrrs r4ruz1VocabularyFileCategoricalColumn.transform_featurev 3:  =9;L  ' ' m DDr6clt|j|j}|j|SrRrrs r4rz2VocabularyFileCategoricalColumn._transform_feature| rr6c4|j|jzSr)rrr<s r4rz+VocabularyFileCategoricalColumn.num_buckets s   $"6"6 66r6c|jSrRrr<s r4rz,VocabularyFileCategoricalColumn._num_buckets rr6cNtj|j||dSrrr@s r4r]z2VocabularyFileCategoricalColumn.get_sparse_tensors rr6cP~~tj|j|dSrRrrs r4r z3VocabularyFileCategoricalColumn._get_sparse_tensors rr6c|jgSrrr<s r4rz'VocabularyFileCategoricalColumn.parents rr6cvtt|j|}|jj|d<|Srrrs r4rz*VocabularyFileCategoricalColumn.get_config rr6ct||jt|}tj|d|d<|di|Srrrs r4rz+VocabularyFileCategoricalColumn.from_config rr6rRr)rGrHrIrJrrrFr.rr!rrrrrrrrurrrr]r rrrrr\r]s@r4rr s`7!$     = = ;:57#7 # *K&E ;:57676 7 7 ;:577 = ;:57.2$(B7B   r6rrcneZdZdZedZedZedZeeje e dZ ddZ dZeje e d Zed Zeeje e d Zd Zeje e  dd ZedZdZeddZy)r z.See `categorical_column_with_vocabulary_list`.cyrrKr<s r4rFz-VocabularyListCategoricalColumn._is_v2_column rr6c|jSrrr<s r4r.z$VocabularyListCategoricalColumn.name rr6cX|jtj|jiSrrr<s r4rz2VocabularyListCategoricalColumn.parse_example_spec rr6c|jSrRrr<s r4rz3VocabularyListCategoricalColumn._parse_example_spec rr6Nc|jj|jjk7r:tdj|j|j|jt j |jdj|j|j}|jjr4tj}tj|tj}dj|j}||j||sttj5tjt!|j"|j$|j&||}ddd|&|j)||n|j+||}j-|S#1swYBxYw)/Creates a lookup table for the vocabulary list.rrrrN)rrrr0r.)r0rrbrrrrr rrrrBr rrindex_table_from_tensorrrrrr@rEr)r-r}rrr.rs r4rz7VocabularyListCategoricalColumn._transform_input_tensor so zz 2 2 = ==  88>hh L$6$698 99  !!-44TXX>@ I$$,,i]]<>l   dhh 'DM$>$>tT$J >> 22!$"6"67,, 00   """4u5((t4e << %%s ;AF>>Gcpt|j|j|}|j||S)rrrs r4ruz1VocabularyListCategoricalColumn.transform_feature rr6clt|j|j}|j|SrRrrs r4rz2VocabularyListCategoricalColumn._transform_feature rr6cFt|j|jzSr)rrrr<s r4rz+VocabularyListCategoricalColumn.num_buckets s  t## $t';'; ;;r6c|jSrRrr<s r4rz,VocabularyListCategoricalColumn._num_buckets rr6cNtj|j||dSrrr@s r4r]z2VocabularyListCategoricalColumn.get_sparse_tensors rr6cP~~tj|j|dSrRrrs r4r z3VocabularyListCategoricalColumn._get_sparse_tensors rr6c|jgSrrr<s r4rz'VocabularyListCategoricalColumn.parents rr6cvtt|j|}|jj|d<|Srrrs r4rz*VocabularyListCategoricalColumn.get_config rr6ct||jt|}tj|d|d<|di|Srrrs r4rz+VocabularyListCategoricalColumn.from_config! rr6rRrrrKr6r4r r  sI7     = = ;:57#7 # &DE ;:57676 < < ;:577 = ;:57.2$(B7B   r6r rcleZdZdZedZedZedZeeje e dZ dZ dZeje e dZed Zeeje e d Zd Zeje e  dd ZedZdZeddZy )r&z'See `categorical_column_with_identity`.cyrrKr<s r4rFz'IdentityCategoricalColumn._is_v2_column2 rr6c|jSrrr<s r4r.zIdentityCategoricalColumn.name6 rr6c`|jtjtjiSr)rrrr rr<s r4rz,IdentityCategoricalColumn.parse_example_spec; s" HHk// = >>r6c|jSrRrr<s r4rz-IdentityCategoricalColumn._parse_example_spec@ rr6c |jjs/tdj|j|j|j }|j jt jk7r&tj|t jd}|jtj|j t jd}tj|jt jd}tjdt jd}tjtj||k||k\dtjtj |tj|jt jd |}t#j$|j&||j( S) ,Returns a SparseTensor with identity values.z-Invalid input, not integer. key: {} dtype: {}rxrrrzero out_of_rangedefault_values)dimsrr.r)r0rrbrrrxr rrrrrrr logical_orfillr/rzr{rr)r-r}rxrrs r4rz1IdentityCategoricalColumn._transform_input_tensorF sb    ( ( FMM ((L&&( ))  F  FLL0}}VV\\Af %}}\00&,,XNfMM   FLL}>k ]]1fll 8d!!   tmV{2 I ..??6*MM$"4"4fllC# %'- .f  ) )$$ ,, ..r6cnt|j|j|}|j|S)rrrs r4ruz+IdentityCategoricalColumn.transform_featurea rr6clt|j|j}|j|SrRrrs r4rz,IdentityCategoricalColumn._transform_featureg rr6c|jSr)r%r<s r4rz%IdentityCategoricalColumn.num_bucketsm s   r6c|jSrRrr<s r4rz&IdentityCategoricalColumn._num_bucketsr rr6cNtj|j||dSrrr@s r4r]z,IdentityCategoricalColumn.get_sparse_tensorsx rr6NcP~~tj|j|dSrRrrs r4r z-IdentityCategoricalColumn._get_sparse_tensors} rr6c|jgSrrr<s r4rz!IdentityCategoricalColumn.parents rr6c@tt|j|Sr)rrrr<s r4rz$IdentityCategoricalColumn.get_config s DLL$' ((r6cTt||jt|}|di|S)rrK)rrrrs r4rz%IdentityCategoricalColumn.from_config s(vs{{+ )& 1F ==r6rrrKr6r4r&r&* sG 0     ? ? ;:57#7 #.66 ;:57676   ;:577 = ;:57.2$(B7B  )r6r&r$cleZdZdZedZedZedZeeje e dZ edZ eeje e dZdZd Zeje e d Zd Zeje e  dd ZedZdZeddZy )rz"See `weighted_categorical_column`.czt|jtjxr|jjSrRrr<s r4rFz'WeightedCategoricalColumn._is_v2_column rr6cbdj|jj|jS)rz{}_weighted_by_{})rrr.r.r<s r4r.zWeightedCategoricalColumn.name s.  % %d&=&=&B&B&*&=&= ??r6c |jj}|j|vr2tdj ||j|jt j |j||j<|S)r&Parse config {} already exists for {}.)rrr.rbrrrr0rs r4rz,WeightedCategoricalColumn.parse_example_spec s{ $ $ 7 7F &( ?FF (( )4+B+BD EE&1&?&? &KF4 " "# Mr6c |jj}|j|vr2tdj ||j|jt j |j||j<|S)Nr)rrr.rbrrrr0rs r4rz-WeightedCategoricalColumn._parse_example_spec s{ $ $ 8 8F &( ?FF (( )4+B+BD EE&1&?&? &KF4 " "# Mr6c.|jjSrrrr<s r4rz%WeightedCategoricalColumn.num_buckets   " " . ..r6c.|jjSrRrrr<s r4rz&WeightedCategoricalColumn._num_buckets   " " / //r6c|$tdj|jtj|}|j |j j k7r/tdj|j |j t|tjs t|d}|j js$tj|tj}|S)NzMissing weights {}.z#Bad dtype, expected {}, but got {}.r)r)rbrr.rzrr0 base_dtyperfr{rrrrr r)r-rXs r4_transform_weight_tensorz2WeightedCategoricalColumn._transform_weight_tensor s ,33D4K4KL MM%HHM zz]((333 <CC **m))+ ,, m%6%C%C D= c+m    * *mmM6>>Bm r6c|j|j|}|j|}t|j|j|}||fSz?Applies weights to tensor generated from `categorical_column`'.)r~r.rrr)r-rrrXsparse_weight_tensorsparse_categorical_tensors r4ruz+WeightedCategoricalColumn.transform_feature s\(,,T-D-D-: >r6rr-cfeZdZdZedZedZedZeeje e dZ dZ eje e dZedZeeje e d Zd Zeje e  dd Zed ZdZeddZy )r7zSee `crossed_column`.ct|D]G}t|tjrt|tj sy|j rGyy)NFT)_collect_leaf_level_keysrfrr5rrrF)r-rs r4rFzCrossedColumn._is_v2_column!sM'- C)) * X33 4     r6cg}t|D]Y}t|tjtj fr|j |jI|j |[djt|S)r_X_) rrfrrr_FeatureColumnrr.rr)r- feature_namesrs r4r.zCrossedColumn.name,snM'-" C(00&2G2GH ISXX&S! " ::f]+ ,,r6cli}|jD]}t|tjr|j |j 9t|t jr|j |jo|j |tjtji|Sr) r3rfrrrrrr rrrr r)r-rrs r4rz CrossedColumn.parse_example_spec7sFyyG C// 0 c,,- c600 1 c--. sK55fmmDEF G Mr6c|jSrRrr<s r4rz!CrossedColumn._parse_example_specDrr6c$g}t|D]}t|tjr"|j |j ||?t|t jtfr^|j||}|j$tdj|j|j |jtdj|tj ||j"|j$Sz7Generates a hashed sparse cross from the input tensors.zmcrossed_column does not support weight_tensor, but the given column populates weight_tensor. Given column: {}z"Unsupported column type. Given: {})rrr4)rrfrr5rr~rrrr]rXrbrr.rWrsparse_cross_hashedrr4)r-rrfeature_tensorsrids_and_weightss r4ruzCrossedColumn.transform_featureJsO'- K C)) *377]KL cF557HI J001E1>@  ( ( 4!!'!13 3 889=DDSIJJ K  ) )))  r6c g}t|D]}t|tjr!|j |j |>t|t tjfr]|j|}|j$tdj|j|j |jtdj|tj ||j"|j$Sr%)rrfrr5rr~rrrr rXrbrr.rWrr&rr4)r-rr'rr(s r4rz CrossedColumn._transform_feature`sO'- K C)) *vzz#/ c-v/H/HI J11&9  ( ( 4!!'!13 3 889=DDSIJJ K  ) )))  r6c|jSrrr<s r4rzCrossedColumn.num_bucketswrr6c|jSrRrr<s r4rzCrossedColumn._num_buckets|rr6cNtj|j||dSrrr@s r4r]z CrossedColumn.get_sparse_tensorsrr6NcP~~tj|j|dSrrrs r4r z!CrossedColumn._get_sparse_tensorss&   ) )&**T*:D AAr6c,t|jSr)rr3r<s r4rzCrossedColumn.parentss  ?r6cddlm}tt|j|}t |j Dcgc] }|| c}|d<|Scc}w)rrr r3)rr rrrrr3)r-r rfcs r4rzCrossedColumn.get_configsHW #dllD) *F499MR4R8MNF6N MNsAc ddlm}t||jt |}t |dDcgc] }||||c}|d<|di|Scc}w)rrrr3rK)rrrrrr)rrrrrrrs r4rzCrossedColumn.from_configsdZvs{{+ )& 1F  #1noFF6N == sAr)rGrHrIrJrrFr.rr!rrrrrurrrr]r rrrrrKr6r4r7r7sB    - -     ;:57#7 # ,;:57 7 * ! ! ;:577 = ;:57.2$(B7B    r6r7r2cg}|jD]>}t|tr|jt |.|j |@|S)zCollects base keys by expanding all nested crosses. Args: cross: A `CrossedColumn`. Returns: A list of strings or `CategoricalColumn` instances. )r3rfr7extendrr)crossleaf_level_keysks r4rrsP/ :: a!]#5a89Q  r6c0tj|jd}|Ctj|t j |jt j}tj||}|tj||}||fS)z7Prune invalid IDs (< 0) from the input ids and weights.r)r0) r greater_equalrx logical_andr ones_liker boolr sparse_retain)r;r` is_id_valids r4_prune_invalid_idsr>s&&z'8'8!<+&&N11EGK'' K@*--nkJN ^ ##r6c|Ltj|jd}tj||}tj||}||fS)z;Prune invalid weights (< 0) from the input ids and weights.r)rgreaterrxrr<)r;r`is_weights_valids r4_prune_invalid_weightsrBsR''(=(=qA))*6FGJ--n>NON ^ ##r6ceZdZdZedZedZdZdZe je e dZ edZee je e dZed Zee je e d Zd Ze je e dd ZdZe je e  ddZedZdZeddZy )r)zRepresents a one-hot column for use in deep networks. Args: categorical_column: A `CategoricalColumn` which is created by `categorical_column_with_*` function. czt|jtjxr|jjSrRrr<s r4rFzIndicatorColumn._is_v2_columnrr6cLdj|jjS)rz {}_indicatorrr<s r4r.zIndicatorColumn.namer r6c|j}|j}|ytj||t |}tj |ddg|j }tj|j|j|j Stj|d}tj||dd}tj|dg S) N)sp_ids sp_valuesrrr )rrr)rrr)axis)rWrXr sparse_merger sparse_slicerr scatter_ndrrxsparse_tensor_to_denserr reduce_sum)r-id_weight_pairsizerWrXweighted_columndense_id_tensorone_hot_id_tensors r4_transform_id_weight_pairz)IndicatorColumn._transform_id_weight_pairs((I"00M "//mD Ko#//!Q0?0K0KMo ! !/"9"9"1"8"8"1"="=??!77%O "))tcSB   0t <*>r*B DDr6c.|jjSrr"r<s r4rz"IndicatorColumn.parse_example_spec$r#r6c.|jjSrRr%r<s r4rz#IndicatorColumn._parse_example_spec)r&r6ct|jtjr+t j d|jj gSt j d|jjgS)zEReturns a `TensorShape` representing the shape of the dense `Tensor`.r)rfrrrrrrrr<s r4r=zIndicatorColumn.variable_shape/s^$))8+A+AB  % %q$*A*A*M*M&N OO  % %q$*A*A*N*N&O PPr6cXtjd|jjgSr)rrrrr<s r4rzIndicatorColumn._variable_shape7s&  # #Q(?(?(L(L$M NNr6ct|jtrCtdj |j t |j|j|j||S)aReturns dense `Tensor` representing feature. Args: transformation_cache: A `FeatureTransformationCache` object to access features. state_manager: A `StateManager` to create / access resources such as lookup tables. Returns: Dense `Tensor` created within `transform_feature`. Raises: ValueError: If `categorical_column` is a `SequenceCategoricalColumn`. a5In indicator_column: {}. categorical_column must not be of type SequenceCategoricalColumn. Suggested fix A: If you wish to use DenseFeatures, use a non-sequence categorical_column_with_*. Suggested fix B: If you wish to create sequence input, use SequenceFeatures instead of DenseFeatures. Given (type {}): {})rfrrrbrr.rr~r@s r4rAz IndicatorColumn.get_dense_tensor=sg$))+DE  !'tyy$t7N7N2O'+'>'>!@ AA # #D- 88r6Nc~~t|jttjfrCt dj |jt|j|j|j|S)Na6In indicator_column: {}. categorical_column must not be of type _SequenceCategoricalColumn. Suggested fix A: If you wish to use DenseFeatures, use a non-sequence categorical_column_with_*. Suggested fix B: If you wish to create sequence input, use SequenceFeatures instead of DenseFeatures. Given (type {}): {}) rfrrrrrbrr.rr~rs r4rz!IndicatorColumn._get_dense_tensorZsz   "F$E$EFH  !'tyy$t7N7N2O'+'>'>!@ AA ::d r6ct|jtsCtdj |j t |j|j|j||}|jj||}tj|j}tj||S)rNzIn indicator_column: {}. categorical_column must be of type SequenceCategoricalColumn to use SequenceFeatures. Suggested fix: Use one of sequence_categorical_column_with_*. Given (type {}): {}rg)rfrrrbrr.rr~r]rrPrWrerfrs r4rkz)IndicatorColumn.get_sequence_dense_tensoros d--/H I  !'tyy$t7N7N2O'+'>'>!@  AA(++D-@L,,??m-NAA  "O  7 7!? 8 DDr6c~~t|jttjfsCt dj |jt|j|j|j|}|jj|}tj|j}tj||S)NzIn indicator_column: {}. categorical_column must be of type _SequenceCategoricalColumn to use SequenceFeatures. Suggested fix: Use one of sequence_categorical_column_with_*. Given (type {}): {}rg)rfrrrrrbrr.rr~r rrPrWrerf)r-rrr1rhrcris r4rRz*IndicatorColumn._get_sequence_dense_tensors    "F$E$EF H  !'tyy$t7N7N2O'+'>'>!@  AA::d#L,,@@HNAA  "O  7 7!? 8 DDr6c|jgSrr*r<s r4rzIndicatorColumn.parentsrTr6czddlm}tt|j|}||j |d<|Srrr rrr rrrrrs r4rzIndicatorColumn.get_config8W #dllD) *F#; $!F  Mr6cddlm}t||jt |}||d|||d<|di|SrrrrrKrrs r4rzIndicatorColumn.from_configJZvs{{+ )& 1F#=#$no$GF  ==r6r)rGrHrIrJrrFr.rUrur!rrrrrrr=rrArrkrRrrrrrKr6r4r)r)s}    ? ?=8C(;:57D7D  6 6 ;:5777 7 Q Q ;:57O7 O9:;:577&D(;:5759+/D7D6 % %r6r)rc dd}tdt|D]}tjtj||j d}|j H||}|}O|j|ratdj|j||j||y)zVerify equality between static batch sizes. Args: tensors: iterable of input tensors. columns: Corresponding feature columns. Raises: ValueError: in case of mismatched batch sizes. NrzcBatch size (first dimension) of each feature must be same. Batch size of columns ({}, {}): ({}, {})) rrr Dimensionr9r/ris_compatible_withrbrr.)rcolumnsexpected_batch_sizerrTbath_size_column_indexs r4"_verify_static_batch_size_equalityrns CL ! 2a''$$WQZ%5%5a%89;J#  $!"("55jA 77=v./44gajoo#Z812 2 2r6cleZdZdZedZedZedZeeje e dZ dZ eje e dZedZeeje e d Zd Zd Zeje e  dd ZedZdZeddZy )rz)Represents sequences of categorical data.czt|jtjxr|jjSrRrr<s r4rFz'SequenceCategoricalColumn._is_v2_columnrr6c.|jjSr)rr.r<s r4r.zSequenceCategoricalColumn.names  " " ' ''r6c.|jjSrr"r<s r4rz,SequenceCategoricalColumn.parse_example_specr#r6c.|jjSrRr%r<s r4rz-SequenceCategoricalColumn._parse_example_specr&r6c:|jj||Sr)rrur@s r4ruz+SequenceCategoricalColumn.transform_features#  " " 4 45I5B DDr6c8|jj|SrR)rrr+s r4rz,SequenceCategoricalColumn._transform_features  " " 5 5f ==r6c.|jjSrrr<s r4rz%SequenceCategoricalColumn.num_bucketsrr6c.|jjSrRr r<s r4rz&SequenceCategoricalColumn._num_bucketsr r6c&|j}|j}tj|}|d|dt j |ddg}t j||}|t j||}tj||S)Nrrr1) rWrXrr/r reduce_prodrrarrV)r-rcrWrXr/ target_shapes r4_get_sparse_tensors_helperz4SequenceCategoricalColumn._get_sparse_tensors_helpers((I"00M OOI &E!HeAh(<( ::r6NcZ|jj|}|j|SrR)rr r{rKs r4r z-SequenceCategoricalColumn._get_sparse_tensors.s+ ,,@@HN  * *> ::r6c|jgSrr*r<s r4rz!SequenceCategoricalColumn.parents7rTr6czddlm}tt|j|}||j |d<|Srbrcrs r4rz$SequenceCategoricalColumn.get_config<rdr6cddlm}t||jt |}||d|||d<|di|Srfrrs r4rz%SequenceCategoricalColumn.from_configDrgr6r)rGrHrIrJrrFr.rr!rrrrrurrrr{r]r rrrrrKr6r4rrsG 2    ( ( 6 6 ;:5777 7D ;:57>7> / / ;:5707 0D ;.;:57.2$(;7; % %r6rct|jt|k7rtdj||y)z+Checks that a config has all expected_keys.z%Invalid config: {}, expected keys: {}N)rr3rbr)r expected_keyss r4rrOs?3}-- <CC   .r6c|j}|jD]$\}}t|tst |||<&|S)aReturns a shallow copy of config with lists turned to tuples. Keras serialization uses nest to listify everything. This causes problems with the NumericColumn shape, which becomes unhashable. We could try to solve this on the Keras side, but that would require lots of tracking to avoid changing existing behavior. Instead, we ensure here that we revive correctly. Args: config: dict that will be used to revive a Feature Column Returns: Shallow copy of config with lists turned to tuples. )rnitemsrfrr)rrr6rms r4rrVsF ;;=&llnda!T(fQi -r6cPtjd}|jd|S)zASanitizes user-provided feature names for use as variable scopes.z[^A-Za-z0-9_.\-]r)recompilesub)r. invalid_chars r4r}r}ms$/0,   #t $$r6)rNNNNTT)rNNNNNTT)Nr rrR)rJrBrYrrrrr'tensorflow.python.data.experimental.opsrrtensorflow.python.data.opsrtensorflow.python.eagerr tensorflow.python.feature_columnrrrrrr rtensorflow.python.frameworkr r r rzr rrtensorflow.python.opsrrrrrrrrrrrrtensorflow.python.platformrrrtensorflow.python.trackablerrrcrtensorflow.python.trainingr tensorflow.python.utilr!r"r#tensorflow.python.util.compatr$ tensorflow.python.util.tf_exportr%tensorflow.tools.docsr&rr#_FEATURE_COLUMN_DEPRECATION_WARNING+_FEATURE_COLUMN_DEPRECATION_RUNTIME_WARNINGobjectr(rNrheaderrrrrrrrrrrrrr"r'r+r/r8rr:rGrNrMrrLregister_feature_columnrer|rrz _DenseColumnr^rrr_SequenceDenseColumnrr_ AutoTrackablerrjrrrr r&rr7rr>rBr)rnrrrrr}rKr6r4rsR     Q.+EP:>.+J<4+1+&/*,*-,,0+,<5977.'-96.#'  /'##,];6];@ 5"=CF CF>CFL,^89 ,  IJ9K : 9x89 ,-IJ%!%'+)-"#/3M;K.:M;`89 89:IJ'-)->B/315&*'+7;WK;:Wt89 &  IJ*0,0AE2648)-*.:>FK : FR89 *+IJ!%!% ^#K,:^#B89 -.IJQ<K/:Q28-- fEKJ:fER89 <  IJ@D5;]]=A?@;? BK : BJ89 CDIJ37:<<= q'KE:q'h89 <=IJHHK>:HHV89 ,-IJ'-K.:'-T89 78IJ'-nnHK9:HV89 *+ |N ,:|N@ 4<" (((" =" J$B# ..# L.d <D&& (00 'E & CKRGRGLRGl,F^*7Z&&{ KCE{'{|&&Y  K-:< Y'Yx&&L  K &' L'L^@ 2=#>#>2j&&F%  K 23 F%'F%R  &&l K4?Al'l^&&^ K) 23^'^B&&v K)OQv'vr&&j K6EGj'jZ&&} K#=?}'}@&&N K?CEN'Nb$ $$&&a  K,/CE a'aH26&&x %%K603x'xv .%r6