2Vh+ddlmZddlmZddlmZddlmZddlmZddlm Z ddl m Z edGd d eZ y ) )backend) keras_export)Layer) backend_utils)numerical_utils)tf_utils tensorflowzkeras.layers.HashingcHeZdZdZ dfd ZdZdZdZfdZxZ S)HashingaBA preprocessing layer which hashes and bins categorical features. This layer transforms categorical inputs to hashed output. It element-wise converts a ints or strings to ints in a fixed range. The stable hash function uses `tensorflow::ops::Fingerprint` to produce the same output consistently across all platforms. This layer uses [FarmHash64](https://github.com/google/farmhash) by default, which provides a consistent hashed output across different platforms and is stable across invocations, regardless of device and context, by mixing the input bits thoroughly. If you want to obfuscate the hashed output, you can also pass a random `salt` argument in the constructor. In that case, the layer will use the [SipHash64](https://github.com/google/highwayhash) hash function, with the `salt` value serving as additional input to the hash function. **Note:** This layer internally uses TensorFlow. It cannot be used as part of the compiled computation graph of a model with any backend other than TensorFlow. It can however be used with any backend when running eagerly. It can also always be used as part of an input preprocessing pipeline with any backend (outside the model itself), which is how we recommend to use this layer. **Note:** This layer is safe to use inside a `tf.data` pipeline (independently of which backend you're using). **Example (FarmHash64)** >>> layer = keras.layers.Hashing(num_bins=3) >>> inp = [['A'], ['B'], ['C'], ['D'], ['E']] >>> layer(inp) array([[1], [0], [1], [1], [2]])> **Example (FarmHash64) with a mask value** >>> layer = keras.layers.Hashing(num_bins=3, mask_value='') >>> inp = [['A'], ['B'], [''], ['C'], ['D']] >>> layer(inp) array([[1], [1], [0], [2], [2]]) **Example (SipHash64)** >>> layer = keras.layers.Hashing(num_bins=3, salt=[133, 137]) >>> inp = [['A'], ['B'], ['C'], ['D'], ['E']] >>> layer(inp) array([[1], [2], [1], [0], [2]]) **Example (Siphash64 with a single integer, same as `salt=[133, 133]`)** >>> layer = keras.layers.Hashing(num_bins=3, salt=133) >>> inp = [['A'], ['B'], ['C'], ['D'], ['E']] >>> layer(inp) array([[0], [0], [2], [1], [0]]) Args: num_bins: Number of hash bins. Note that this includes the `mask_value` bin, so the effective number of bins is `(num_bins - 1)` if `mask_value` is set. mask_value: A value that represents masked inputs, which are mapped to index 0. `None` means no mask term will be added and the hashing will start at index 0. Defaults to `None`. salt: A single unsigned integer or None. If passed, the hash function used will be SipHash64, with these values used as an additional input (known as a "salt" in cryptography). These should be non-zero. If `None`, uses the FarmHash64 hash function. It also supports tuple/list of 2 unsigned integer numbers, see reference paper for details. Defaults to `None`. output_mode: Specification for the output of the layer. Values can be `"int"`, `"one_hot"`, `"multi_hot"`, or `"count"` configuring the layer as follows: - `"int"`: Return the integer bin indices directly. - `"one_hot"`: Encodes each individual element in the input into an array the same size as `num_bins`, containing a 1 at the input's bin index. If the last dimension is size 1, will encode on that dimension. If the last dimension is not size 1, will append a new dimension for the encoded output. - `"multi_hot"`: Encodes each sample in the input into a single array the same size as `num_bins`, containing a 1 for each bin index index present in the sample. Treats the last dimension as the sample dimension, if input shape is `(..., sample_length)`, output shape will be `(..., num_tokens)`. - `"count"`: As `"multi_hot"`, but the int array contains a count of the number of times the bin index appeared in the sample. Defaults to `"int"`. sparse: Boolean. Only applicable to `"one_hot"`, `"multi_hot"`, and `"count"` output modes. Only supported with TensorFlow backend. If `True`, returns a `SparseTensor` instead of a dense `Tensor`. Defaults to `False`. **kwargs: Keyword arguments to construct a layer. Input shape: A single string, a list of strings, or an `int32` or `int64` tensor of shape `(batch_size, ...,)`. Output shape: An `int32` tensor of shape `(batch_size, ...)`. Reference: - [SipHash with salt](https://www.131002.net/siphash/siphash.pdf) c tjs tdd|vs|d|dk(rdntj|d<t |di|||dkrtd|d|dk(r)|jjdvrtd |dd }||vrtd |d ||r|dk(rtd |d|||_ ||_ |dnd|_ ||_ ||_d|_|^t!|t"t$frt'|dk(rt%||_n)t!|t(r ||g|_ntd|dd|_d|_d|_y)NzKLayer Hashing requires TensorFlow. Install it via `pip install tensorflow`.dtypeintint64rzYThe `num_bins` for `Hashing` cannot be `None` or non-positive values. Received: num_bins=.)int32rz`When `output_mode="int"`, `dtype` should be an integer type, 'int32' or 'in64'. Received: dtype=)rone_hot multi_hotcountz:Invalid value for argument `output_mode`. Expected one of z. Received: output_mode=zi`sparse` may only be true if `output_mode` is `"one_hot"`, `"multi_hot"`, or `"count"`. Received: sparse=z and output_mode=TFznThe `salt` argument for `Hashing` can only be a tuple of size 2 integers, or a single integer. Received: salt=)tf available ImportErrorrfloatxsuper__init__ ValueError dtype_policynamenum_bins mask_value strong_hash output_modesparsesalt isinstancetuplelistlenr_convert_input_args!_allow_non_tensor_positional_args supports_jit) selfr!r"r&r$r%kwargsaccepted_output_modes __class__s V/home/dcms/DCMS/lib/python3.12/site-packages/keras/src/layers/preprocessing/hashing.pyrzHashing.__init__s||;  & F7O$;&%/W^^5E 7O "6"  x1};;C*AG  %     " "*< << J D#&!4L  &&*V1. $) 15.!cFddlm}tj|}|jdk(r/|j ddk(r|j j|d}t|tjrFtj|j|j|j|j}n|j|}tj ||j|j"|j$|j&|}t)j*|S) Nrr r)axis)indicesvalues dense_shape)r$depthr%rbackend_module)keras.src.backendr r ensure_tensorr$shapenumpysqueezer'r SparseTensorr8_hash_values_to_binsr9r:rencode_categorical_inputsr!r%rrconvert_tf_tensor)r.inputs tf_backendr8outputss r2callz Hashing.calls>''/   y (V\\"-=-B%%--f2->F fboo .oo00?"..G //7G!;; ((--;;**%  ..w77r3c|j}d}|j*|dkDr%|dz}tj||j}|jj rtj |d}|jtjk7rtj|}|jr.tjj||d|j}n"tjj||d}|Stj|tj|}tj |tj"||}|S)z6Converts a non-sparse tensor of values to bin indices.Nr6r)rhash)r key)r )r!r"requalr is_floatingcaststring as_stringr#stringsto_hash_bucket_strongr&to_hash_bucket_fastadd ones_likewhere zeros_like)r.r9 hash_binsmasks r2rCzHashing._hash_values_to_binss MM  ?? &9q= NI88FDOO4D << # #WWV73F <<299 $\\&)F   ZZ55 DII6FZZ33 4F  VVFBLL$89FXXdBMM&$96BF r3c:|jdk(r+tj|j|jSt |jdk\rt |jdd}nd}tj||jfz|jS)Nr)r?rr6r5r)r$r KerasTensorr?rr*r(r!)r.rF base_shapes r2compute_output_speczHashing.compute_output_spec s~   u $&&V\\L L v||  !v||,Sb1JJ"" //tzz  r3ct|}|j|j|j|j |j |jd|S)N)r!r&r"r$r%)r get_configupdater!r&r"r$r%)r.configr1s r2r`zHashing.get_configsM#%  MM "oo#//++    r3)NNrF) __name__ __module__ __qualname____doc__rrIrCr^r` __classcell__)r1s@r2r r s8{@  I"V828    r3r N) keras.srcrkeras.src.api_exportrkeras.src.layers.layerrkeras.src.utilsrrrkeras.src.utils.module_utilsr rr rr3r2rms>-()+$9$%TeT&Tr3