2Vh6ddlZddlmZddlmZddlmZddlm Z ddlm Z ddl m Z edGd d eZd Zd Zd ZdZy)N)backend) keras_export) TFDataLayer)argument_validation)numerical_utils) tensorflowzkeras.layers.DiscretizationceZdZdZ d fd ZedZddZdZdZ dZ dZ d Z d Z d Zedd ZxZS)Discretizationaj A preprocessing layer which buckets continuous features by ranges. This layer will place each element of its input data into one of several contiguous ranges and output an integer index indicating which range each element was placed in. **Note:** This layer is safe to use inside a `tf.data` pipeline (independently of which backend you're using). Input shape: Any array of dimension 2 or higher. Output shape: Same as input shape. Arguments: bin_boundaries: A list of bin boundaries. The leftmost and rightmost bins will always extend to `-inf` and `inf`, so `bin_boundaries=[0., 1., 2.]` generates bins `(-inf, 0.)`, `[0., 1.)`, `[1., 2.)`, and `[2., +inf)`. If this option is set, `adapt()` should not be called. num_bins: The integer number of bins to compute. If this option is set, `bin_boundaries` should not be set and `adapt()` should be called to learn the bin boundaries. epsilon: Error tolerance, typically a small fraction close to zero (e.g. 0.01). Higher values of epsilon increase the quantile approximation, and hence result in more unequal buckets, but could improve performance and resource consumption. output_mode: Specification for the output of the layer. Values can be `"int"`, `"one_hot"`, `"multi_hot"`, or `"count"` configuring the layer as follows: - `"int"`: Return the discretized bin indices directly. - `"one_hot"`: Encodes each individual element in the input into an array the same size as `num_bins`, containing a 1 at the input's bin index. If the last dimension is size 1, will encode on that dimension. If the last dimension is not size 1, will append a new dimension for the encoded output. - `"multi_hot"`: Encodes each sample in the input into a single array the same size as `num_bins`, containing a 1 for each bin index index present in the sample. Treats the last dimension as the sample dimension, if input shape is `(..., sample_length)`, output shape will be `(..., num_tokens)`. - `"count"`: As `"multi_hot"`, but the int array contains a count of the number of times the bin index appeared in the sample. Defaults to `"int"`. sparse: Boolean. Only applicable to `"one_hot"`, `"multi_hot"`, and `"count"` output modes. Only supported with TensorFlow backend. If `True`, returns a `SparseTensor` instead of a dense `Tensor`. Defaults to `False`. Examples: Discretize float values based on provided buckets. >>> input = np.array([[-1.5, 1.0, 3.4, .5], [0.0, 3.0, 1.3, 0.0]]) >>> layer = Discretization(bin_boundaries=[0., 1., 2.]) >>> layer(input) array([[0, 2, 3, 1], [1, 3, 2, 1]]) Discretize float values based on a number of buckets to compute. >>> input = np.array([[-1.5, 1.0, 3.4, .5], [0.0, 3.0, 1.3, 0.0]]) >>> layer = Discretization(num_bins=4, epsilon=0.01) >>> layer.adapt(input) >>> layer(input) array([[0, 2, 3, 2], [1, 3, 3, 1]]) cp||dk(rdntj}t| |||r0tjs t dtj|r|dk(rt d|d|t j|d|jjd ||d krt d |d ||t d |d|d | | t d||_ ||_ ||_ ||_ ||_|jrd|_yt!j"gggd|_y)Nintint64)namedtypez*`sparse=True` cannot be used with backend zn`sparse=True` may only be used if `output_mode` is `'one_hot'`, `'multi_hot'`, or `'count'`. Received: sparse=z and output_mode=)r one_hot multi_hotcount output_mode)allowable_strings caller_namearg_namerzC`num_bins` must be greater than or equal to 0. Received: `num_bins=`zLBoth `num_bins` and `bin_boundaries` should not be set. Received: `num_bins=z` and `bin_boundaries=z6You need to set either `num_bins` or `bin_boundaries`.float32r)rfloatxsuper__init__SUPPORTS_SPARSE_TENSORS ValueErrorrvalidate_string_arg __class____name__bin_boundariesnum_binsepsilonrsparsesummarynparray) selfr"r#r$rr%rrr s ]/home/dcms/DCMS/lib/python3.12/site-packages/keras/src/layers/preprocessing/discretization.pyrzDiscretization.__init__Xs =*e3G9IE d%0 '99''/j1##1"2!5    6H -   &   DL88RHI>DLc*tjSN)rrr)s r* input_dtypezDiscretization.input_dtypes~~r+c4|j td|jt|tj j r,||j|}|D]}|j|n|j||jy)aComputes bin boundaries from quantiles in a input dataset. Calling `adapt()` on a `Discretization` layer is an alternative to passing in a `bin_boundaries` argument during construction. A `Discretization` layer should always be either adapted over a dataset or passed `bin_boundaries`. During `adapt()`, the layer will estimate the quantile boundaries of the input dataset. The number of quantiles can be controlled via the `num_bins` argument, and the error tolerance for quantile boundaries can be controlled via the `epsilon` argument. Arguments: data: The data to train on. It can be passed either as a batched `tf.data.Dataset`, or as a NumPy array. steps: Integer or `None`. Total number of steps (batches of samples) to process. If `data` is a `tf.data.Dataset`, and `steps` is `None`, `adapt()` will run until the input dataset is exhausted. When passing an infinitely repeating dataset, you must specify the `steps` argument. This argument is not supported with array inputs or list inputs. NzlCannot adapt a Discretization layer that has been initialized with `bin_boundaries`, use `num_bins` instead.) r#r reset_state isinstancetfdataDatasettake update_statefinalize_state)r)r4stepsbatchs r*adaptzDiscretization.adapts2 == A   dBGGOO , yy' )!!%( )   d # r+ctj|jd}t||j}t ||j |j|_y)Nr)r'r(astype summarizer$merge_summariesr&)r)r4r&s r*r7zDiscretization.update_statesBxx~$$Y/D$,,/&w dllK r+c|jyt|j|jj|_yr-)r#get_bin_boundariesr&tolistr"r.s r*r8zDiscretization.finalize_states3 == 0 LL$-- &( r+cZ|jytjgggd|_y)Nrr)r#r'r(r&r.s r*r1zDiscretization.reset_states% == xxR : r+cXtj|j|jS)N)shaper)r KerasTensorrE compute_dtype)r)inputss r*compute_output_specz"Discretization.compute_output_specs""T=O=OPPr+c4t|dk(r |d|_y)N0)lenr&)r)stores r*load_own_variablesz!Discretization.load_own_variabless u:? :DLr+cB|j td|jjj ||j}t j ||jt|jdz|j|j|jS)NzYou need to either pass the `bin_boundaries` argument at construction time or call `adapt(dataset)` before you can start using the `Discretization` layer.rK)rdepthrr%backend_module) r"rrnumpydigitizerencode_categorical_inputsrrMrGr%)r)rHindicess r*callzDiscretization.calls    &:  ,,$$--fd6I6IJ88 ((d))*Q.$$;;<<   r+c|j|j|j|j|j|j |j dS)Nr"r#r$rr%rrrYr.s r* get_configzDiscretization.get_configsA"11 ||++kkIIZZ  r+c|jddD|jdd2|j}|jd}|di|}||_|S|di|S)Nr"r#)getcopypopr")clsconfigcustom_objectsr"discretizations r* from_configzDiscretization.from_configsh JJ' . : :t,8 [[]F#ZZ(89N ]6]N,:N )! !}V}r+)NNg{Gz?r FNNr-)r! __module__ __qualname____doc__rpropertyr/r;r7r8r1rIrOrWrZ classmethodrd __classcell__)r s@r*r r s{IZ @?D  &PL ; Q  $  r+r cJtj|dg}tj|}tj|}d|z }||z }|}t |d}|t |dt |}tj |}||z}tj||gS)a[Reduce a 1D sequence of values to a summary. This algorithm is based on numpy.quantiles but modified to allow for intermediate steps between multiple data sets. It first finds the target number of bins as the reciprocal of epsilon and then takes the individual values spaced at appropriate intervals to arrive at that target. The final step is to return the corresponding counts between those values If the target num_bins is larger than the size of values, the whole array is returned (with weights of 1). Args: values: 1D `np.ndarray` to be summarized. epsilon: A `'float32'` that determines the approximate desired precision. Returns: A 2D `np.ndarray` that is a summary of the inputs. First column is the interpolated partition values, the second is the weights (counts). ?rKN)r'reshapesortsizemaxr ones_likestack) valuesr$elements num_buckets incrementstartstep boundariesweightss r*r>r>s(ZZ %F WWV_FwwvH-K;&I E y! DE /c$i/0Jll:&GnG 88Z) **r+ctj||fd}tj|tj|dd}t ||S)aCWeighted merge sort of summaries. Given two summaries of distinct data, this function merges (and compresses) them to stay within `epsilon` error tolerance. Args: prev_summary: 2D `np.ndarray` summary to be merged with `next_summary`. next_summary: 2D `np.ndarray` summary to be merged with `prev_summary`. epsilon: A float that determines the approximate desired precision. Returns: A 2-D `np.ndarray` that is a merged summary. First column is the interpolated partition values, the second is the weights (counts). rK)axisr)r' concatenater6argsortcompress_summary) prev_summary next_summaryr$mergeds r*r?r?0sE^^\<8q AF WWVRZZq 2 ;F FG ,,r+c.t|d|z dddfS)Nrmrrl)r)r&r#s r*rArADs GS8^ 4QV <>) $$r+)rSr' keras.srcrkeras.src.api_exportr,keras.src.layers.preprocessing.tf_data_layerrkeras.src.utilsrrkeras.src.utils.module_utilsrr3r r>r?rArr\r+r*rsT-D/+9+,@[@-@F+B-(=%r+