2VhCddlZddlZddlZddlmZddlmZddlmZddl m Z  ddl Z ejfZe ree j e j"fzZGddZGdd eZGd d eZGd d eZGddeZGddeZGddeZGddeZGddeZGddeZGddeZej:dgdZdZdZ d Z!d#d!Z"d"Z#y#e $rdZ YwxYw)$N)backend)tree)data_adapter_utils tensorflowcleZdZdZdZdZedZedZedZ edZ edZ y ) Sliceableaa`Sliceable` wrapping a tensor. A `Sliceable` implements the subscript operator to slice or index against the first dimension of the array. It also has conversion methods for each one of the backends. Args: array: the native array or tensor to wrap. Attributes: shape: the shape of the full dense native array. c||_yNarray)selfr s ^/home/dcms/DCMS/lib/python3.12/site-packages/keras/src/trainers/data_adapters/array_slicing.py__init__zSliceable.__init__'s  c |j|S)a/Select elements in the 0th dimension. Args: indices: the indices to select. Only needs to support one dimension, the 0th dimension. Should support a `slice` or a list, tuple, `np.array` or 1D tensor. Returns: A slice of `self.array`. r rindicess r __getitem__zSliceable.__getitem__*szz'""rc$|j|S)zCast a tensor to a different dtype. Only called on a full array as provided by the user. Args: x: the tensor to cast. Returns: the cast tensor. )astype)clsxdtypes rcastzSliceable.cast5sxxrc|S)zConvert a tensor to a NumPy array. Only called after slicing using `__getitem__`. Args: x: the tensor to convert. Returns: the converted tensor. rrs rconvert_to_numpyzSliceable.convert_to_numpyAs rc|S)akConvert a tensor to something compatible with `tf.data.Dataset`. This can be a NumPy array, `tf.Tensor` or any other type of tensor that `tf.data.Dataset.from_tensors` can consume. Only called on a full array as provided by the user. Args: x: the tensor to convert. Returns: converted version tensor. rrs r convert_to_tf_dataset_compatiblez*Sliceable.convert_to_tf_dataset_compatibleM rc|S)a`Convert a tensor to something that the JAX backend can consume. This can be a `JAX` array, `JAXSparse` or a NumPy array. Only called after slicing using `__getitem__`. Used to convert sparse tensors and densify ragged tensors. Args: x: the tensor to convert. Returns: the converted tensor. rrs rconvert_to_jax_compatiblez#Sliceable.convert_to_jax_compatible[r"rc|S)aConvert a tensor to something that the Torch backend can consume. This can be a Torch tensor, NumPy array or any other type of tensor that `keras.backend.torch.core.convert_to_tensor()` can consume. Only called after slicing using `__getitem__`. Used to densify sparse tensors and ragged tensors. Args: x: the tensor to convert. Returns: the converted tensor. rrs rconvert_to_torch_compatiblez%Sliceable.convert_to_torch_compatibleis rN) __name__ __module__ __qualname____doc__rr classmethodrrr!r$r&rrrr r su  #          rr c eZdZy)NumpySliceableNr'r(r)rrrr-r-yrr-c2eZdZdZedZedZy)TensorflowSliceablecddlm}t|tr|j|S|j |j|dS)Nrraxis)keras.src.utils.module_utilsr isinstanceslicer gather)rrtfs rrzTensorflowSliceable.__getitem__~s7A gu %::g& &299TZZq9 9rc ddlm}|||SNr)r)!keras.src.backend.tensorflow.corerrrrrs rrzTensorflowSliceable.casts:Au~rcddlm}||SNrr)r<rrrrs rrz$TensorflowSliceable.convert_to_numpysF""rN)r'r(r)rr+rrrrrr1r1}s/: ##rr1c,eZdZedZedZy)TensorflowRaggedSliceablec$|j|Sr r@rs rr$z3TensorflowRaggedSliceable.convert_to_jax_compatible##A&&rc"|jSr ) to_tensorrs rr&z5TensorflowRaggedSliceable.convert_to_torch_compatibles{{}rN)r'r(r)r+r$r&rrrrCrCs(''rrCcdeZdZfdZedZdZedZedZ edZ xZ S)TensorflowSparseSliceablec6t|t|yr )superrto_tensorflow_sparse_wrapperrr __class__s rrz"TensorflowSparseSliceable.__init__s 5e<=rcB|jjjSr )r sparseshape)rs rrQzTensorflowSparseSliceable.shapeszz  &&&rc.t|j|Sr )slice_tensorflow_sparse_wrapperr rs rrz%TensorflowSparseSliceable.__getitem__s.tzz7CCrct|Sr )rLrs rr!z:TensorflowSparseSliceable.convert_to_tf_dataset_compatibles +A..rc,tj|Sr )rtf_sparse_to_jax_sparsers rr$z3TensorflowSparseSliceable.convert_to_jax_compatibles!99!<''D//==,,rrIcBeZdZdZedZedZedZy)JaxSparseSliceablec$|j|dfS)N.r rs rrzJaxSparseSliceable.__getitem__szz'3,''rcddlm}||Sr?)keras.src.backend.jax.corerrAs rrz#JaxSparseSliceable.convert_to_numpys?""rc>ttj|Sr )rLrjax_sparse_to_tf_sparse)rr s rr!z3JaxSparseSliceable.convert_to_tf_dataset_compatibles+  6 6u =  rc"|jSr todensers rr&z.JaxSparseSliceable.convert_to_torch_compatibleyy{rN)r'r(r)rr+rr!r&rrrr_r_sC(##   rr_c,eZdZedZedZy)TorchSliceablec ddlm}|||Sr;)keras.src.backend.torch.corerr=s rrzTorchSliceable.casts5Au~rcddlm}||Sr?)rlrrAs rrzTorchSliceable.convert_to_numpysA""rN)r'r(r)r+rrrrrrjrjs( ##rrjcReZdZdZedZedZedZedZy)PandasSliceablec4|jj|Sr )r ilocrs rrzPandasSliceable.__getitem__szzw''rc"|jSr )to_numpyrs rrz PandasSliceable.convert_to_numpyszz|rc$|j|Sr r@rs rr!z0PandasSliceable.convert_to_tf_dataset_compatiblerErc$|j|Sr r@rs rr$z)PandasSliceable.convert_to_jax_compatiblerErc$|j|Sr r@rs rr&z+PandasSliceable.convert_to_torch_compatiblerErN) r'r(r)rr+rr!r$r&rrrrorosW(''''''rroc eZdZy)PandasDataFrameSliceableNr.rrrrxrxr/rrxceZdZedZy)PandasSeriesSliceablecLtj|jdS)Nr3)np expand_dimsrsrs rrz&PandasSeriesSliceable.convert_to_numpys~~ajjl44rN)r'r(r)r+rrrrrzrzs55rrzc^eZdZfdZedZedZedZedZxZ S)ScipySparseSliceablec@t||jyr )rKrtocsrrMs rrzScipySparseSliceable.__init__s 'rc"|jSr rfrs rrz%ScipySparseSliceable.convert_to_numpyrhrc>ttj|Sr )rLrscipy_sparse_to_tf_sparsers rr!z5ScipySparseSliceable.convert_to_tf_dataset_compatibles+  8 8 ;  rc,tj|Sr )rscipy_sparse_to_jax_sparsers rr$z.ScipySparseSliceable.convert_to_jax_compatible s!<= IIrcddlm}t|tr8|j|}|j |}|j |jz }n||j|j|}|j|j |}t|tr t|}n&|jd}||j|d}|j}|jddddf}|j|j|d|gd}|j}|ft!|j"jj%ddz}|j&|||S)Nrrr|r3)r5rr6r7rrstopstartr8listlenrQ value_rowids flat_valuesconcatr~tuplerPas_list SparseTensor)sparse_wrapperrr9sparse_indices sparse_values batch_dimr sparse_shapes rrSrS4sN='5!'66w?&44W= LL7==0 ">#@#@'J! .">">H gt $G I a(I $BHHW-a0 ))+G#//126NRYY  $n5AN"--M<%##++-ab1#L 2??>=, GGrc|duxszt|txshtj|xsQtj|xs:tj |xs#tj |xs t|dS)N __array__)r6 ARRAY_TYPESris_tensorflow_tensor is_jax_arrayis_torch_tensoris_scipy_sparsehasattr)rs rcan_slice_arrayrRs T  # a % #  2 21 5 #  * *1 - #  - -a 0  #  - -a 0  # 1k "rc:fd}tj||S)aConvert a structure of arrays into `Sliceable` instances Args: arrays: the arrays to convert. target_backend: the target backend for the output: - `None` indicates that `arrays` will be wrapped into `Sliceable`s as-is without using a different representation. This is used by `train_validation_split()`. - `tensorflow` indicates that `Sliceable.convert_to_tf_dataset_compatible` will be called. The returned structure therefore contains arrays, not `Sliceable`s. - `numpy`, `jax` or `torch` indices that the arrays will eventually be converted to this backend type after slicing. In this case, the intermediary `Sliceable`s may use a different representation from the input `arrays` for better performance. Returns: the same structure with `Sliceable` instances or arrays. c||St|tjrXt|jdk(r@t j dk(r)t d|Drtj|d}t|tjrt}nmtj|rBtj|rt}n;tj|rt}nt }ntj"|r8tj$|rt&}ntj(|}t}ntj*|rt,}nt.!t|t.j0rt2}nt.!t|t.j4rt6}n_tj8|rt:}nCt=|drtj(|}t}nt?d|dtA|d d d}t.Wt|t.j0r=tCfd |jDjFDr;t jH}n&|jrt jH}||jK||}||Sdk(r|jM|Sd k(r)|t t,fvrtj(|}t}||S) Nobjectrc3<K|]}t|tywr )r6str).0es r zEconvert_to_sliceable..convert_single_array..zs21Jq#&2sstringrrzExpected a NumPy array, tf.Tensor, tf.RaggedTensor, tf.SparseTensor, jax.np.ndarray, jax.experimental.sparse.JAXSparse, torch.Tensor, Pandas Dataframe, or Pandas Series. Received invalid input: z (of type )c|tuxrBtj|xr+tj|tjk( Sr )rris_float_dtypestandardize_dtypefloatxrs ris_non_floatx_floatzOconvert_to_sliceable..convert_single_array..is_non_floatx_floatsFV#M**51M11%8GNN.convert_single_array..sCa&q)Csjax)'r6r}ndarrayrrrallr9convert_to_tensorr-rris_tensorflow_raggedrCis_tensorflow_sparserIr1r is_jax_sparser_asarrayrrjpandas DataFramerxSeriesrzrrr ValueErrortypeanydtypesrrrr!)rsliceable_class cast_dtypertarget_backends @rconvert_single_arrayz2convert_to_sliceable..convert_single_arrayqsV 9H q"** %AGG (!\1222$$Qh7A a $,O  4 4Q 7!66q9";#88;";"5  , ,Q /!//2"4JJqM"0  / / 2,O  Jq&2B2B$C6O  Jq&--$@3O  / / 22O Q $ 1 A,OO#ZQy +    *Q0@0@"AC188??CC$^^- "177+$^^-  !$$Q 3A  !"1% % \ )"CCAF F U "  ; (  1 A,Oq!!r)r map_structure)arraysrrs ` rconvert_to_sliceabler^s &W"r   2F ;;rc, tj|}|Dcgc]}t|rt|}}|rt d|t d|Dr||fSd}|D]}||}nt |jd t tj d|z z dk(s k(rt d d|dd t|}tj fd |}tj fd |}||fScc}w) aSplit arrays into train and validation subsets in deterministic order. The last part of data will become validation data. Args: arrays: Tensors to split. Allowed inputs are arbitrarily nested structures of Tensors and NumPy arrays. validation_split: Float between 0 and 1. The proportion of the dataset to include in the validation split. The rest of the dataset will be included in the training split. Returns: `(train_arrays, validation_arrays)` zpArgument `validation_split` is only supported for tensors or NumPy arrays.Found incompatible type in the input: c3$K|]}|du ywr r)rts rrz)train_validation_split..s *19 *sNrg?zTraining data contains zs samples, which is not sufficient to split it into a validation and training set as specified by `validation_split=zV`. Either provide more data, or a different value for the `validation_split` argument.c||S|||Sr r)rrends r_splitz&train_validation_split.._splits 9Hs|rc|dS)Nrrrr)rrsplit_ats rz(train_validation_split..s&!2rc|S)Nrr)rrrrs rrz(train_validation_split..s&( :r) rflattenrrrrintrQmathfloorrr) rvalidation_split flat_arraysr unsplitablefirst_non_none sliceables train_arrays val_arraysrrrs @@@rtrain_validation_splitrsJ ,,v&K$/Jqq7I47JKJ 55@M C   *k **v~N  =N  N((+,I4::i31A+ABCDH1}I-%i[1..>-?@+ +   &f-J%%2JL##:JJ  ##UKs D Dr )$ collectionsrnumpyr} keras.srcrr keras.src.trainers.data_adaptersrr5rr9r ImportErrorrrrrr r-r1rCrIr_rjrorxrzr namedtuplerrLrSrrrrrrrs" ?9zzm  0@0@ AAK]]@ Y #)#, 3, 3,4* #Y #'i'*  5O5 9N1+00L JH< l<^;$A FsC22C<;C<