2Vh+dZddlZddlZddlZddlmZddlmZedGddeZ edd d Z ed  d d Z y)z4Deprecated sequence preprocessing APIs from Keras 1.N) keras_export) PyDatasetz8keras._legacy.preprocessing.sequence.TimeseriesGeneratorc>eZdZdZ ddZdZdZdZdZy) TimeseriesGeneratoralUtility class for generating batches of temporal data. DEPRECATED. This class takes in a sequence of data-points gathered at equal intervals, along with time series parameters such as stride, length of history, etc., to produce batches for training/validation. Arguments: data: Indexable generator (such as list or Numpy array) containing consecutive data points (timesteps). The data should be at 2D, and axis 0 is expected to be the time dimension. targets: Targets corresponding to timesteps in `data`. It should have same length as `data`. length: Length of the output sequences (in number of timesteps). sampling_rate: Period between successive individual timesteps within sequences. For rate `r`, timesteps `data[i]`, `data[i-r]`, ... `data[i - length]` are used for create a sample sequence. stride: Period between successive output sequences. For stride `s`, consecutive output samples would be centered around `data[i]`, `data[i+s]`, `data[i+2*s]`, etc. start_index: Data points earlier than `start_index` will not be used in the output sequences. This is useful to reserve part of the data for test or validation. end_index: Data points later than `end_index` will not be used in the output sequences. This is useful to reserve part of the data for test or validation. shuffle: Whether to shuffle output samples, or instead draw them in chronological order. reverse: Boolean: if `true`, timesteps in each output sample will be in reverse chronological order. batch_size: Number of timeseries samples in each batch (except maybe the last one). Returns: A PyDataset instance. Nc t|t|k7r#tdt|dt|||_||_||_||_||_||z|_|t|dz }||_||_ | |_ | |_ |j|jkDr&td|jd|jdy)Nz;Data and targets have to be of same length. Data length is z while target length is z`start_index+length=z > end_index=zU` is disallowed, as no part of the sequence would be left to be used as current step.) len ValueErrordatatargetslength sampling_ratestride start_index end_indexshufflereverse batch_size) selfr r r rrrrrrrs W/home/dcms/DCMS/lib/python3.12/site-packages/keras/src/legacy/preprocessing/sequence.py__init__zTimeseriesGenerator.__init__7s t9G $225d)=**-g,9     * &/  D A I"  $   dnn ,&t'7'7&89#~~./<<  -c|j|jz |j|jzz|j|jzzS)N)rrrr)rs r__len__zTimeseriesGenerator.__len__`s@ NNT-- -$++0M Moo +- -rc |jrDtjj|j|j dz|j }n{|j|j |jz|zz}tj|t||j |jzz|j dz|j}tj|Dcgc]+}|j||jz ||j-c}}tj|Dcgc]}|j|c}}|jr|ddddddf|fS||fScc}wcc}w)Nr)size.)rnprandomrandintrrrrarangeminarrayr r rr r)rindexrowsirowsamplesr s r __getitem__zTimeseriesGenerator.__getitem__es> <<99$$  $..1"44??%D  4??T[[#@5#HHA99A$++55t~~7IJ D ((   # +cD4F4FFG   ((>#DLL->? <<1ddC<('1 1  ?s 0E%)E*c |j}t|jjtjk(r|jj } t j|}|j}t|jjtjk(r|jj } t j|}|||j|j|j|j|j|j|j |j"d S#t$r}td||d}~wwxYw#t$r}td||d}~wwxYw)zReturns the TimeseriesGenerator configuration as Python dictionary. Returns: A Python dictionary with the TimeseriesGenerator configuration. zData not JSON Serializable: NzTargets not JSON Serializable: ) r r r rrrrrrr)r type __module__r__name__tolistjsondumps TypeErrorr r rrrrrrr)rr json_dataer json_targetss r get_configzTimeseriesGenerator.get_config~s8 yy  ? % % 499##%D J 4(I,,   ( (BKK 7ll))+G P::g.L #kk!//kk++||||//   J:4&AB I J P=gYGHa O Ps0D6E6 E?EE E3E..E3c |j}|jj|d}tj|fi|S)aReturns a JSON string containing the generator's configuration. Args: **kwargs: Additional keyword arguments to be passed to `json.dumps()`. Returns: A JSON string containing the tokenizer configuration. ) class_nameconfig)r5 __class__r-r/r0)rkwargsr8timeseries_generator_configs rto_jsonzTimeseriesGenerator.to_jsons?"..11' #zz5@@@r)rrrNFF) r-r, __qualname____doc__rrr)r5r<rrrr s;'\'R-  2! FArrz8keras._legacy.preprocessing.sequence.make_sampling_tablecd}tj|}d|d<|tj||zzdzdd|zz z }||z}tjd|tj|z S)aCGenerates a word rank-based probabilistic sampling table. DEPRECATED. Used for generating the `sampling_table` argument for `skipgrams`. `sampling_table[i]` is the probability of sampling the word i-th most common word in a dataset (more common words should be sampled less frequently, for balance). The sampling probabilities are generated according to the sampling distribution used in word2vec: ``` p(word) = (min(1, sqrt(word_frequency / sampling_factor) / (word_frequency / sampling_factor))) ``` We assume that the word frequencies follow Zipf's law (s=1) to derive a numerical approximation of frequency(rank): `frequency(rank) ~ 1/(rank * (log(rank) + gamma) + 1/2 - 1/(12*rank))` where `gamma` is the Euler-Mascheroni constant. Args: size: Int, number of possible words to sample. sampling_factor: The sampling factor in the word2vec formula. Returns: A 1D Numpy array of length `size` where the ith entry is the probability that a word of rank i should be sampled. gX9v?rrg??g(@)rr!logminimumsqrt)rsampling_factorgammarankinv_fqfs rmake_sampling_tablerKsqB E 99T?DDG RVVD\E) *S 03$+3F FF& A ::c1rwwqz> **rz.keras._legacy.preprocessing.sequence.skipgramsc g}g} t|D]\} } | s ||| tjkr&td| |z } tt || |zdz} t | | D]J}|| k7s ||}|s|j | |g|r| j ddg:| j dL|dkDrtt | |z}|Dcgc]}|d }}tj||t |D cgc],} || t |ztjd|dz g.c} z }|r | ddgg|zz } n | dg|zz } |rl|tjdd}tj|tj|tj|tj| || fScc}wcc} w)aGenerates skipgram word pairs. DEPRECATED. This function transforms a sequence of word indexes (list of integers) into tuples of words of the form: - (word, word in the same window), with label 1 (positive samples). - (word, random word from the vocabulary), with label 0 (negative samples). Read more about Skipgram in this gnomic paper by Mikolov et al.: [Efficient Estimation of Word Representations in Vector Space](http://arxiv.org/pdf/1301.3781v3.pdf) Args: sequence: A word sequence (sentence), encoded as a list of word indices (integers). If using a `sampling_table`, word indices are expected to match the rank of the words in a reference dataset (e.g. 10 would encode the 10-th most frequently occurring token). Note that index 0 is expected to be a non-word and will be skipped. vocabulary_size: Int, maximum possible word index + 1 window_size: Int, size of sampling windows (technically half-window). The window of a word `w_i` will be `[i - window_size, i + window_size+1]`. negative_samples: Float >= 0. 0 for no negative (i.e. random) samples. 1 for same number as positive samples. shuffle: Whether to shuffle the word couples before returning them. categorical: bool. if False, labels will be integers (eg. `[0, 1, 1 .. ]`), if `True`, labels will be categorical, e.g. `[[1,0],[0,1],[0,1] .. ]`. sampling_table: 1D array of size `vocabulary_size` where the entry i encodes the probability to sample a word of rank i. seed: Random seed. Returns: couples, labels: where `couples` are int pairs and `labels` are either 0 or 1. Note: By convention, index 0 in the vocabulary is a non-word and will be skipped. rrgcA) enumeratermaxr"r rangeappendintrr seed)sequencevocabulary_size window_sizenegative_samplesr categoricalsampling_tablerRcoupleslabelsr&wi window_start window_endjwjnum_negative_samplescwordss r skipgramsrcsnG F8$%2   %b!FMMO31a+o. XK!(;< |Z0 %AAva[Bx(MM1a&)MM!$ %%(!"3v;1A#AB&'!1''u/0 1s5z> "FNN1o6I$J K    1vh!55 5F qc00 0F <>>!T*D Dw Dv F?)( s  G  1G)gh㈵>)rBTFNN) r?r/rnumpyrkeras.src.api_exportr3keras.src.trainers.data_adapters.py_dataset_adapterrrrKrcr@rrrhs: -IHIcA)cAJcALHI&+J&+R>?  b@br