6biC6SrSSKrSSKrSSKrSSKJr SSKJr Sr \"S5"SS\R55r \"S 5S S j5r \"S 5SS j5r g)aiUtilities for preprocessing sequence data. Deprecated: `tf.keras.preprocessing.sequence` APIs are not recommended for new code. Prefer `tf.keras.utils.timeseries_dataset_from_array` and the `tf.data` APIs which provide a much more flexible mechanisms for dealing with sequences. See the [tf.data guide](https://www.tensorflow.org/guide/data) for more details. N) data_utils) keras_exportc//pC[X5H8upV[U5U:dMURU5 URU5 M: X44$)a'Removes sequences that exceed the maximum length. Args: maxlen: Int, maximum length of the output sequences. seq: List of lists, where each sublist is a sequence. label: List where each element is an integer. Returns: new_seq, new_label: shortened lists for `seq` and `label`. )ziplenappend)maxlenseqlabelnew_seq new_labelxys ]/home/james-whalen/.local/lib/python3.13/site-packages/tf_keras/src/preprocessing/sequence.py_remove_long_seqr$sKRYC q6F? NN1    Q    z0keras.preprocessing.sequence.TimeseriesGeneratorcH\rSrSrSrS SjrSrSrSrSr S r g) TimeseriesGenerator7a8 Utility class for generating batches of temporal data. Deprecated: `tf.keras.preprocessing.sequence.TimeseriesGenerator` does not operate on tensors and is not recommended for new code. Prefer using a `tf.data.Dataset` which provides a more efficient and flexible mechanism for batching, shuffling, and windowing input. See the [tf.data guide](https://www.tensorflow.org/guide/data) for more details. This class takes in a sequence of data-points gathered at equal intervals, along with time series parameters such as stride, length of history, etc., to produce batches for training/validation. Arguments: data: Indexable generator (such as list or Numpy array) containing consecutive data points (timesteps). The data should be at 2D, and axis 0 is expected to be the time dimension. targets: Targets corresponding to timesteps in `data`. It should have same length as `data`. length: Length of the output sequences (in number of timesteps). sampling_rate: Period between successive individual timesteps within sequences. For rate `r`, timesteps `data[i]`, `data[i-r]`, ... `data[i - length]` are used for create a sample sequence. stride: Period between successive output sequences. For stride `s`, consecutive output samples would be centered around `data[i]`, `data[i+s]`, `data[i+2*s]`, etc. start_index: Data points earlier than `start_index` will not be used in the output sequences. This is useful to reserve part of the data for test or validation. end_index: Data points later than `end_index` will not be used in the output sequences. This is useful to reserve part of the data for test or validation. shuffle: Whether to shuffle output samples, or instead draw them in chronological order. reverse: Boolean: if `true`, timesteps in each output sample will be in reverse chronological order. batch_size: Number of timeseries samples in each batch (except maybe the last one). Returns: A [Sequence]( https://www.tensorflow.org/api_docs/python/tf/tf_keras/utils/Sequence) instance. Examples: ```python from tf_keras.src.preprocessing.sequence import TimeseriesGenerator import numpy as np data = np.array([[i] for i in range(50)]) targets = np.array([[i] for i in range(50)]) data_gen = TimeseriesGenerator(data, targets, length=10, sampling_rate=2, batch_size=2) assert len(data_gen) == 20 batch_0 = data_gen[0] x, y = batch_0 assert np.array_equal(x, np.array([[[0], [2], [4], [6], [8]], [[1], [3], [5], [7], [9]]])) assert np.array_equal(y, np.array([[10], [11]])) ``` Nc [U5[U5:wa)[SS[U53-S[U53-5eXlX lX0lX@lXPlXc-UlUc[U5S- nXplXl Xl Xl URUR:a$[SURUR4-5eg)NzData and targets have to bez of same length. Data length is z while target length is zz`start_index+length=%i > end_index=%i` is disallowed, as no part of the sequence would be left to be used as current step.) r ValueErrordatatargetslength sampling_ratestride start_index end_indexshufflereverse batch_size) selfrrrrrrrr r!r"s r__init__TimeseriesGenerator.__init__{s t9G $-4SYK@A,S\N;<     * &/  D A I"  $   dnn ,<##T^^45  -rcURUR- URUR--URUR--$)N)rrr"r)r#s r__len__TimeseriesGenerator.__len__s@ NNT-- -$++0M Moo +- -rc UR(aA[RRURUR S-UR S9nO{URUR UR-U--n[R"U[X0R UR--UR S-5UR5n[R"UVs/sH*nURX@R- X@R2PM, sn5n[R"UVs/sHo@RUPM sn5nUR(aUSS2SSS2S4U4$XV4$s snfs snf)Nr)size.)r nprandomrandintrrr"rarangeminarrayrrrrr!)r#indexrowsirowsamplesrs r __getitem__TimeseriesGenerator.__getitem__s= <<99$$  $..1"44??%D  4??T[[#@5#HHA99A$++55t~~7IJ D ((  C # +c4F4FFG   ((>#LL->? <<1ddC<('1 1  ?s "1E0.E5c URn[UR5R[R:XaURR 5n[ R"U5nURn[UR5R[R:XaURR 5n[ R"U5nUUURURURURURURUR UR"S. $![an[SU5UeSnAff=f![an[SU5UeSnAff=f)zReturns the TimeseriesGenerator configuration as Python dictionary. Returns: A Python dictionary with the TimeseriesGenerator configuration. zData not JSON Serializable:NzTargets not JSON Serializable:) rrrrrrrr r!r")rtype __module__r,__name__tolistjsondumps TypeErrorrrrrrrr r!r")r#r json_dataer json_targetss r get_configTimeseriesGenerator.get_configs. yy  ? % % 499##%D H 4(I,,   ( (BKK 7ll))+G N::g.L #kk!//kk++||||//   H94@a G H Nr?)r#kwargsrHtimeseries_generator_configs rto_jsonTimeseriesGenerator.to_jsons?"..11' #zz5@@@r) r"rrrr!rr rrr)rrrNFF) r<r; __qualname____firstlineno____doc__r$r'r7rDrL__static_attributes__rrrr7s<@N'R-  2! FArrz0keras.preprocessing.sequence.make_sampling_tablecSn[R"U5nSUS'U[R"U5U--S-SSU-- - nX-n[R"SU[R"U5- 5$)aGenerates a word rank-based probabilistic sampling table. Used for generating the `sampling_table` argument for `skipgrams`. `sampling_table[i]` is the probability of sampling the word i-th most common word in a dataset (more common words should be sampled less frequently, for balance). The sampling probabilities are generated according to the sampling distribution used in word2vec: ``` p(word) = (min(1, sqrt(word_frequency / sampling_factor) / (word_frequency / sampling_factor))) ``` We assume that the word frequencies follow Zipf's law (s=1) to derive a numerical approximation of frequency(rank): `frequency(rank) ~ 1/(rank * (log(rank) + gamma) + 1/2 - 1/(12*rank))` where `gamma` is the Euler-Mascheroni constant. Args: size: Int, number of possible words to sample. sampling_factor: The sampling factor in the word2vec formula. Returns: A 1D Numpy array of length `size` where the ith entry is the probability that a word of rank i should be sampled. gX9v?rrg??g(@)r,r/logminimumsqrt)r*sampling_factorgammarankinv_fqfs rmake_sampling_tabler^sn> E 99T?DDG RVVD\E) *S 03$+3F FF A ::c1rwwqz> **rz&keras.preprocessing.sequence.skipgramsc /n/n [U5HupU (dMUbXk[R"5:aM.[SX- 5n [[ U5X-S-5n [ X5HVnX:wdM XnU(dMUR X/5 U(aU R SS/5 MEU R S5 MX M US:a[[ U 5U-5nUVs/sHnUSPM nn[R"U5 U[ U5V s/sH.n UU [ U5-[R"SUS- 5/PM0 sn - nU(a U SS//U-- n O U S/U-- n U(arUc[R"SS5n[R"U5 [R"U5 [R"U5 [R"U 5 X4$s snfs sn f)aGenerates skipgram word pairs. This function transforms a sequence of word indexes (list of integers) into tuples of words of the form: - (word, word in the same window), with label 1 (positive samples). - (word, random word from the vocabulary), with label 0 (negative samples). Read more about Skipgram in this gnomic paper by Mikolov et al.: [Efficient Estimation of Word Representations in Vector Space](http://arxiv.org/pdf/1301.3781v3.pdf) Args: sequence: A word sequence (sentence), encoded as a list of word indices (integers). If using a `sampling_table`, word indices are expected to match the rank of the words in a reference dataset (e.g. 10 would encode the 10-th most frequently occurring token). Note that index 0 is expected to be a non-word and will be skipped. vocabulary_size: Int, maximum possible word index + 1 window_size: Int, size of sampling windows (technically half-window). The window of a word `w_i` will be `[i - window_size, i + window_size+1]`. negative_samples: Float >= 0. 0 for no negative (i.e. random) samples. 1 for same number as positive samples. shuffle: Whether to shuffle the word couples before returning them. categorical: bool. if False, labels will be integers (eg. `[0, 1, 1 .. ]`), if `True`, labels will be categorical, e.g. `[[1,0],[0,1],[0,1] .. ]`. sampling_table: 1D array of size `vocabulary_size` where the entry i encodes the probability to sample a word of rank i. seed: Random seed. Returns: couples, labels: where `couples` are int pairs and `labels` are either 0 or 1. Note: By convention, index 0 in the vocabulary is a non-word and will be skipped. rrgcA) enumerater-maxr0rrangerintr r.seed)sequencevocabulary_size window_sizenegative_samplesr categoricalsampling_tablerdcoupleslabelsr4wi window_start window_endjwjnum_negative_samplescwordss r skipgramsrusjG F8$   %!FMMO31ao. X!(;< |0Av[x(MM1a&)MM!$1%(!"3v;1A#AB&'w!1w'u/0 01s5z> "FNN1o6I$J K0    1vh!55 5F qc00 0F <>>!T*D Dw Dv ?)( s 0G7%5G<)gh㈵>)rUTFNN)rQr>r-numpyr,tf_keras.src.utilsr tensorflow.python.util.tf_exportrrSequencerr^rurSrrr{s ):&@A|A*--|AB|A~@A$+B$+N67  `8`r