1Vh(HdZddlZddlZddlZGddZGddZy)zEA key-value[] store that implements reservoir sampling on the values.Nc8eZdZdZd dZdZdZdfdZd dZy) ReservoiraA map-to-arrays container, with deterministic Reservoir Sampling. Items are added with an associated key. Items may be retrieved by key, and a list of keys can also be retrieved. If size is not zero, then it dictates the maximum number of items that will be stored with each key. Once there are more items for a given key, they are replaced via reservoir sampling, such that each item has an equal probability of being included in the sample. Deterministic means that for any given seed and bucket size, the sequence of values that are kept for any given tag will always be the same, and that this is independent of any insertions on other tags. That is: >>> separate_reservoir = reservoir.Reservoir(10) >>> interleaved_reservoir = reservoir.Reservoir(10) >>> for i in range(100): >>> separate_reservoir.AddItem('key1', i) >>> for i in range(100): >>> separate_reservoir.AddItem('key2', i) >>> for i in range(100): >>> interleaved_reservoir.AddItem('key1', i) >>> interleaved_reservoir.AddItem('key2', i) separate_reservoir and interleaved_reservoir will be in identical states. See: https://en.wikipedia.org/wiki/Reservoir_sampling Adding items has amortized O(1) runtime. Fields: always_keep_last: Whether the latest seen sample is always at the end of the reservoir. Defaults to True. size: An integer of the maximum number of samples. cdkstk7rtdztjfd|_t j |_|_|_ y)a4Creates a new reservoir. Args: size: The number of values to keep in the reservoir for each tag. If 0, all values will be kept. seed: The seed of the random number generator to use when sampling. Different values for |seed| will produce different samples from the same input items. always_keep_last: Whether to always keep the latest seen item in the end of the reservoir. Defaults to True. Raises: ValueError: If size is negative or not an integer. rz(size must be nonnegative integer, was %scDttjSN)_ReservoirBucketrandomRandom)always_keep_lastseedsizes^/home/dcms/DCMS/lib/python3.12/site-packages/tensorboard/backend/event_processing/reservoir.pyz$Reservoir.__init__..Ms$fmmD)+;N) round ValueError collections defaultdict_buckets threadingLock_mutexr r )selfr r r s ```r__init__zReservoir.__init__;s^ !8tuT{*G$NO O#//    nn&  0rc|j5t|jjcdddS#1swYyxYw)zsReturn all the keys in the reservoir. Returns: ['list', 'of', 'keys'] in the Reservoir. N)rlistrkeysrs rKeyszReservoir.KeysWs5 [[ . **,- . . .s #:Ac|j5||jvrtd|z|j|}ddd|jS#1swYjSxYw)aReturn items associated with given key. Args: key: The key for which we are finding associated items. Raises: KeyError: If the key is not found in the reservoir. Returns: [list, of, items] associated with that key. z!Key %s was not found in ReservoirN)rrKeyErrorItems)rkeybuckets rr"zReservoir.Items`sb[[ ($--'BSHII]]3'F (||~  (||~s ,AA)c|Srxs rrzReservoir.rsQrc|j5|j|}dddj||y#1swYxYw)aAdd a new item to the Reservoir with the given tag. If the reservoir has not yet reached full size, the new item is guaranteed to be added. If the reservoir is full, then behavior depends on the always_keep_last boolean. If always_keep_last was set to true, the new item is guaranteed to be added to the reservoir, and either the previous last item will be replaced, or (with low probability) an older item will be replaced. If always_keep_last was set to false, then the new item will replace an old item with low probability. If f is provided, it will be applied to transform item (lazily, iff item is going to be included in the reservoir). Args: key: The key to store the item under. item: The item to add to the reservoir. f: An optional function to transform the item prior to addition. N)rrAddItem)rr#itemfr$s rr*zReservoir.AddItemrs<,[[ (]]3'F (tQ ( (s 8ANc"|j5|r?||jvr'|j|jcdddS dddytfd|jj DcdddS#1swYyxYw)aDFilter items within a Reservoir, using a filtering function. Args: filterFn: A function that returns True for the items to be kept. key: An optional bucket key to filter. If not specified, will filter all all buckets. Returns: The number of items removed. Nrc3@K|]}|jywr) FilterItems).0r$filterFns r z(Reservoir.FilterItems..s$&&x0s)rrr/sumvalues)rr1r#s ` rr/zReservoir.FilterItemss[[ $--'==-99(C     "&--"6"6"8   s.BB+BB)rTr) __name__ __module__ __qualname____doc__rrr"r*r/r&rrrrs& D18.$$/ 4rrc0eZdZdZddZdfdZdZdZy) rzA container for items from a stream, that implements reservoir sampling. It always stores the most recent item as its final item. Nc|dks|t|k7rtd|zg|_tj|_||_d|_|||_||_ ytjd|_||_ y)aCreate the _ReservoirBucket. Args: _max_size: The maximum size the reservoir bucket may grow to. If size is zero, the bucket has unbounded size. _random: The random number generator to use. If not specified, defaults to random.Random(0). always_keep_last: Whether the latest seen item should always be included in the end of the bucket. Raises: ValueError: if the size is not a nonnegative integer. rz)_max_size must be nonnegative int, was %sN) rritemsrrr _max_size_num_items_seen_randomr r r )rr<r>r s rrz_ReservoirBucket.__init__s q=Iy)99;iG   nn& "   "DL!1"==+DL 0rc|Srr&r's rrz_ReservoirBucket.src<|j5t|j|jks|jdk(r"|jj ||n|j j d|j}||jkr=|jj||jj ||n!|jr|||jd<|xjdz c_dddy#1swYyxYw)a;Add an item to the ReservoirBucket, replacing an old item if necessary. The new item is guaranteed to be added to the bucket, and to be the last element in the bucket. If the bucket has reached capacity, then an old item will be replaced. With probability (_max_size/_num_items_seen) a random item in the bucket will be popped out and the new item will be appended to the end. With probability (1 - _max_size/_num_items_seen) the last item in the bucket will be replaced. Since the O(n) replacements occur with O(1/_num_items_seen) likelihood, the amortized runtime is O(1). Args: item: The item to add to the bucket. f: A function to transform item before addition, if it will be kept in the reservoir. rN) rlenr;r<appendr>randintr=popr )rr+r,rs rr*z_ReservoirBucket.AddItems&[[ &4::/4>>Q3F !!!D'*LL((D,@,@At~~%JJNN1%JJ%%ag.**%&tWDJJrN  A %  & & &s CB$rr)r8rr rrrr&rrrPs/ L IIXg$g$r