1Vh|dZddlmZddlmZddlmZddlmZejZ e Z GddZ y) z1Implementation for a multi-file directory loader.)directory_watcher) io_wrapper)tf) tb_loggingc2eZdZdZddfdZdZdZdZy) DirectoryLoaderaLoader for an entire directory, maintaining multiple active file loaders. This class takes a directory, a factory for loaders, and optionally a path filter and watches all the paths inside that directory for new data. Each file loader created by the factory must read a path and produce an iterator of (timestamp, value) pairs. Unlike DirectoryWatcher, this class does not assume that only one file receives new data at a time; there can be arbitrarily many active files. However, any file whose maximum load timestamp fails an "active" predicate will be marked as inactive and no longer checked for new data. cyNT)xs e/home/dcms/DCMS/lib/python3.12/site-packages/tensorboard/backend/event_processing/directory_loader.pyzDirectoryLoader.3cyr r ) timestamps r rzDirectoryLoader.4rrc| td| td||_||_||_||_i|_i|_y)aConstructs a new MultiFileDirectoryLoader. Args: directory: The directory to load files from. loader_factory: A factory for creating loaders. The factory should take a path and return an object that has a Load method returning an iterator yielding (unix timestamp as float, value) pairs for any new data path_filter: If specified, only paths matching this filter are loaded. active_filter: If specified, any loader whose maximum load timestamp does not pass this filter will be marked as inactive and no longer read. Raises: ValueError: If directory or loader_factory are None. NzA directory is requiredzA loader factory is required) ValueError _directory_loader_factory _path_filter_active_filter_loaders_max_timestamps)self directoryloader_factory path_filter active_filters r __init__zDirectoryLoader.__init__/sV*  67 7  !;< <#-'+ !rc#K tjj}tfd|D}|D]}j |D]}|y#t j j$rw}t jjjjs"tjdjztjd|zYd}~yd}~wwxYww)aLoads new values from all active files. Yields: All values that have not been yielded yet. Raises: DirectoryDeletedError: If the directory has been permanently deleted (as opposed to being temporarily unavailable). c3FK|]}j|s|yw)N)r).0prs r z'DirectoryLoader.Load..[sH43D3DQ3G1Hs!!z)Directory %s has been permanently deletedz&Ignoring error during file loading: %sN)rListDirectoryAbsolutersorted _LoadPathrerrorsOpErroriogfileexistsrDirectoryDeletedErrorloggerinfo)r all_pathspathspathvaluees` r LoadzDirectoryLoader.LoadOs J"88IIHiHHE !^^D1 EK  yy   J55;;%%doo6'==?oo&  DqHII Js0C1AAC1C.7A-C)$C1)C..C1c#PK|jj|d}|tus|j||rtj d|y|j j|d}|! |j|}||j |<tjd||jD]\}}|||kDr|}||j||s||j|<yy#tjj$rtj d|YywxYww)zGenerator for values from a single path's loader. Args: path: the path to load from Yields: All values from this path's loader that have not been yielded yet. NzSkipping inactive path %szSkipping nonexistent path %szLoading data from path %s) rget _INACTIVE_MarkIfInactiver/debugrrrr) NotFoundErrorr0r6)rr3 max_timestamploaderrr4s r r(zDirectoryLoader._LoadPathhs!,,00t< I %)=)= -*  LL4d ; ""4. > --d3 #)DMM$  /6 &   Iu$ M(A ) K ##D-8)6D  &999**  ;TB s+A,D&/C-A-D&-3D# D&"D##D&ctjd|||2|j|s!t|j|<|j |=yy)zNIf max_timestamp is inactive, returns True and marks the path as such.z"Checking active status of %s at %sTF)r/r;rr9rr)rr3r=s r r:zDirectoryLoader._MarkIfInactivesJ  94O  $T-@-@-O)2D  & d#rN)__name__ __module__ __qualname____doc__r r6r(r:r rr rr s& $#, "@J27@rrN) rC$tensorboard.backend.event_processingrrtensorboard.compatrtensorboard.utilr get_loggerr/objectr9rr rr rIs@ 8C;!'     H ppr