BVh /dZddlZddlmZddlmZddlmZddlm Z ddl m Z ddl mZdd lmZdd lmZed g Gd deZy)zManages a Checkpoint View.N)trackable_object_graph_pb2)trackable_view) errors_impl) tf_logging)base)py_checkpoint_reader)object_identity) tf_exportztrain.CheckpointView)v1c4eZdZdZdZdZdZdZdZdZ y) CheckpointViewaaGathers and serializes a checkpoint view. This is for loading specific portions of a module from a checkpoint, and be able to compare two modules by matching components. Example usage: >>> class SimpleModule(tf.Module): ... def __init__(self, name=None): ... super().__init__(name=name) ... self.a_var = tf.Variable(5.0) ... self.b_var = tf.Variable(4.0) ... self.vars = [tf.Variable(1.0), tf.Variable(2.0)] >>> root = SimpleModule(name="root") >>> root.leaf = SimpleModule(name="leaf") >>> ckpt = tf.train.Checkpoint(root) >>> save_path = ckpt.save('/tmp/tf_ckpts') >>> checkpoint_view = tf.train.CheckpointView(save_path) Pass `node_id=0` to `tf.train.CheckpointView.children()` to get the dictionary of all children directly linked to the checkpoint root. >>> for name, node_id in checkpoint_view.children(0).items(): ... print(f"- name: '{name}', node_id: {node_id}") - name: 'a_var', node_id: 1 - name: 'b_var', node_id: 2 - name: 'vars', node_id: 3 - name: 'leaf', node_id: 4 - name: 'root', node_id: 0 - name: 'save_counter', node_id: 5 c>tj|} |jtj}tj}|j|||_ y#t j $r&}td|dtjd|d}~wwxYw)zConfigure the checkpoint view. Args: save_path: The path to the checkpoint. Raises: ValueError: If the save_path does not lead to a TF2 checkpoint. zThe specified checkpoint "zS" does not appear to be object-based (saved with TF2) since it is missing the key "zg". Likely it was created with the TF1 name-based saver and does not contain an object dependency graph.N) rNewCheckpointReader get_tensorrOBJECT_GRAPH_PROTO_KEYr NotFoundError ValueErrorrTrackableObjectGraphParseFromString_object_graph_proto)self save_pathreaderobject_graph_stringnot_found_errorobject_graph_protos \/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/checkpoint/checkpoint_view.py__init__zCheckpointView.__init__@s" 5 5i @F"--d.I.IJ5IIK&&':;1D  $ $  ' {3**+,R R   sA##B6!BBc|jj|jDcic]}|j|jc}Scc}w)zReturns all child trackables attached to obj. Args: node_id: Id of the node to return its children. Returns: Dictionary of all children attached to the object with name to node_id. )rnodeschildren local_namenode_id)rr#childs rr!zCheckpointView.childrenXsJ--33G<EE   %--'  sAcPt|jjS)z8Returns a list of trackables by node_id attached to obj.)list_descendants_with_pathskeys)rs r descendantszCheckpointView.descendantsfs! ,,.335 66ci}tjdg}d|d<|jd}|r|j}|jj |}|j D]}|jdk(s|j|jvr/|j|}|j|jvr|j|j|dz|jz||j<|r|S)zReturns a dict of descendants by node_id and paths to node. The names returned by this private method are subject to change. rroot.) collectionsdequegetpopleftrr r!r#r(appendr")rall_nodes_with_pathsto_visitpathr#objr$s rr'z&CheckpointView._descendants_with_pathsks   !%H$  # #A &D   "g  $ $ * *7 3c<<L% ==A 2F2K2K2M!M #''0 == 4 9 9 ; ; //%-- (.2Sj5;K;K.KU]]+ L  r*c~t|tjstd|dt |di}||d<t j d|fg}t}tj|}|r|j\}}|j|}|j|jD]m\} } | |vs| dk(r| |vs|j| } | || || <|j| || fG| || usOtj d| d|| do|j#||r|S)a>Returns all matching trackables between CheckpointView and Trackable. Matching trackables represents trackables with the same name and position in graph. Args: obj: `Trackable` root. Returns: Dictionary containing all overlapping trackables that maps `node_id` to `Trackable`. Example usage: >>> class SimpleModule(tf.Module): ... def __init__(self, name=None): ... super().__init__(name=name) ... self.a_var = tf.Variable(5.0) ... self.b_var = tf.Variable(4.0) ... self.vars = [tf.Variable(1.0), tf.Variable(2.0)] >>> root = SimpleModule(name="root") >>> leaf = root.leaf = SimpleModule(name="leaf") >>> leaf.leaf3 = tf.Variable(6.0, name="leaf3") >>> leaf.leaf4 = tf.Variable(7.0, name="leaf4") >>> ckpt = tf.train.Checkpoint(root) >>> save_path = ckpt.save('/tmp/tf_ckpts') >>> checkpoint_view = tf.train.CheckpointView(save_path) >>> root2 = SimpleModule(name="root") >>> leaf2 = root2.leaf2 = SimpleModule(name="leaf2") >>> leaf2.leaf3 = tf.Variable(6.0) >>> leaf2.leaf4 = tf.Variable(7.0) Pass `node_id=0` to `tf.train.CheckpointView.children()` to get the dictionary of all children directly linked to the checkpoint root. >>> checkpoint_view_match = checkpoint_view.match(root2).items() >>> for item in checkpoint_view_match: ... print(item) (0, ...) (1, ) (2, ) (3, ListWrapper([, ])) (6, ) (7, ) zExpected a Trackable, got z of type r-rzjInconsistent references when matching the checkpoint into this object graph. The referenced objects are: (z and z).) isinstancer Trackablertyper.r/setr TrackableViewr1r!itemsr0r2loggingwarningadd) rr6overlapping_nodesr4visitedviewcurrent_node_idcurrent_trackabletrackable_children child_name child_node_idcurrent_assignments rmatchzCheckpointView.matchsnd c4>> * 3C5 $s)AN OOa  1c(,HeG  ' ' ,D +3+;+;+=(o(==):;'+}}_'E'K'K'M9 #*m G #}'9  + +044]C   '/A*/M m , OO],>z,JK L ");J)GGoo()' 34B899&kk/"- . r*cf|j|}g}g}|jD]&}||jvs|j|(t j |jD]9}|t j|jvs)|j|;|||fS)a: Returns diff between CheckpointView and Trackable. This method is intended to be used to compare the object stored in a checkpoint vs a live model in Python. For example, if checkpoint restoration fails the `assert_consumed()` or `assert_existing_objects_matched()` checks, you can use this to list out the objects/checkpoint nodes which were not restored. Example Usage: >>> class SimpleModule(tf.Module): ... def __init__(self, name=None): ... super().__init__(name=name) ... self.a_var = tf.Variable(5.0) ... self.b_var = tf.Variable(4.0) ... self.vars = [tf.Variable(1.0), tf.Variable(2.0)] >>> root = SimpleModule(name="root") >>> leaf = root.leaf = SimpleModule(name="leaf") >>> leaf.leaf3 = tf.Variable(6.0, name="leaf3") >>> leaf.leaf4 = tf.Variable(7.0, name="leaf4") >>> ckpt = tf.train.Checkpoint(root) >>> save_path = ckpt.save('/tmp/tf_ckpts') >>> checkpoint_view = tf.train.CheckpointView(save_path) >>> root2 = SimpleModule(name="root") >>> leaf2 = root2.leaf2 = SimpleModule(name="leaf2") >>> leaf2.leaf3 = tf.Variable(6.0) >>> leaf2.leaf4 = tf.Variable(7.0) Pass `node_id=0` to `tf.train.CheckpointView.children()` to get the dictionary of all children directly linked to the checkpoint root. >>> checkpoint_view_diff = checkpoint_view.diff(root2) >>> checkpoint_view_match = checkpoint_view_diff[0].items() >>> for item in checkpoint_view_match: ... print(item) (0, ...) (1, ) (2, ) (3, ListWrapper([, ])) (6, ) (7, ) >>> only_in_checkpoint_view = checkpoint_view_diff[1] >>> print(only_in_checkpoint_view) [4, 5, 8, 9, 10, 11, 12, 13, 14] >>> only_in_trackable = checkpoint_view_diff[2] >>> print(only_in_trackable) [..., , , ListWrapper([, ]), , , , ] Args: obj: `Trackable` root. Returns: Tuple of ( - Overlaps: Dictionary containing all overlapping trackables that maps `node_id` to `Trackable`, same as CheckpointView.match(). - Only in CheckpointView: List of `node_id` that only exist in CheckpointView. - Only in Trackable: List of `Trackable` that only exist in Trackable. ) ) rJr)r(r2rr<r ObjectIdentitySetvalues)rr6rAonly_in_checkpoint_viewonly_in_trackabler# trackables rdiffzCheckpointView.diffsV 3 ##%0 )..0 0&&w/0$11#6BBD, /;;  " " $& &  +, 57H HHr*N) __name__ __module__ __qualname____doc__rr!r)r'rJrQr*rr r s* D20 7  .TlUIr*r )rUr.tensorflow.core.protobufrtensorflow.python.checkpointrtensorflow.python.frameworkrtensorflow.python.platformrr>tensorflow.python.trackablertensorflow.python.trainingrtensorflow.python.utilr tensorflow.python.util.tf_exportr objectr rVr*rr`sN ?73<,;26 !b)PIVPI*PIr*