BVhDdZddlZddlZddlmZddlmZddlmZddlm Z ddlm Z ddl m Z dd l m Z dd l mZdd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZddlm Z ddl!m"Z"ddl#m$Z$dZ%e$dgGdde&Z'e$ddgGdde'Z(d(d Z)d!Z*d"Z+d#Z,d$Z- d(d%Z.d&Z/d'Z0y))z"SavedModel builder implementation.N)Any) types_pb2)meta_graph_pb2)saved_model_pb2) saver_pb2)dtypes)ops)tensor)file_io) variables) tf_logging)fingerprinting_utils) path_helpers)signature_def_utils) constants)metrics)saver)compat)deprecated_args) tf_exportsave_v1_builderz*__internal__.saved_model.SavedModelBuilder)v1cneZdZdZdZd dZdZdZdZd dZ d d Z dd Z dd Z y)_SavedModelBuilderaBuilds the `SavedModel` protocol buffer and saves variables and assets. The `SavedModelBuilder` class provides the functionality to build a `SavedModel` protocol buffer. Specifically, this allows multiple meta graphs to be saved as part of a single language-neutral `SavedModel`, while sharing variables and assets. To build a SavedModel, the first meta graph must be saved with variables. Subsequent meta graphs will simply be saved with their graph definitions. If assets need to be saved and written or copied to disk, they can be provided when the meta graph def is added. If multiple meta graph defs are associated an asset of the same name, only the first version is retained. Each meta graph added to the SavedModel must be annotated with tags. The tags provide a means to identify the specific meta graph to load and restore, along with the shared set of variables and assets. Typical usage for the `SavedModelBuilder`: ```python ... builder = tf.compat.v1.saved_model.Builder(export_dir) with tf.compat.v1.Session(graph=tf.Graph()) as sess: ... builder.add_meta_graph_and_variables(sess, ["foo-tag"], signature_def_map=foo_signatures, assets_list=foo_assets) ... with tf.compat.v1.Session(graph=tf.Graph()) as sess: ... builder.add_meta_graph(["bar-tag", "baz-tag"]) ... builder.save() ``` Note: This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.builder.SavedModelBuilder or tf.compat.v1.saved_model.Builder. Tensorflow 2.0 will introduce a new object-based method of creating SavedModels. c^tj|_tj|j_||_tj|r$tj|r.td|dtj|j d|_ t|_y)NzExport directory z already exists, and isn't empty. Please choose a different export directory, or delete all the contents of the specified directory.F)r SavedModel _saved_modelrSAVED_MODEL_SCHEMA_VERSIONsaved_model_schema_version _export_dirr file_existslist_directoryAssertionErrorrecursive_create_dir_has_saved_variablesset_saved_asset_files)self export_dirs Z/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/saved_model/builder_impl.py__init__z_SavedModelBuilder.__init__^s'224D,, 0"D:&    + |,3 34 4 ""4#3#34!&D!eDNctjt|}t||}|st j dyt ||j|jy)zSaves asset to the meta graph and writes asset files to disk. Args: meta_graph_def: The meta graph def to which the assets will be added. assets_list: The list where the asset paths are setup. No assets to write.N) functoolspartial_add_asset_to_metagraph_maybe_save_assetsr infocopy_assets_to_destination_dirr r')r(meta_graph_def assets_listwrite_fnasset_filename_maps r*_save_and_write_assetsz)_SavedModelBuilder._save_and_write_assetsusW  !8.IH+HkB oo+, ##5t7G7G#'#:#:!!.1r,c8| td|jdtd|d|jddk(r-|jjD]}|j |y|j t jk(rtd|dy)akValidates the `TensorInfo` proto. Checks if the `encoding` (`name` or `coo_sparse` or `type_spec`) and `dtype` fields exist and are non-empty. Args: tensor_info: `TensorInfo` protocol buffer to validate. Raises: AssertionError: If the `encoding` or `dtype` fields of the supplied `TensorInfo` proto are not populated. NzXAll TensorInfo protos used in the SignatureDefs must have the name and dtype fields set.encodingzInvalid `tensor_info`: zx. All TensorInfo protos used in the SignatureDefs must have one of the 'encoding' fields (e.g., name or coo_sparse) set.composite_tensorzP. All TensorInfo protos used in the SignatureDefs must have the dtype field set.)r# WhichOneofrI components_validate_tensor_infodtyper DT_INVALID)r( tensor_info components r*rLz(_SavedModelBuilder._validate_tensor_infos  " ##j)1  #K=1% % &&j)-??"33>>.) ""9-.   i22 2  #K=1> > ?? 3r,ct|D]U}||}|j}|j}|D]}|j|||D]}|j||Wtj|vrt dtjdtj |vrt dtj dy)aEValidates the `SignatureDef` entries in the signature def map. Validation of entries in the signature def map includes ensuring that the `name` and `dtype` fields of the TensorInfo protos of the `inputs` and `outputs` of each `SignatureDef` are populated. Also ensures that reserved SignatureDef keys for the initialization and train ops are not used. Args: signature_def_map: The map of signature defs to be validated. Raises: AssertionError: If a TensorInfo is not valid. KeyError: If a reserved signature key is used in the map. zSignatureDef map key "z=" is reserved for initialization. Please use a different key.z;" is reserved for the train op. Please use a different key.N)inputsoutputsrLrINIT_OP_SIGNATURE_KEYKeyErrorTRAIN_OP_SIGNATURE_KEY)r(rBsignature_def_keyr>rRrS inputs_key outputs_keys r*_validate_signature_def_mapz._SavedModelBuilder._validate_signature_def_maps/9'(9:m##f%%g7* ""6*#567 9+ ""7;#789 9&&*;;  #I$C$C#DEE E FF''+<<  #I$D$D#EFC D EE=r,c|sCtjtjdtj j d}|S)z6Creates a sharded saver if one does not already exist.T)sharded write_version allow_empty)tf_saverSaverr _all_saveable_objectsrSaverDefV2)r(rs r*_maybe_create_saverz&_SavedModelBuilder._maybe_create_savers= nn  ) ) +!**-- e Lr,c^|js td|xsi}|j|t||tj t||tj |j|}|j|d}|j|||j|||y)aAdds the current meta graph to the SavedModel. Creates a Saver in the current scope and uses the Saver to export the meta graph def. Invoking this API requires the `add_meta_graph_and_variables()` API to have been invoked before. Args: tags: The set of tags to annotate the meta graph def with. signature_def_map: The map of signature defs to be added to the meta graph def. assets_list: Assets to be saved with SavedModel. Note that this list should be a subset of the assets saved as part of the first meta graph in the SavedModel. clear_devices: Set to true if the device info on the default graph should be cleared. init_op: Op or group of ops to execute when the graph is loaded. Note that when the init_op is specified it is run after the restore op at load-time. train_op: Op or group of opts that trains the model when run. This will not be run automatically when the graph is loaded, instead saved in a SignatureDef accessible through the exported MetaGraph. saver: An instance of tf.compat.v1.train.Saver that will be used to export the metagraph. If None, a sharded Saver that restores all variables will be used. Raises: AssertionError: If the variables for the SavedModel have not been saved yet, or if the graph already contains one or more legacy init ops. xGraph state including variables and assets has not been saved yet. Please invoke `add_meta_graph_and_variables()` first.T clear_devicesstrip_default_attrsN) r%r#rZ_add_op_to_signature_def_maprrTrVrdexport_meta_graphr9rF) r(r<rBr6rhinit_optrain_oprr5s r*add_meta_graphz!_SavedModelBuilder.add_meta_graphsJ  $ $  B CC */R$$%67!!2G!*!@!@B !2H!*!A!AC  $ $U +E,,#-?N  <   7HIr,c |jr td|xsi}|j|t||tj t||tj tj|jtj|j} |j| } | j|| dd| j||} |j| ||j| ||d|_y)aAdds the current meta graph to the SavedModel and saves variables. Creates a Saver to save the variables from the provided session. Exports the corresponding meta graph def. This function assumes that the variables to be saved have been initialized. For a given `SavedModelBuilder`, this API must be called exactly once and for the first meta graph to save. For subsequent meta graph defs to be added, the `add_meta_graph()` API must be used. Args: sess: The TensorFlow session from which to save the meta graph and variables. tags: The set of tags with which to save the meta graph. signature_def_map: The map of signature def map to add to the meta graph def. assets_list: Assets to be saved with SavedModel. clear_devices: Set to true if the device info on the default graph should be cleared. init_op: Op or group of ops to execute when the graph is loaded. Note that when the init_op is specified it is run after the restore op at load-time. train_op: Op or group of ops that trains the model when run. This will not be run automatically when the graph is loaded, instead saved in a SignatureDef accessible through the exported MetaGraph. strip_default_attrs: Boolean. If `True`, default-valued attributes will be removed from the NodeDefs. For a detailed guide, see [Stripping Default-Valued Attributes](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/saved_model/README.md#stripping-default-valued-attributes). saver: An instance of tf.compat.v1.train.Saver that will be used to export the metagraph and save variables. If None, a sharded Saver that restores all variables will be used. lGraph state including variables and assets has already been saved. Please invoke `add_meta_graph()` instead.Fwrite_meta_graph write_statergTN)r%r#rZrjrrTrVrget_or_create_variables_dirr get_variables_pathrdsaverkr9rF) r(sessr<rBr6rhrlrmrirvariables_pathr5s r*add_meta_graph_and_variablesz/_SavedModelBuilder.add_meta_graph_and_variables2s V   9 :: */R$$%67!!2G!*!@!@B !2H!*!A!AC,,T-=-=>!44T5E5EFN  $ $U +E  JJt^eJO,,#9L-NN  <   7HI!%Dr,cvtjttj|j stj |j |rtjtj|j tjtj}tj|t|jn!|rtj|j tj}t!j#dt%j#d t'dt(j+|jj-||}ntjtj|j tjtj.}tj||jj1dt3j4dtj6|tj8d|S)awWrites a `SavedModel` protocol buffer to disk. The function writes the SavedModel protocol buffer to the export directory in a serialized format. Args: as_text: Writes the SavedModel protocol buffer in text format to disk. Protocol buffers in text format are useful for debugging, but parsing fails when it encounters an unknown field and so is not forward compatible. This means changes to TensorFlow may prevent deployment of new text format SavedModels to existing serving binaries. Do not deploy `as_text` SavedModels to production. experimental_image_format: Writes the SavedModel protobuf in the experimental image format. See https://www.tensorflow.org/api_docs/python/tf/saved_model/SaveOptions for more details. This allows `SavedModelBuilder` to save models larger than 2 GiB. experimental_image_writer_options: Optional options for the experimental image writer. See https://github.com/google/riegeli/blob/master/doc/record_writer_options.md for available options. Raises: RuntimeError: When trying to use `proto_splitter` but `proto_splitter` is not imported. This check is here because `proto_splitter` is not available in OSS at the moment. Returns: The path to which the SavedModel protocol buffer was written. proto_splitterzDNo proto_splitter is provided, cannot use experimental_image_format.T) deterministiczSavedModel written to: %s1)r])rIncrementWriteApi_SAVE_BUILDER_LABELr r!r r$joinras_bytesrSAVED_MODEL_FILENAME_PBTXTwrite_string_to_filestrrSAVED_MODEL_FILENAME_PREFIXlocalsgetglobals RuntimeErrorr{SavedModelSplitterwriteSAVED_MODEL_FILENAME_PBSerializeToStringr r3as_textIncrementWrite)r(rexperimental_image_format!experimental_image_writer_optionspaths r*rvz_SavedModelBuilder.saves@ 12   t// 0 ""4#3#34 \\ //$** + //)>> ?Ad""4T->->)?@ "||     1 1  HLL)79==9I+J K ,  001B1BCII 3 || OOD,, - OOI== >  $$ $##55D5I OO/1EF - Kr,N)NNFNNN)NNFNNFN)FFN) __name__ __module__ __qualname____doc__r+r9rFrLrZrdrnryrvr,r*rr/sq+Z$.<(2,?>E@ (,!%#(!"FJV6:/316+/,07<)-[%z;@-1Hr,rzsaved_model.Builderz%saved_model.builder.SavedModelBuilderc|eZdZejj ddZfdZdZd dZdZ dZ e dd d  dd Z e dd d  dd Z ejjj dde _ejjj dde _xZS)SavedModelBuilderr6assets_collectionc.tt| |y)N)r))superrr+)r(r) __class__s r*r+zSavedModelBuilder.__init__s T+z+Br,cj|j||j||j|y)z)Add asset and op collections to be saved.N)r9_maybe_add_main_op _add_train_op)r(rmain_oprms r*_add_collectionsz"SavedModelBuilder._add_collectionss/  12G$x r,Nctt|}|stjdyt ||j |j y)zSaves asset to the meta graph and writes asset files to disk. Args: assets_collection_to_add: The collection where the asset paths are setup. r.N)r2_add_asset_to_collectionr r3r4r r')r(assets_collection_to_addr8s r*r9z(SavedModelBuilder._save_and_write_assetssI,,D,DF oo+, ##5t7G7G#'#:#:!44T5E5EFN  $ $U +E  JJt^eJO,,#9L-NN   7HI!%Dr,r)NNNFNFN)rrrrrreplacer+rr9rrrrnry __classcell__)rs@r*rrs  & & . .}/B D'C!<(:8>&4N#% (,'+$(#(!).'J%'JR4N#% 6:592616+/7<)-7%%7%r.<<DDLL(*.55==EE ,.&&r,rci}|tjd|S|D]8}t|}|std|dt ||}||||||<:tjd|S)aMSaves assets to the meta graph. Args: write_fn: A function callback that writes assets into meta graph. assets_to_add: The list where the asset paths are setup. Returns: A dict of asset basenames for saving to the original full path to the asset. Raises: ValueError: Indicating an invalid filepath tensor. zNo assets to save.zAsset filepath tensor z in is invalid.zAssets added to graph.)r r3_asset_path_from_tensorrget_asset_filename_to_add)r7 assets_to_addr8 asset_tensorasset_source_filepathasset_filenames r*r2r2sOO() $?l3LA / ~_M NN.13N ^\* *?~&%?( //*+ r,ctjj|}||vr|S||}||k(r|Stj||s t ||S|S)a Get a unique basename to add to the SavedModel if this file is unseen. Assets come from users as full paths, and we save them out to the SavedModel as basenames. In some cases, the basenames collide. Here, we dedupe asset basenames by first checking if the file is the same, and, if different, generate and return an index-suffixed basename that can be used to add the asset to the SavedModel. Args: asset_filepath: the full path to the asset that is being saved asset_filename_map: a dict of filenames used for saving the asset in the SavedModel to full paths from which the filenames were derived. Returns: Uniquified filename string if the file is not a duplicate, or the original filename if the file has already been seen and saved. )osrbasenamer filecmp_get_unique_asset_filename)asset_filepathr8rother_asset_filepaths r*rrsi$77##N3.-- +N;^+  )= > %n6H II r,cd}|}||vr_tjdjtj|tjt|g}|dz }||vr_|S)N_)rrrr)rr8iunique_filenames r*rrsi!"/--ooc*//  (&//#a&*ABDOFA -- r,ct|tjstd|d|jj dk7r&td|d|jj d|j tjk7rtd|d|j d|jjdj}t|dk7rtd|d |d S) aReturns the filepath value stored in constant `path_tensor`. Args: path_tensor: Tensor of a file-path. Returns: The string value i.e. path of the tensor, if valid. Raises: TypeError if tensor does not match expected op type, dtype or value. zAsset path tensor z must be a Tensor.Constz# must be of type constant.Has type rz#` must be of dtype string.Has type valuerz must be a scalar.r) rr rroprrMrstringget_attr string_vallen) path_tensor str_valuess r*rrs K / ( 5GH II^^G# ( 6 + 3 34I? @@&--' ( 6 + 1 12)= >>~~&&w/::*_ ( 5GH II Ar,c||jj}||_|j|j_y)a&Builds an asset proto and adds it to the meta graph def. Args: meta_graph_def: The meta graph def to which the asset will be added. asset_filename: The filename of the asset to be added. asset_tensor: The asset tensor used to populate the tensor info of the asset proto. N)asset_file_defrAfilenamenamerO)r5rr asset_protos r*r1r1s4--113+'+!-!2!2+r,c| t}tj|}|jD]\}}t j t j|t j|}t j|sX||k7s^||vsct j||d|j|tjdt j|y)aCopy all assets from source path to destination path. Args: asset_filename_map: a dict of filenames used for saving the asset in the SavedModel to full paths from which the filenames were derived. destination_dir: the destination directory that assets are stored in. saved_files: a set of destination filepaths that have already been copied and will be skipped NT) overwritezAssets written to: %s)r&rget_or_create_assets_diritemsr rrrr!copyrAr r3r)r8destination_dir saved_filesassets_destination_dirasset_basenamerasset_destination_filepaths r*r4r4$s%K'@@0B/G/G/I 2+n+!(./'") 12!;;"+5 ll !;tMoo01 2 //)..!78:r,ctj}||_|j|j_t }|j |tjtj|y)zBuilds an asset proto and adds it to the asset collection of the graph. Args: asset_filename: The filename of the asset to be added. asset_tensor: The asset tensor used to populate the tensor info of the asset proto. N) r AssetFileDefrrrOrPackr rr ASSETS_KEY)rrrasset_any_protos r*rrHsY++-+'+!-!2!2+E/{# ,,o>r,c<|tj||||<yyr)rop_signature_def)rBrrDs r*rjrjYs$^0AA"cJcr,r)1rr/rgoogle.protobuf.any_pb2rtensorflow.core.frameworkrtensorflow.core.protobufrrrtensorflow.python.frameworkrr r tensorflow.python.lib.ior tensorflow.python.opsr tensorflow.python.platformr tensorflow.python.saved_modelrrr0tensorflow.python.saved_model.pywrap_saved_modelrrtensorflow.python.trainingrr_tensorflow.python.utilr"tensorflow.python.util.deprecationr tensorflow.python.util.tf_exportrrobjectrrr2rrrr1r4rrjrr,r*rs) '/34..+.,+1>6=FD8)>6(  7B?gg@gT  $&MNO.*.P.D+\$N4 304!:H?"Kr,