2Vh o d Z ddlmZ ddlmZ ddlmZ ddlmZ ddlmZ ddlm Z ddl
m Z dd l m
Z ej d
k( rdd lmZ n ej d k( rdd
lmZ nl ej dk( rddlmZ nQ ej dk( rddlmZ n6 ej dk( rddlmZ n ed ej d dZ ed G d de Z d dZ d d Z d Z y) z9Library for exporting SavedModel for Keras models/layers. )backend)layers)tree) keras_export)get_input_signature)make_tf_tensor_spec)io_utils)
tensorflowr
)TFExportArchivejax)JaxExportArchivetorch)TorchExportArchivenumpy)NumpyExportArchiveopenvino)OpenvinoExportArchivez Backend 'z ' must implement ExportArchive.servezkeras.export.ExportArchivec e Zd ZdZ fdZed Zed Zed Zd Z d fd Z
fdZ d Z dd
Z
d Zd Zd
Zd Z xZS )
ExportArchivea ExportArchive is used to write SavedModel artifacts (e.g. for inference).
If you have a Keras model or layer that you want to export as SavedModel for
serving (e.g. via TensorFlow-Serving), you can use `ExportArchive`
to configure the different serving endpoints you need to make available,
as well as their signatures. Simply instantiate an `ExportArchive`,
use `track()` to register the layer(s) or model(s) to be used,
then use the `add_endpoint()` method to register a new serving endpoint.
When done, use the `write_out()` method to save the artifact.
The resulting artifact is a SavedModel and can be reloaded via
`tf.saved_model.load`.
Examples:
Here's how to export a model for inference.
```python
export_archive = ExportArchive()
export_archive.track(model)
export_archive.add_endpoint(
name="serve",
fn=model.call,
input_signature=[keras.InputSpec(shape=(None, 3), dtype="float32")],
)
export_archive.write_out("path/to/location")
# Elsewhere, we can reload the artifact and serve it.
# The endpoint we added is available as a method:
serving_model = tf.saved_model.load("path/to/location")
outputs = serving_model.serve(inputs)
```
Here's how to export a model with one endpoint for inference and one
endpoint for a training-mode forward pass (e.g. with dropout on).
```python
export_archive = ExportArchive()
export_archive.track(model)
export_archive.add_endpoint(
name="call_inference",
fn=lambda x: model.call(x, training=False),
input_signature=[keras.InputSpec(shape=(None, 3), dtype="float32")],
)
export_archive.add_endpoint(
name="call_training",
fn=lambda x: model.call(x, training=True),
input_signature=[keras.InputSpec(shape=(None, 3), dtype="float32")],
)
export_archive.write_out("path/to/location")
```
**Note on resource tracking:**
`ExportArchive` is able to automatically track all `keras.Variables` used
by its endpoints, so most of the time calling `.track(model)`
is not strictly required. However, if your model uses lookup layers such
as `IntegerLookup`, `StringLookup`, or `TextVectorization`,
it will need to be tracked explicitly via `.track(model)`.
Explicit tracking is also required if you need to be able to access
the properties `variables`, `trainable_variables`, or
`non_trainable_variables` on the revived archive.
c j t | t j dvr t d g | _ i | _ t j | _ t j j j | _ g | j _
g | j _ g | j _ y )N)r
r r zK`ExportArchive` is only compatible with TensorFlow, JAX and Torch backends.)super__init__r NotImplementedError_endpoint_names_endpoint_signaturestf __version__tensorflow_version __internal__tracking
AutoTrackable
_tf_trackable variablestrainable_variablesnon_trainable_variables)self __class__s L/home/dcms/DCMS/lib/python3.12/site-packages/keras/src/export/saved_model.pyr zExportArchive.__init__l s
?? $B B%"
" $&!"$.. __ 5 5 C C E')$13.572 c . | j j S N)r# r$ r' s r) r$ zExportArchive.variables} s !!+++r* c . | j j S r, )r# r% r- s r) r% z!ExportArchive.trainable_variables s !!555r* c . | j j S r, )r# r& r- s r) r& z%ExportArchive.non_trainable_variables s !!999r* c t |t j r|j s t d t |t
j j j r.t | d sg | _
| j j | t |t j r| j | yt |t
j j j st d| dt | d y)a+ Track the variables (of a layer or model) and other assets.
By default, all variables used by an endpoint function are automatically
tracked when you call `add_endpoint()`. However, non-variables assets
such as lookup tables need to be tracked manually. Note that lookup
tables used by built-in Keras layers (`TextVectorization`,
`IntegerLookup`, `StringLookup`) are automatically tracked by
`add_endpoint()`.
Args:
resource: A layer, model or a TensorFlow trackable resource.
JThe layer provided has not yet been built. It must be built before export._trackedzoInvalid resource type. Expected a Keras `Layer` or `Model` or a TensorFlow `Trackable` object. Received object z
of type 'z'. N)
isinstancer Layerbuilt
ValueErrorr r r! Trackablehasattrr2 append _track_layertype)r' resources r) trackzExportArchive.track s h -hnn 2
h 8 8 B B C4, "
MM * h - h 'Hboo&>&>&H&HI ##+*JtH~6FcK
Jr* c || j v rt d| d t j dk7 r(d|v sd|v r t dt j |t |t j
j j r"|j st d| d |}n t d t | j || | j j | |S t j t | }t | @ |||fi |}|| j" |< t | j || | j j | |S )
a Register a new serving endpoint.
Args:
name: `str`. The name of the endpoint.
fn: A callable. It should only leverage resources
(e.g. `keras.Variable` objects or `tf.lookup.StaticHashTable`
objects) that are available on the models/layers tracked by the
`ExportArchive` (you can call `.track(model)` to track a new
model).
The shape and dtype of the inputs to the function must be
known. For that purpose, you can either 1) make sure that `fn`
is a `tf.function` that has been called at least once, or 2)
provide an `input_signature` argument that specifies the shape
and dtype of the inputs (see below).
input_signature: Optional. Specifies the shape and dtype of `fn`.
Can be a structure of `keras.InputSpec`, `tf.TensorSpec`,
`backend.KerasTensor`, or backend tensor (see below for an
example showing a `Functional` model with 2 input arguments). If
not provided, `fn` must be a `tf.function` that has been called
at least once. Defaults to `None`.
**kwargs: Additional keyword arguments:
- Specific to the JAX backend:
- `is_static`: Optional `bool`. Indicates whether `fn` is
static. Set to `False` if `fn` involves state updates
(e.g., RNG seeds).
- `jax2tf_kwargs`: Optional `dict`. Arguments for
`jax2tf.convert`. See [`jax2tf.convert`](
https://github.com/google/jax/blob/main/jax/experimental/jax2tf/README.md).
If `native_serialization` and `polymorphic_shapes` are
not provided, they are automatically computed.
Returns:
The `tf.function` wrapping `fn` that was added to the archive.
Example:
Adding an endpoint using the `input_signature` argument when the
model has a single input argument:
```python
export_archive = ExportArchive()
export_archive.track(model)
export_archive.add_endpoint(
name="serve",
fn=model.call,
input_signature=[keras.InputSpec(shape=(None, 3), dtype="float32")],
)
```
Adding an endpoint using the `input_signature` argument when the
model has two positional input arguments:
```python
export_archive = ExportArchive()
export_archive.track(model)
export_archive.add_endpoint(
name="serve",
fn=model.call,
input_signature=[
keras.InputSpec(shape=(None, 3), dtype="float32"),
keras.InputSpec(shape=(None, 4), dtype="float32"),
],
)
```
Adding an endpoint using the `input_signature` argument when the
model has one input argument that is a list of 2 tensors (e.g.
a Functional model with 2 inputs):
```python
model = keras.Model(inputs=[x1, x2], outputs=outputs)
export_archive = ExportArchive()
export_archive.track(model)
export_archive.add_endpoint(
name="serve",
fn=model.call,
input_signature=[
[
keras.InputSpec(shape=(None, 3), dtype="float32"),
keras.InputSpec(shape=(None, 4), dtype="float32"),
],
],
)
```
This also works with dictionary inputs:
```python
model = keras.Model(inputs={"x1": x1, "x2": x2}, outputs=outputs)
export_archive = ExportArchive()
export_archive.track(model)
export_archive.add_endpoint(
name="serve",
fn=model.call,
input_signature=[
{
"x1": keras.InputSpec(shape=(None, 3), dtype="float32"),
"x2": keras.InputSpec(shape=(None, 4), dtype="float32"),
},
],
)
```
Adding an endpoint that is a `tf.function`:
```python
@tf.function()
def serving_fn(x):
return model(x)
# The function must be traced, i.e. it must be called at least once.
serving_fn(tf.random.normal(shape=(2, 3)))
export_archive = ExportArchive()
export_archive.track(model)
export_archive.add_endpoint(name="serve", fn=serving_fn)
```
Combining a model with some TensorFlow preprocessing, which can use
TensorFlow resources:
```python
lookup_table = tf.lookup.StaticHashTable(initializer, default_value=0.0)
export_archive = ExportArchive()
model_fn = export_archive.track_and_add_endpoint(
"model_fn",
model,
input_signature=[tf.TensorSpec(shape=(None, 5), dtype=tf.float32)],
)
export_archive.track(lookup_table)
@tf.function()
def serving_fn(x):
x = lookup_table.lookup(x)
return model_fn(x)
export_archive.add_endpoint(name="serve", fn=serving_fn)
```
Endpoint name '' is already taken.r
jax2tf_kwargs is_staticZ'jax2tf_kwargs' and 'is_static' are only supported with the jax backend. Current backend: zThe provided tf.function 'z' has never been called. To specify the expected shape and dtype of the function's arguments, you must either provide a function that has been called at least once, or alternatively pass an `input_signature` argument in `add_endpoint()`.ao If the `fn` argument provided is not a `tf.function`, you must provide an `input_signature` argument to specify the shape and dtype of the function arguments. Example:
export_archive.add_endpoint(
name='call',
fn=model.call,
input_signature=[
keras.InputSpec(
shape=(None, 224, 224, 3),
dtype='float32',
)
],
))r r6 r r3 r types experimentalGenericFunction _list_all_concrete_functionssetattrr# r9 r
map_structurer r add_endpointr )r' namefninput_signaturekwargs decorated_fnr( s r) rJ zExportArchive.add_endpoint sl ^ 4'' ' tf4G HI I ?? % &(K6,A 99@ 9J8KM ""bhh 3 3 C CD668$4RD 9M M "
D&&l ; ' ' - ,,
w+D"oPP *9!!$'""D,7 ##D)r* c || j v rt d| d t |t j st d| dt | d |j s t d t j dk7 r(d|v sd |v r t d
t j t j t | }t t d s0| j | | j ||j |fi |S t! | D |||fi |}|| j$ |< t' | j( || | j j+ | |S ) a Track the variables and register a new serving endpoint.
This function combines the functionality of `track` and `add_endpoint`.
It tracks the variables of the `resource` (either a layer or a model)
and registers a serving endpoint using `resource.__call__`.
Args:
name: `str`. The name of the endpoint.
resource: A trackable Keras resource, such as a layer or model.
input_signature: Optional. Specifies the shape and dtype of `fn`.
Can be a structure of `keras.InputSpec`, `tf.TensorSpec`,
`backend.KerasTensor`, or backend tensor (see below for an
example showing a `Functional` model with 2 input arguments). If
not provided, `fn` must be a `tf.function` that has been called
at least once. Defaults to `None`.
**kwargs: Additional keyword arguments:
- Specific to the JAX backend:
- `is_static`: Optional `bool`. Indicates whether `fn` is
static. Set to `False` if `fn` involves state updates
(e.g., RNG seeds).
- `jax2tf_kwargs`: Optional `dict`. Arguments for
`jax2tf.convert`. See [`jax2tf.convert`](
https://github.com/google/jax/blob/main/jax/experimental/jax2tf/README.md).
If `native_serialization` and `polymorphic_shapes` are
not provided, they are automatically computed.
r? r@ z^Invalid resource type. Expected an instance of a Keras `Layer` or `Model`. Received: resource=z
(of type )r1 r rA rB rC track_and_add_endpoint)r r6 r3 r r4 r; r5 r r rI r r8 BackendExportArchiver= rJ __call__r rR r rH r# r9 )r' rK r< rM rN rO r( s r) rR z$ExportArchive.track_and_add_endpointr sy 8 4'' ' tf4G HI I(FLL1 &&.Zz$x.9IL
~~ 2
?? % &(K6,A 99@ 9J8KM
,,
+-EF JJx $4$$h'' D % %d + D&&l ; ' ' - r* c t |t t t f st dt | d t
d |D s't dt t d |D t j dk( r3t j t j | j | }t | j |t | y)a Register a set of variables to be retrieved after reloading.
Arguments:
name: The string name for the collection.
variables: A tuple/list/set of `keras.Variable` instances.
Example:
```python
export_archive = ExportArchive()
export_archive.track(model)
# Register an endpoint
export_archive.add_endpoint(
name="serve",
fn=model.call,
input_signature=[keras.InputSpec(shape=(None, 3), dtype="float32")],
)
# Save a variable collection
export_archive.add_variable_collection(
name="optimizer_variables", variables=model.optimizer.variables)
export_archive.write_out("path/to/location")
# Reload the object
revived_object = tf.saved_model.load("path/to/location")
# Retrieve the variables
optimizer_variables = revived_object.optimizer_variables
```
zNExpected `variables` to be a list/tuple/set. Received instead object of type 'z'.c 3 p K | ]. }t |t j t j f 0 y wr, )r3 r Variabler .0vs r) z8ExportArchive.add_variable_collection.. s+
?@Jq2;;(8(89 :
s 46zgExpected all elements in `variables` to be `tf.Variable` instances. Found instead the following types: c 3 2 K | ] }t | y wr, )r; rX s r) r[ z8ExportArchive.add_variable_collection.. s 7DG7s r N)r3 listtuplesetr6 r; allr r flattenrI _convert_to_tf_variablerH r# )r' rK r$ s r) add_variable_collectionz%ExportArchive.add_variable_collection s : )dE3%78 448O3DBH
DM
O7Y7789;
?? % ""4#?#?KI ""D$y/:r* c j s t d j i } j D ] } j | ||< d j vr! j j d |d< t j
j
j ||| rAdj fd j D }t j d| d| y y )
a" Write the corresponding SavedModel to disk.
Arguments:
filepath: `str` or `pathlib.Path` object.
Path where to save the artifact.
options: `tf.saved_model.SaveOptions` object that specifies
SavedModel saving options.
verbose: whether to print all the variables of an
exported SavedModel.
**Note on TF-Serving**: all endpoints registered via `add_endpoint()`
are made visible for TF-Serving in the SavedModel artifact. In addition,
the first endpoint registered is made visible under the alias
`"serving_default"` (unless an endpoint with the name
`"serving_default"` was already registered manually),
since TF-Serving requires this endpoint to be set.
z4No endpoints have been set yet. Call add_endpoint().serving_defaultr )options
signaturesz
c 3 b K | ]& }t t j | | ( yw)verboseN)_print_signaturegetattrr# )rY rK r' rj s r) r[ z*ExportArchive.write_out.. s9 $ !D . .5tW $s ,/zSaved artifact at 'z+'. The following endpoints are available:
N) r r6 _filter_and_track_resources_get_concrete_fnr saved_modelsaver# joinr print_msg)r' filepathrf rj rg rK endpointss ` ` r) write_outzExportArchive.write_out s
$ ## F
((*
(( ;D # 4 4T :Jt ; D$8$8 8,0,A,A$$Q'-J( ) !
$ ! 0 0 $ I
%hZ 0= +
r* c t |t j st d| dt | d t j |j |j |j |j S )NzL`backend_variable` must be a `backend.Variable`. Recevied: backend_variable=z
of type (rQ )dtype trainablerK )
r3 r rW TypeErrorr; r valuerw rx rK )r' backend_variables r) rb z%ExportArchive._convert_to_tf_variable s{ *G,<,<=..>-? @)*+1.
{{ " ""((&00!&&
r* c || j v rt | j | S t | j | j d }t |j d S )z&Workaround for some SavedModel quirks.ro r )r rl r# _trackable_childrenr] values)r' endpointtracess r) rn z ExportArchive._get_concrete_fn, s[ t00 04--x8 8 T / /:NN F
(+ +r* c r | j D cg c] }| j | }}t | S c c}w r, )r rn _list_variables_used_by_fns)r' rK fnss r) _get_variables_used_by_endpointsz.ExportArchive._get_variables_used_by_endpoints6 s7 7;7K7KLtt$$T*LL*3// Ms 4c | j D cg c] }| j | }}t | \ }}t ||z | j _ g | j _ ddlm} t | d rx| j D ]h }t j j | j }|D ]4 }t || s| j j j! | 6 j yyc c}w )zBTrack resources used by endpoints / referenced in `track()` calls.r )TrackableResourcer2 N)r rn r r] r# _all_variables _misc_assets#tensorflow.saved_model.experimentalr r8 r2 r train
TrackableView descendantsr3 r9 ) r' rK r tvsntvsr rootr trackables r) rm z)ExportArchive._filter_and_track_resources: s 8<7K7KLtt$$T*LL/4 T,0t,<) +-'I 4 $
J hh 4 4T : F F H !, JI!)->? **77>>yIJ
J % Ms C0r, )NT)__name__
__module__ __qualname____doc__r propertyr$ r% r& r= rJ rR rc ru rb rn r rm
__classcell__)r( s @r) r r ) s ?B8" , , 6 6 : :"HC JB H0;d4l
,0Jr* r Nc |d}t }| t | } |j t | |fi | |j || y)a
Export the model as a TensorFlow SavedModel artifact for inference.
This method lets you export a model to a lightweight SavedModel artifact
that contains the model's forward pass only (its `call()` method)
and can be served via e.g. TensorFlow Serving. The forward pass is
registered under the name `serve()` (see example below).
The original code of the model (including any custom layers you may
have used) is *no longer* necessary to reload the artifact -- it is
entirely standalone.
Args:
filepath: `str` or `pathlib.Path` object. The path to save the artifact.
verbose: `bool`. Whether to print a message during export. Defaults to
`None`, which uses the default value set by different backends and
formats.
input_signature: Optional. Specifies the shape and dtype of the model
inputs. Can be a structure of `keras.InputSpec`, `tf.TensorSpec`,
`backend.KerasTensor`, or backend tensor. If not provided, it will
be automatically computed. Defaults to `None`.
**kwargs: Additional keyword arguments:
- Specific to the JAX backend:
- `is_static`: Optional `bool`. Indicates whether `fn` is
static. Set to `False` if `fn` involves state updates
(e.g., RNG seeds).
- `jax2tf_kwargs`: Optional `dict`. Arguments for
`jax2tf.convert`. See [`jax2tf.convert`](
https://github.com/google/jax/blob/main/jax/experimental/jax2tf/README.md).
If `native_serialization` and `polymorphic_shapes` are not
provided, they are automatically computed.
**Note:** This feature is currently supported only with TensorFlow, JAX and
Torch backends. Support for the Torch backend is experimental.
**Note:** The dynamic shape feature is not yet supported with Torch
backend. As a result, you must fully define the shapes of the inputs using
`input_signature`. If `input_signature` is not provided, all instances of
`None` (such as the batch size) will be replaced with `1`.
Example:
```python
# Export the model as a TensorFlow SavedModel artifact
model.export("path/to/location", format="tf_saved_model")
# Load the artifact in a different process/environment
reloaded_artifact = tf.saved_model.load("path/to/location")
predictions = reloaded_artifact.serve(input_data)
```
If you would like to customize your serving endpoints, you can
use the lower-level `keras.export.ExportArchive` class. The
`export()` method relies on `ExportArchive` internally.
NTri )r r rR DEFAULT_ENDPOINT_NAMEru )modelrs rj rM rN export_archives r) export_saved_modelr N sZ r "_N -e4)N)) uo9? Xw 7r* c | j d }|j | }|j d }d| dg|dd z }dj | }|S )Nr ri
z * Endpoint '' )rG pretty_printed_signaturesplitrq )rL rK rj concrete_fnpprinted_signaturelinesr s r) rk rk sh 113A6K$==g=N $ $T *ED6
# $uQRy 0Eyy H Or* c g }g }t }t }| D ] }t |d r
|j }n!t |d r|j g}n|g}|D ] }|j D ]; }t | |vs|j
| |j t | = |j D ]I }t | |vst | |vs |j
| |j t | K ||fS )Nconcrete_functionsget_concrete_function) r_ r8 r r r% idr9 addr$ ) r r% r& trainable_variables_idsnon_trainable_variables_idsrL r r rZ s r) r r s !e"%% ; 2+ ,!#!6!6
R0
1"$":":"r s ? - = = $ 97?? $ W__% W__'! W__'! W__*$
OGOO%&&EF
*+aJ( aJ ,aJJ 48B8J8r*