2Vh5jddlZddlZddlmZddlmZddlmZddlm Z ddlm Z ddl m Z ddl mZdd lmZdd lmZdd lmZed Gd de ZedGddeZy)N)backend)tree) keras_export)is_float_dtype)standardize_dtype)Layer)serialization_lib) jax_utils)tracking)jaxzkeras.layers.JaxLayerceZdZdZ d fd ZdZejdZdZ dZ dZ d dZ fd Z efd ZxZS) JaxLayera Keras Layer that wraps a JAX model. This layer enables the use of JAX components within Keras when using JAX as the backend for Keras. ## Model function This layer accepts JAX models in the form of a function, `call_fn`, which must take the following arguments with these exact names: - `params`: trainable parameters of the model. - `state` (*optional*): non-trainable state of the model. Can be omitted if the model has no non-trainable state. - `rng` (*optional*): a `jax.random.PRNGKey` instance. Can be omitted if the model does not need RNGs, neither during training nor during inference. - `inputs`: inputs to the model, a JAX array or a `PyTree` of arrays. - `training` (*optional*): an argument specifying if we're in training mode or inference mode, `True` is passed in training mode. Can be omitted if the model behaves the same in training mode and inference mode. The `inputs` argument is mandatory. Inputs to the model must be provided via a single argument. If the JAX model takes multiple inputs as separate arguments, they must be combined into a single structure, for instance in a `tuple` or a `dict`. ## Model weights initialization The initialization of the `params` and `state` of the model can be handled by this layer, in which case the `init_fn` argument must be provided. This allows the model to be initialized dynamically with the right shape. Alternatively, and if the shape is known, the `params` argument and optionally the `state` argument can be used to create an already initialized model. The `init_fn` function, if provided, must take the following arguments with these exact names: - `rng`: a `jax.random.PRNGKey` instance. - `inputs`: a JAX array or a `PyTree` of arrays with placeholder values to provide the shape of the inputs. - `training` (*optional*): an argument specifying if we're in training mode or inference mode. `True` is always passed to `init_fn`. Can be omitted regardless of whether `call_fn` has a `training` argument. ## Models with non-trainable state For JAX models that have non-trainable state: - `call_fn` must have a `state` argument - `call_fn` must return a `tuple` containing the outputs of the model and the new non-trainable state of the model - `init_fn` must return a `tuple` containing the initial trainable params of the model and the initial non-trainable state of the model. This code shows a possible combination of `call_fn` and `init_fn` signatures for a model with non-trainable state. In this example, the model has a `training` argument and an `rng` argument in `call_fn`. ```python def stateful_call(params, state, rng, inputs, training): outputs = ... new_state = ... return outputs, new_state def stateful_init(rng, inputs): initial_params = ... initial_state = ... return initial_params, initial_state ``` ## Models without non-trainable state For JAX models with no non-trainable state: - `call_fn` must not have a `state` argument - `call_fn` must return only the outputs of the model - `init_fn` must return only the initial trainable params of the model. This code shows a possible combination of `call_fn` and `init_fn` signatures for a model without non-trainable state. In this example, the model does not have a `training` argument and does not have an `rng` argument in `call_fn`. ```python def stateless_call(params, inputs): outputs = ... return outputs def stateless_init(rng, inputs): initial_params = ... return initial_params ``` ## Conforming to the required signature If a model has a different signature than the one required by `JaxLayer`, one can easily write a wrapper method to adapt the arguments. This example shows a model that has multiple inputs as separate arguments, expects multiple RNGs in a `dict`, and has a `deterministic` argument with the opposite meaning of `training`. To conform, the inputs are combined in a single structure using a `tuple`, the RNG is split and used the populate the expected `dict`, and the Boolean flag is negated: ```python def my_model_fn(params, rngs, input1, input2, deterministic): ... if not deterministic: dropout_rng = rngs["dropout"] keep = jax.random.bernoulli(dropout_rng, dropout_rate, x.shape) x = jax.numpy.where(keep, x / dropout_rate, 0) ... ... return outputs def my_model_wrapper_fn(params, rng, inputs, training): input1, input2 = inputs rng1, rng2 = jax.random.split(rng) rngs = {"dropout": rng1, "preprocessing": rng2} deterministic = not training return my_model_fn(params, rngs, input1, input2, deterministic) keras_layer = JaxLayer(my_model_wrapper_fn, params=initial_params) ``` ## Usage with Haiku modules `JaxLayer` enables the use of [Haiku](https://dm-haiku.readthedocs.io) components in the form of [`haiku.Module`](https://dm-haiku.readthedocs.io/en/latest/api.html#module). This is achieved by transforming the module per the Haiku pattern and then passing `module.apply` in the `call_fn` parameter and `module.init` in the `init_fn` parameter if needed. If the model has non-trainable state, it should be transformed with [`haiku.transform_with_state`]( https://dm-haiku.readthedocs.io/en/latest/api.html#haiku.transform_with_state). If the model has no non-trainable state, it should be transformed with [`haiku.transform`]( https://dm-haiku.readthedocs.io/en/latest/api.html#haiku.transform). Additionally, and optionally, if the module does not use RNGs in "apply", it can be transformed with [`haiku.without_apply_rng`]( https://dm-haiku.readthedocs.io/en/latest/api.html#without-apply-rng). The following example shows how to create a `JaxLayer` from a Haiku module that uses random number generators via `hk.next_rng_key()` and takes a training positional argument: ```python class MyHaikuModule(hk.Module): def __call__(self, x, training): x = hk.Conv2D(32, (3, 3))(x) x = jax.nn.relu(x) x = hk.AvgPool((1, 2, 2, 1), (1, 2, 2, 1), "VALID")(x) x = hk.Flatten()(x) x = hk.Linear(200)(x) if training: x = hk.dropout(rng=hk.next_rng_key(), rate=0.3, x=x) x = jax.nn.relu(x) x = hk.Linear(10)(x) x = jax.nn.softmax(x) return x def my_haiku_module_fn(inputs, training): module = MyHaikuModule() return module(inputs, training) transformed_module = hk.transform(my_haiku_module_fn) keras_layer = JaxLayer( call_fn=transformed_module.apply, init_fn=transformed_module.init, ) ``` Args: call_fn: The function to call the model. See description above for the list of arguments it takes and the outputs it returns. init_fn: the function to call to initialize the model. See description above for the list of arguments it takes and the outputs it returns. If `None`, then `params` and/or `state` must be provided. params: A `PyTree` containing all the model trainable parameters. This allows passing trained parameters or controlling the initialization. If both `params` and `state` are `None`, `init_fn` is called at build time to initialize the trainable parameters of the model. state: A `PyTree` containing all the model non-trainable state. This allows passing learned state or controlling the initialization. If both `params` and `state` are `None`, and `call_fn` takes a `state` argument, then `init_fn` is called at build time to initialize the non-trainable state of the model. seed: Seed for random number generator. Optional. dtype: The dtype of the layer's computations and weights. Can also be a `keras.DTypePolicy`. Optional. Defaults to the default policy. c dtjdk7r tdtj|| | tdt| d i|||_||_tj j||_|j|d|_ |j|d|_ |j |j|j|j|dhdd h|_d |j v|_|r|j|d hd d h|_yy)Nr zBJaxLayer is only supported with the JAX backend. Current backend: z5`init_fn`, `params` and `state` cannot all be `None`.T trainableFcall_fn>rngstateinputsparamstrainingrrinit_fn>rrr)r ValueErrorsuper__init__rrrandom SeedGeneratorseed_generator_create_variablestracked_params tracked_staterr_build_at_init_validate_signaturecall_fn_arguments has_stateinit_fn_arguments)selfrrrrseedkwargs __class__s I/home/dcms/DCMS/lib/python3.12/site-packages/keras/src/utils/jax_layer.pyrzJaxLayer.__init__s9 ??  %#OO-.0  ?v~%-G  "6"  %nn::4@"44Vt4L!33EU3K ;; "djj&<    !!%!9!9   < J "  !D$:$:: %)%=%=$AH:&D " c Ztj|j}|D]}||vstd|d|dg}|j D]Y}|j |vr.td|d|j ddj |d|j|j [|S)NzMissing required argument in `z`: ``zUnsupported argument in `z`, supported arguments are `z`, `)inspect signature parametersrvaluesnamejoinappend) r(fnfn_nameallowedrequired fn_parametersparameter_nameparameter_names parameters r,r$zJaxLayer._validate_signatures))"-88 & N]2 4WI>&'q* &--/ 3I~~W, /yY^^> 2  3r-cfd}tjj||}r|_n|_tjj |\}}|S)aCreate a structure of variables from a structure of JAX arrays. `values` is traversed via JAX's `tree_map`. When a leaf is a JAX array or a tensor-like object, a corresponding variable is created with it as the initial value. The resulting structure of variables is assigned to `self.params` or `self.state` depending on `trainable`. Then, a flattened version of the variables is returned for tracking. `self.params` or `self.state` are intentionally not tracked because structures like `TrackedList` interfere with `jax.tree_utils`. Note that leaf objects that are not JAX arrays and not tensor-like are left intact as they are assumed to be configuration used by the model. Args: values: the structure of values to traverse. trainable: whether to create trainable variables. Returns: flat list of variables initialized with `values` for tracking. ctj|s*t|tjtj fr8|j }t|rd}j|j||St|tttfrItt|}t|rd}jdtj||S|S)N) initializerdtyperr)r is_tensor isinstancenpndarraygenericrBr add_weightshapeboolintfloatrtypeconvert_to_tensor)valuerBr(rs r,create_variablez3JaxLayer._create_variables..create_variable)s  ': BJJ/, !%( EKK %' ' ED#u#56)$u+6!%( E ' 9 9% @' ' r-)r tree_utiltree_maprr tree_flatten)r(r3rrP variablesflat_variables_s` ` r,r zJaxLayer._create_variablessP, 6MM**?FC #DK"DJMM66yAr-c6|jjS)a Returns a JAX `PRNGKey` or structure of `PRNGKey`s to pass to `init_fn`. By default, this returns a single `PRNGKey` retrieved by calling `self.seed_generator.next()`. Override this to return a different structure. Returns: a JAX `PRNGKey` or structure of `PRNGKey`s that will be passed as the `rng` argument of `init_fn`. rnextr(s r, _get_init_rngzJaxLayer._get_init_rngNs""''))r-c<|r|jjSy)a Returns a JAX `PRNGKey` or structure of `PRNGKey`s to pass to `call_fn`. By default, this returns a single `PRNGKey` retrieved by calling `self.seed_generator.next()` when `training` is `True`, and `None` when `training` is `False`. Override this to return a different structure or to pass RNGs in inference mode too. Returns: a JAX `PRNGKey` or structure of `PRNGKey`s that will be passed as the `rng` argument of `call_fn`. NrXr(rs r, _get_call_rngzJaxLayer._get_call_rng\s &&++- -r-c|j |jytjr t dd}t j ||}g}|jD]U}|dk(r |j|j(|dk(r|j|?|dk(sE|jdW|j|}|jr|\}}n|d}}|j|d|_ |j|d|_y) Nz+'JaxLayer' cannot be built in tracing scopecp|Dcgc]}||nd }}tjj|Scc}w)N)r numpyones)rIds r, create_inputz$JaxLayer.build..create_inputxs68=>1!-QQ.>E>99>>%( (?s 3rrrTrF)rrr is_in_jax_tracing_scoperrmap_shape_structurer'r6r[rr&r r!r") r( input_shapere init_inputs init_args argument_name init_result init_params init_states r,buildzJaxLayer.buildns ;; "djj&<   , , .JK K )..|[I  !33 'M%  !3!3!56(*  -*,  &  '#dllI. >>&1 #K&14K"44 45 "33J%3Pr-cd}g}|jD]}|dk(r:|jtjj ||j B|dk(r:|jtjj ||j |dk(r!|j|j||dk(r|j||dk(s|j|d}|jr?|j|\}}tjj |||j |S|j|S)Nc"|dS|jSN)rO)variables r,unwrap_variablez&JaxLayer.call..unwrap_variables#+4 ? ?r-rrrrrcTt|ds td|j|y)NassignzStructure mismatch: the structure of the state returned by `call` does not match the structure of the state at initialization time.)hasattrrrv)rOrss r,assign_state_to_variablez/JaxLayer.call..assign_state_to_variables,8X. + OOE "r-) r%r6r rQrRrrr^r&r) r(rrrt call_argsrkrx predictions new_states r,callz JaxLayer.calls* @ !33 +M(  MM**?DKKH')  MM**?DJJG%'  !3!3H!=>(*  (*,  * +  # >>%1T\\9%= "K MM " "()TZZ  4<<+ +r-ctj|jtj|jd}t|}t t|jt|jzS)N)rr) r serialize_keras_objectrrr get_configdictlistitems)r(config base_configr+s r,rzJaxLayer.get_configsf(?? M(?? M g(* D**,-V\\^0DDEEr-ctj|d}tj|d}||d<||d<t| |S)Nrr)r deserialize_keras_objectr from_config)clsrrrr+s r,rzJaxLayer.from_configsQ#<.apply_with_training:sB;;$$33FEB{{%! % r-cxjjj||||jS)N)rrrr)rrrrrr(s r,apply_without_trainingz2FlaxLayer.__init__..apply_without_trainingDs?;;$$33FEB{{% % r-ctjjj||j|S)N)rr_variables_to_params_and_staterinitr)rrrr(s r,init_with_trainingz.FlaxLayer.__init__..init_with_trainingMs=66   ;;% ! r-crjjj||jS)N)rr)rrr(s r,init_without_trainingz1FlaxLayer.__init__..init_without_trainingWs:66   ;;! r-r)rrrrr) flax.corerrrrrDenyListr0r1__call__r2rrr)r(rrrTr* flax_scoperrrrrrrrrr+s` @r,rzFlaxLayer.__init__%s 2 ??  %#OO-.0    "++XJ7        !:6??;FF G 34FWG57LWG;;IF       r-c&|r |ri||S|S|r|SiSrrr)r(rrs r,rz(FlaxLayer._params_and_state_to_variablesrs* *&*E** L r-c|yd|vri|fSt|dk(r|ifSd|di}|jDcic]\}}|dk7s ||}}}||fScc}}w)NNNrra)lenr)r(rTrkvrs r,rz(FlaxLayer._variables_to_params_and_state|sz   9 $y= y>Q b= Ih/0"+//"3E$!QqH}AEEu}Fs AAcl|jj|jjdS)N)rdropoutrXrZs r,r[zFlaxLayer._get_init_rngs0))..0**//1  r-cB|rd|jjiSiS)NrrXr]s r,r^zFlaxLayer._get_call_rngs$ t22779: :Ir-c|j}t|jdr9|jj|jk(r|jj}t j |jt j |d}t|!}|jd|jdtt|jt|jzS)N__self__)rrrr) rrwrrrr r~rrpoprrr)r( config_methodrrr+s r,rzFlaxLayer.get_configs DKK , $$ 3!KK00M'>>t{{K'>>}M g(*  " "D**,-V\\^0DDEEr-ctj|d}tj|d}t|dtr t ||}||d<||d<|di|S)Nrrr)r rrDstrgetattr)rrrrs r,rzFlaxLayer.from_configsg";;F8rsr-=@(.%$,%&u+uu+'u+p &'gg(gr-