AVhdZddlZddlZddlZddlZddlmZddlm Z ddlm Z ddlm Z ddlm Z ddl mZdd l mZdd lmZdd lmZdd lmZdd lmZddlmZgdZgdZGddej4ZdZdZd$dZGddej4ZedgGddZ e dZ!edgjEe#de dZ$ed gjEe#ded!gGd"d#eeZ%y)%z+Base classes for probability distributions.N)context)dtypes)ops) tensor_shape) tensor_util) array_ops)math_ops)kullback_leibler)util) deprecation) tf_inspect) tf_export)ReparameterizationTypeFULLY_REPARAMETERIZEDNOT_REPARAMETERIZED Distribution) batch_shapebatch_shape_tensorcdf covariance cross_entropyentropy event_shapeevent_shape_tensor kl_divergencelog_cdflog_problog_survival_functionmeanmodeprobsamplestddevsurvival_functionvarianceceZdZdZy)_BaseDistributionzz$_update_docstring..osLTDLszargs:Nz )splitjoin enumeratestriplower)old_str append_str old_str_linesixrE has_args_ix final_args_ixs r._update_docstringrShs Mr'--%-yyLZ5E5Ed5KLL*#=1* R     ( *+*OM IImN]3 4 ! "$* +ii mn56 78 V j ((*s -B>ctjrI|G|js |jr/t j ||}|j jr|St j |||S)zDConverts to tensor avoiding an eager bug that loses float precision.r3r3preferred_dtype)rexecuting_eagerly is_integeris_boolrconvert_to_tensordtype is_floating)valuer3rWvs r._convert_to_tensorr`~sd!o&A!!_%<%< e$/Aww h   $ 99r-ceZdZdZy)_DistributionMetac6|s td|Dcgc]}|tk(st|tr|}}|d}|tk(r"tj j ||||St|tstd|d|jdtD]}d|z}|j|d}||vrt||d} | std|jd |d|j|d} | _tj| } | swt| }tj| } | td |jd |t!|j"d |d | |_|||<tj j ||||Scc}w)aqControl the creation of subclasses of the Distribution class. The main purpose of this method is to properly propagate docstrings from private Distribution methods, like `_log_prob`, into their public wrappers as inherited by the Distribution base class (e.g. `log_prob`). Args: classname: The name of the subclass being created. baseclasses: A tuple of parent classes. attrs: A dict mapping new attributes to their values. Returns: The class object. Raises: TypeError: If `Distribution` is not a subclass of `BaseDistribution`, or the new class is derived via multiple inheritance and the first parent class is not a subclass of `BaseDistribution`. AttributeError: If `Distribution` does not implement e.g. `log_prob`. ValueError: If a `Distribution` public method lacks a docstring. zOExpected non-empty baseclass. Does Distribution not subclass _BaseDistribution?rz First parent class declared for z must be Distribution, but saw ''z_%sNz%Internal error: expected base class 'z' to implement method 'z/Expected base class fn to contain a docstring: .zAdditional documentation from `z`: )r7r' issubclassrabcABCMeta__new__r($_DISTRIBUTION_PUBLIC_METHOD_WRAPPERSgetgetattrAttributeErrorr getdocr? ValueErrorrSr+) mcs classname baseclassesattrsbase which_baseattr special_attrclass_attr_valuebase_attr_valueclass_special_attr_valueclass_special_attr_docstringclass_attr_docstrings r.riz_DistributionMeta.__new__s.  8 99%H $ $ 4(F HJH a=D   [[ ie DD dL ) 6?P QQ4%T\l4. dD1o }}d $% %"'<!> ! )%/%6%67O%P" )!/2'..?  %}}d $% %"3  " "5 7"9%eDk;%> ;;  sI{E BBQHs"FN)r(r)r*rir,r-r.rbrbs BCr-rbz$distributions.ReparameterizationType)v1cReZdZdZej ddddZdZdZy ) raInstances of this class represent how sampling is reparameterized. Two static instances exist in the distributions library, signifying one of two possible properties for samples from a distribution: `FULLY_REPARAMETERIZED`: Samples from the distribution are fully reparameterized, and straight-through gradients are supported. `NOT_REPARAMETERIZED`: Samples from the distribution are not fully reparameterized, and straight-through gradients are either partially unsupported or are not supported at all. In this case, for purposes of e.g. RL or variational inference, it is generally safest to wrap the sample results in a `stop_gradients` call and use policy gradients / surrogate loss instead. 2019-01-01The TensorFlow Distributions library has moved to TensorFlow Probability (https://github.com/tensorflow/probability). You should update all references to use `tfp.distributions` instead of `tf.distributions`.T warn_oncec||_yN _rep_type)selfrep_types r.__init__zReparameterizationType.__init__s DNr-c d|jzS)Nzrrs r.__repr__zReparameterizationType.__repr__s *T^^ ;;r-c ||uS)a+Determine if this `ReparameterizationType` is equal to another. Since ReparameterizationType instances are constant static global instances, equality checks if two instances' id() values are equal. Args: other: Object to compare against. Returns: `self is other`. r,rothers r.__eq__zReparameterizationType.__eq__s 5=r-N) r(r)r*r+r deprecatedrrrr,r-r.rrs? ;' < r-rrz#distributions.FULLY_REPARAMETERIZEDrz!distributions.NOT_REPARAMETERIZEDzdistributions.DistributionceZdZdZej ddd dLdZedZejd Ze dMd Z e d Z e d Zed ZedZedZedZedZedZdZdZdNdZdZedZdZdOdZdZedZdPdZdQdZdRdZ dZ!dSd Z"d!Z#d"Z$dTd#Z%d$Z&d%Z'dUd&Z(d'Z)d(Z*dVd)Z+d*Z,d+Z-dWd,Z.d-Z/d.Z0dXd/Z1d0Z2d1Z3dYd2Z4d3Z5dZd4Z6d5Z7d[d6Z8d7Z9d8Z:d\d9Z;d:Zd^d=Z?d>Z@d_d?ZAd@ZBd`dAZCdBZDdadCZEdDZFdbdEZGdFZHdGZIeJjdcdHZLdIZMdJZNdKZOy)draA generic probability distribution base class. `Distribution` is a base class for constructing and organizing properties (e.g., mean, variance) of random variables (e.g, Bernoulli, Gaussian). #### Subclassing Subclasses are expected to implement a leading-underscore version of the same-named function. The argument signature should be identical except for the omission of `name="..."`. For example, to enable `log_prob(value, name="log_prob")` a subclass should implement `_log_prob(value)`. Subclasses can append to public-level docstrings by providing docstrings for their method specializations. For example: ```python @util.AppendDocstring("Some other details.") def _log_prob(self, value): ... ``` would add the string "Some other details." to the `log_prob` function docstring. This is implemented as a simple decorator to avoid python linter complaining about missing Args/Returns/Raises sections in the partial docstrings. #### Broadcasting, batching, and shapes All distributions support batches of independent distributions of that type. The batch shape is determined by broadcasting together the parameters. The shape of arguments to `__init__`, `cdf`, `log_cdf`, `prob`, and `log_prob` reflect this broadcasting, as does the return value of `sample` and `sample_n`. `sample_n_shape = [n] + batch_shape + event_shape`, where `sample_n_shape` is the shape of the `Tensor` returned from `sample_n`, `n` is the number of samples, `batch_shape` defines how many independent distributions there are, and `event_shape` defines the shape of samples from each of those independent distributions. Samples are independent along the `batch_shape` dimensions, but not necessarily so along the `event_shape` dimensions (depending on the particulars of the underlying distribution). Using the `Uniform` distribution as an example: ```python minval = 3.0 maxval = [[4.0, 6.0], [10.0, 12.0]] # Broadcasting: # This instance represents 4 Uniform distributions. Each has a lower bound at # 3.0 as the `minval` parameter was broadcasted to match `maxval`'s shape. u = Uniform(minval, maxval) # `event_shape` is `TensorShape([])`. event_shape = u.event_shape # `event_shape_t` is a `Tensor` which will evaluate to []. event_shape_t = u.event_shape_tensor() # Sampling returns a sample per distribution. `samples` has shape # [5, 2, 2], which is [n] + batch_shape + event_shape, where n=5, # batch_shape=[2, 2], and event_shape=[]. samples = u.sample_n(5) # The broadcasting holds across methods. Here we use `cdf` as an example. The # same holds for `log_cdf` and the likelihood functions. # `cum_prob` has shape [2, 2] as the `value` argument was broadcasted to the # shape of the `Uniform` instance. cum_prob_broadcast = u.cdf(4.0) # `cum_prob`'s shape is [2, 2], one per distribution. No broadcasting # occurred. cum_prob_per_dist = u.cdf([[4.0, 5.0], [6.0, 7.0]]) # INVALID as the `value` argument is not broadcastable to the distribution's # shape. cum_prob_invalid = u.cdf([4.0, 5.0, 6.0]) ``` #### Shapes There are three important concepts associated with TensorFlow Distributions shapes: - Event shape describes the shape of a single draw from the distribution; it may be dependent across dimensions. For scalar distributions, the event shape is `[]`. For a 5-dimensional MultivariateNormal, the event shape is `[5]`. - Batch shape describes independent, not identically distributed draws, aka a "collection" or "bunch" of distributions. - Sample shape describes independent, identically distributed draws of batches from the distribution family. The event shape and the batch shape are properties of a Distribution object, whereas the sample shape is associated with a specific call to `sample` or `log_prob`. For detailed usage examples of TensorFlow Distributions shapes, see [this tutorial]( https://github.com/tensorflow/probability/blob/master/tensorflow_probability/examples/jupyter_notebooks/Understanding_TensorFlow_Distributions_Shapes.ipynb) #### Parameter values leading to undefined statistics or distributions. Some distributions do not have well-defined statistics for all initialization parameter values. For example, the beta distribution is parameterized by positive real numbers `concentration1` and `concentration0`, and does not have well-defined mode if `concentration1 < 1` or `concentration0 < 1`. The user is given the option of raising an exception or returning `NaN`. ```python a = tf.exp(tf.matmul(logits, weights_a)) b = tf.exp(tf.matmul(logits, weights_b)) # Will raise exception if ANY batch member has a < 1 or b < 1. dist = distributions.beta(a, b, allow_nan_stats=False) mode = dist.mode().eval() # Will return NaN for batch members with either a < 1 or b < 1. dist = distributions.beta(a, b, allow_nan_stats=True) # Default behavior mode = dist.mode().eval() ``` In all cases, an exception is raised if *invalid* parameters are passed, e.g. ```python # Will raise an exception if any Op is run. negative_a = -1.0 * a # beta distribution by definition has a > 0. dist = distributions.beta(negative_a, b, allow_nan_stats=True) dist.mean().eval() ``` rrTrNc|gn|}t|D],\}} | tj| rtd|| fz|r|ddk7r8|xst |j } t j| 5} ddd||_||_ ||_ ||_ |xsi|_ ||_ ||_y#1swY?xYw)aConstructs the `Distribution`. **This is a private method for subclass use.** Args: dtype: The type of the event samples. `None` implies no type-enforcement. reparameterization_type: Instance of `ReparameterizationType`. If `distributions.FULLY_REPARAMETERIZED`, this `Distribution` can be reparameterized in terms of some standard distribution with a function whose Jacobian is constant for the support of the standard distribution. If `distributions.NOT_REPARAMETERIZED`, then no such reparameterization is available. validate_args: Python `bool`, default `False`. When `True` distribution parameters are checked for validity despite possibly degrading runtime performance. When `False` invalid inputs may silently render incorrect outputs. allow_nan_stats: Python `bool`, default `True`. When `True`, statistics (e.g., mean, mode, variance) use the value "`NaN`" to indicate the result is undefined. When `False`, an exception is raised if one or more of the statistic's batch members are undefined. parameters: Python `dict` of parameters used to instantiate this `Distribution`. graph_parents: Python `list` of graph prerequisites of this `Distribution`. name: Python `str` name prefixed to Ops created by this class. Default: subclass name. Raises: ValueError: if any member of graph_parents is `None` or not a `Tensor`. Nz)Graph parent item %d is not a Tensor; %s.rG/)rJr is_tf_typerotyper(r name_scope_dtype_reparameterization_type_allow_nan_stats_validate_args _parameters_graph_parents_name) rr\reparameterization_type validate_argsallow_nan_stats parameters graph_parentsr3itnon_unique_names r.rzDistribution.__init__s\(/B]M-(O1 +003D1vMNNO 48s?3T 3 3o >>/ * d  DK$;D!+D'D!'RD'DDJ  s 9B99Cc|jSr_parameter_dictrs r.rzDistribution._parameterss   r-c d|vr|d=||_y)a1Intercept assignments to self._parameters to avoid reference cycles. Parameters are often created using locals(), so we need to clean out any references to `self` before assigning it to an attribute. Args: value: A dictionary of parameters to assign to the `_parameters` property. rNrrr^s r.rzDistribution._parameterss - Dr-ctj||g5|j|cdddS#1swYyxYw)a,Shapes of parameters given the desired shape of a call to `sample()`. This is a class method that describes what key/value arguments are required to instantiate the given `Distribution` so that a particular shape is returned for that instance's call to `sample()`. Subclasses should override class method `_param_shapes`. Args: sample_shape: `Tensor` or python list/tuple. Desired shape of a call to `sample()`. name: name to prepend ops with. Returns: `dict` of parameter name to `Tensor` shapes. valuesN)rr _param_shapes)cls sample_shaper3s r. param_shapeszDistribution.param_shapess7$ l^ 4-   | ,---s4=cZt|tjr+|js t d|j }|j |}i}|jD]?\}}tj|}| t dtj|||<A|S)aparam_shapes with static (i.e. `TensorShape`) shapes. This is a class method that describes what key/value arguments are required to instantiate the given `Distribution` so that a particular shape is returned for that instance's call to `sample()`. Assumes that the sample's shape is known statically. Subclasses should override class method `_param_shapes` to return constant-valued tensors when constant values are fed. Args: sample_shape: `TensorShape` or python list/tuple. Desired shape of a call to `sample()`. Returns: `dict` of parameter name to `TensorShape`. Raises: ValueError: if `sample_shape` is a `TensorShape` and is not fully defined. z.TensorShape sample_shape must be fully definedz>sample_shape must be a fully-defined TensorShape or list/tuple) isinstancer TensorShapeis_fully_definedroas_listritemsrconstant_value)rrparams static_paramsr3shape static_shapes r.param_static_shapesz Distribution.param_static_shapess,, 8 89  * * ,IJJ!))+l   l +FM||~C e //6l   LN N(44\BmD C r-ctd)Nz_param_shapes not implemented)NotImplementedError)rs r.rzDistribution._param_shapes&s = >>r-c|jS)z9Name prepended to all ops created by this `Distribution`.)rrs r.r3zDistribution.name*s ::r-c|jS)z8The `DType` of `Tensor`s handled by this `Distribution`.)rrs r.r\zDistribution.dtype/s ;;r-c|jjDcic]\}}|jds|dk7r|| c}}Scc}}w)zADictionary of parameters used to instantiate this `Distribution`.__r)rr startswith)rkr_s r.rzDistribution.parameters4sL "--335 7TQ<<%!v+ qD 77 7s#Ac|jS)a Describes how samples from the distribution are reparameterized. Currently this is one of the static instances `distributions.FULLY_REPARAMETERIZED` or `distributions.NOT_REPARAMETERIZED`. Returns: An instance of `ReparameterizationType`. )rrs r.rz$Distribution.reparameterization_type=s  ( ((r-c|jS)aPython `bool` describing behavior when a stat is undefined. Stats return +/- infinity when it makes sense. E.g., the variance of a Cauchy distribution is infinity. However, sometimes the statistic is undefined, e.g., if a distribution's pdf does not achieve a maximum within the support of the distribution, the mode is undefined. If the mean is undefined, then by definition the variance is undefined. E.g. the mean for Student's T for df = 1 is undefined (no clear way to say it is either + or - infinity), so the variance = E[(X - mean)**2] is also undefined. Returns: allow_nan_stats: Python `bool`. )rrs r.rzDistribution.allow_nan_statsJs   r-c|jS)z?Python `bool` indicating possibly expensive checks are enabled.)rrs r.rzDistribution.validate_args[s   r-c Pt|jfi|}t|di|S)aCreates a deep copy of the distribution. Note: the copy distribution may continue to depend on the original initialization arguments. Args: **override_parameters_kwargs: String/value dictionary of initialization arguments to override with new values. Returns: distribution: A new instance of `type(self)` initialized from the union of self.parameters and override_parameters_kwargs, i.e., `dict(self.parameters, **override_parameters_kwargs)`. r,)dictrr)roverride_parameters_kwargsrs r.copyzDistribution.copy`s,dooD)CDJ 4: # ##r-c\tdjt|j)Nz)batch_shape_tensor is not implemented: {}rformatrr(rs r._batch_shape_tensorz Distribution._batch_shape_tensorr) 3::4:;N;NO QQr-c2|j|5|jjrGtj|jj t jdcdddS|jcdddS#1swYyxYw)aShape of a single sample from a single event index as a 1-D `Tensor`. The batch dimensions are indexes into independent, non-identical parameterizations of this distribution. Args: name: name to give to the op Returns: batch_shape: `Tensor`. rr\r3N) _name_scoperrrr[rrint32rrr3s r.rzDistribution.batch_shape_tensorvsz   $ (   * * ,$$T%5%5%=%=%?+1<<*79((  % % ' (((AB 4B  Bc,tjdSrrrrs r. _batch_shapezDistribution._batch_shape  # #D ))r-cHtj|jS)a)Shape of a single sample from a single event index as a `TensorShape`. May be partially defined or unknown. The batch dimensions are indexes into independent, non-identical parameterizations of this distribution. Returns: batch_shape: `TensorShape`, possibly unknown. )ras_shaperrs r.rzDistribution.batch_shapes  !2!2!4 55r-c\tdjt|j)Nz)event_shape_tensor is not implemented: {}rrs r._event_shape_tensorz Distribution._event_shape_tensorrr-c2|j|5|jjrGtj|jj t jdcdddS|jcdddS#1swYyxYw)zShape of a single sample from a single batch as a 1-D int32 `Tensor`. Args: name: name to give to the op Returns: event_shape: `Tensor`. rrN) rrrrr[rrrrrs r.rzDistribution.event_shape_tensorsz   $ (   * * ,$$T%5%5%=%=%?+1<<*79((  % % ' (((rc,tjdSrrrs r. _event_shapezDistribution._event_shaperr-cHtj|jS)zShape of a single sample from a single batch as a `TensorShape`. May be partially defined or unknown. Returns: event_shape: `TensorShape`, possibly unknown. )rrrrs r.rzDistribution.event_shapes  !2!2!4 55r-c|j|5tj|j|j|j dcdddS#1swYyxYw)zIndicates that `event_shape == []`. Args: name: Python `str` prepended to names of ops created by this function. Returns: is_scalar_event: `bool` scalar `Tensor`. is_scalar_eventrUN)rrr[_is_scalar_helperrrrs r.rzDistribution.is_scalar_eventT   $ "  " "  !1!143J3J K """" ;AA c|j|5tj|j|j|j dcdddS#1swYyxYw)zIndicates that `batch_shape == []`. Args: name: Python `str` prepended to names of ops created by this function. Returns: is_scalar_batch: `bool` scalar `Tensor`. is_scalar_batchrUN)rrr[rrrrs r.rzDistribution.is_scalar_batchrrc\tdjt|j)Nzsample_n is not implemented: {}r)rnseeds r. _sample_nzDistribution._sample_n* ?FF T  r-c |j||g5tj|tjd}|j |d\}}|j ||fi|}tj|dd}tj||gd}tj||}|j||}|cdddS#1swYyxYw)Nrrrr) rrr[rr_expand_sample_shape_to_vectorrrrconcatreshape_set_sample_static_shape) rrrr3kwargsrsamplesbatch_event_shape final_shapes r._call_sample_nzDistribution._call_sample_ns  $ ~  6 ** fllAl;; (olAq$1&1g#//'2126$$l4E%FJk!!';7g--g|Dg    s B)CCc(|j|||S)a~Generate samples of the specified shape. Note that a call to `sample()` without arguments will generate a single sample. Args: sample_shape: 0D or 1D `int32` `Tensor`. Shape of the generated samples. seed: Python integer seed for RNG name: name to give to the op. Returns: samples: a `Tensor` with prepended dimensions `sample_shape`. )r)rrrr3s r.r"zDistribution.samples   |T4 88r-c\tdjt|j)Nzlog_prob is not implemented: {}rrs r. _log_probzDistribution._log_probrr-c X|j||g5t|d|j} |j|fi|cdddS#t$rI} t j |j|fi|cYd}~cdddS#t$r|wxYwd}~wwxYw#1swYyxYwNrr^rV)rr`r\rrr log_probrr^r3roriginal_exceptions r._call_log_probzDistribution._call_log_probs  $w  / # gtzz;e#t~~e.v. # # !# #jdjj9&9: : # ## #" " ## # #@B A  B%B :B;B  BBBB  B)c&|j||S)a/Log probability density/mass function. Args: value: `float` or `double` `Tensor`. name: Python `str` prepended to names of ops created by this function. Returns: log_prob: a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`. )rrr^r3s r.rzDistribution.log_prob s   ud ++r-c\tdjt|j)Nzprob is not implemented: {}rrs r.rzDistribution._prob* ;BB T  r-c X|j||g5t|d|j} |j|fi|cdddS#t$rI} t j |j|fi|cYd}~cdddS#t$r|wxYwd}~wwxYw#1swYyxYwr)rr`r\rrr exprrs r. _call_probzDistribution._call_probs  $w  / # gtzz;e#tzz%*6* # # !# #ndnnU=f=> > # ## #" " ## # #rc&|j||S)a'Probability density/mass function. Args: value: `float` or `double` `Tensor`. name: Python `str` prepended to names of ops created by this function. Returns: prob: a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`. )r rs r.r!zDistribution.prob&s ??5$ ''r-c\tdjt|j)Nzlog_cdf is not implemented: {}rrs r._log_cdfzDistribution._log_cdf3* >EE T  r-c X|j||g5t|d|j} |j|fi|cdddS#t$rI} t j |j|fi|cYd}~cdddS#t$r|wxYwd}~wwxYw#1swYyxYwr)rr`r\rrr r_cdfrs r. _call_log_cdfzDistribution._call_log_cdf7s  $w  / # gtzz;e#t}}U-f- # # !# #idii889 9 # ## #" " ## # #rc&|j||S)a_Log cumulative distribution function. Given random variable `X`, the cumulative distribution function `cdf` is: ```none log_cdf(x) := Log[ P[X <= x] ] ``` Often, a numerical approximation can be used for `log_cdf(x)` that yields a more accurate answer than simply taking the logarithm of the `cdf` when `x << -1`. Args: value: `float` or `double` `Tensor`. name: Python `str` prepended to names of ops created by this function. Returns: logcdf: a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`. )rrs r.rzDistribution.log_cdfCs*   eT **r-c\tdjt|j)Nzcdf is not implemented: {}rrs r.rzDistribution._cdfZs* :AA T  r-c X|j||g5t|d|j} |j|fi|cdddS#t$rI} t j |j|fi|cYd}~cdddS#t$r|wxYwd}~wwxYw#1swYyxYwr)rr`r\rrr r rrs r. _call_cdfzDistribution._call_cdf^s  $w  / # gtzz;e#tyy)&) # # !# #mdmmE>% &&r-c\tdjt|j)Nz,log_survival_function is not implemented: {}rrs r._log_survival_functionz#Distribution._log_survival_function}s* 6== J   ! ""r-c Z|j||g5t|d|j} |j|fi|cdddS#t$rJ} t j |j|fi| cYd}~cdddS#t$r|wxYwd}~wwxYw#1swYyxYwr)rr`r\rrr log1prrs r._call_log_survival_functionz(Distribution._call_log_survival_functions  $w  / # gtzz;e#*t**5;F; # # !# #%!:6!: :; ; # ## #" " ## # #s@B!A  B&B ;B<B! BBBB!!B*c&|j||S)aLog survival function. Given random variable `X`, the survival function is defined: ```none log_survival_function(x) = Log[ P[X > x] ] = Log[ 1 - P[X <= x] ] = Log[ 1 - cdf(x) ] ``` Typically, different numerical approximations can be used for the log survival function, which are more accurate than `1 - cdf(x)` when `x >> 1`. Args: value: `float` or `double` `Tensor`. name: Python `str` prepended to names of ops created by this function. Returns: `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`. )rrs r.rz"Distribution.log_survival_functions,  + +E4 88r-c\tdjt|j)Nz(survival_function is not implemented: {}rrs r._survival_functionzDistribution._survival_functions* HOO T  r-c 8|j||g5t|d|j} |j|fi|cdddS#t$r9} d|j |fi|z cYd}~cdddS#t$r|wxYwd}~wwxYw#1swYyxYw)Nrr^rVg?)rr`r\r!rrrs r._call_survival_functionz$Distribution._call_survival_functions  $w  / # gtzz;e#&t&&u77 # # !# #hdhhu/// / # ## #" " ## # #s@BA  B A9*B +B9 BBB  BBc&|j||S)aSurvival function. Given random variable `X`, the survival function is defined: ```none survival_function(x) = P[X > x] = 1 - P[X <= x] = 1 - cdf(x). ``` Args: value: `float` or `double` `Tensor`. name: Python `str` prepended to names of ops created by this function. Returns: `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`. )r#rs r.r$zDistribution.survival_functions&  ' 't 44r-c\tdjt|j)Nzentropy is not implemented: {}rrs r._entropyzDistribution._entropyrr-cp|j|5|jcdddS#1swYyxYw)zShannon entropy in nats.N)rr&rs r.rzDistribution.entropys.  $  ]]_,5c\tdjt|j)Nzmean is not implemented: {}rrs r._meanzDistribution._meanr r-cp|j|5|jcdddS#1swYyxYw)zMean.N)rr*rs r.rzDistribution.mean.  $  ZZ\r(c\tdjt|j)Nzquantile is not implemented: {}rrs r. _quantilezDistribution._quantilerr-c |j||g5t|d|j}|j|fi|cdddS#1swYyxYwr)rr`r\r.)rr^r3rs r._call_quantilezDistribution._call_quantilesT  $w  /- gtzz;e T^^E ,V ,---s +A  Ac&|j||S)aQuantile function. Aka "inverse cdf" or "percent point function". Given random variable `X` and `p in [0, 1]`, the `quantile` is: ```none quantile(p) := x such that P[X <= x] == p ``` Args: value: `float` or `double` `Tensor`. name: Python `str` prepended to names of ops created by this function. Returns: quantile: a `Tensor` of shape `sample_shape(x) + self.batch_shape` with values of type `self.dtype`. )r0rs r.quantilezDistribution.quantiles"   ud ++r-c\tdjt|j)Nzvariance is not implemented: {}rrs r. _variancezDistribution._variancerr-c|j|5 |jcdddS#t$rF} tj|j cYd}~cdddS#t$r|wxYwd}~wwxYw#1swYyxYw)aVariance. Variance is defined as, ```none Var = E[(X - E[X])**2] ``` where `X` is the random variable associated with this distribution, `E` denotes expectation, and `Var.shape = batch_shape + event_shape`. Args: name: Python `str` prepended to names of ops created by this function. Returns: variance: Floating-point `Tensor` with shape identical to `batch_shape + event_shape`, i.e., the same shape as `self.mean()`. N)rr4rr square_stddevrr3rs r.r%zDistribution.variances&   $ ##~~##!# #0 0 ## # #" " ####=A?- A<"A(A<A?( A44A77A<<A??Bc\tdjt|j)Nzstddev is not implemented: {}rrs r.r7zDistribution._stddevs* =DD T  r-c|j|5 |jcdddS#t$rF} tj|j cYd}~cdddS#t$r|wxYwd}~wwxYw#1swYyxYw)aStandard deviation. Standard deviation is defined as, ```none stddev = E[(X - E[X])**2]**0.5 ``` where `X` is the random variable associated with this distribution, `E` denotes expectation, and `stddev.shape = batch_shape + event_shape`. Args: name: Python `str` prepended to names of ops created by this function. Returns: stddev: Floating-point `Tensor` with shape identical to `batch_shape + event_shape`, i.e., the same shape as `self.mean()`. N)rr7rr sqrtr4r8s r.r#zDistribution.stddevs(   $ ##||~##!# #t~~/0 0 ## # #" " ####r9c\tdjt|j)Nz!covariance is not implemented: {}rrs r. _covariancezDistribution._covariance;s* AHH T  r-cp|j|5|jcdddS#1swYyxYw)aCovariance. Covariance is (possibly) defined only for non-scalar-event distributions. For example, for a length-`k`, vector-valued distribution, it is calculated as, ```none Cov[i, j] = Covariance(X_i, X_j) = E[(X_i - E[X_i]) (X_j - E[X_j])] ``` where `Cov` is a (batch of) `k x k` matrix, `0 <= (i, j) < k`, and `E` denotes expectation. Alternatively, for non-vector, multivariate distributions (e.g., matrix-valued, Wishart), `Covariance` shall return a (batch of) matrices under some vectorization of the events, i.e., ```none Cov[i, j] = Covariance(Vec(X)_i, Vec(X)_j) = [as above] ``` where `Cov` is a (batch of) `k' x k'` matrices, `0 <= (i, j) < k' = reduce_prod(event_shape)`, and `Vec` is some function mapping indices of this distribution's event dimensions to indices of a length-`k'` vector. Args: name: Python `str` prepended to names of ops created by this function. Returns: covariance: Floating-point `Tensor` with shape `[B1, ..., Bn, k', k']` where the first `n` dimensions are batch coordinates and `k' = reduce_prod(self.event_shape)`. N)rr>rs r.rzDistribution.covariance?s4H   $        r(c\tdjt|j)Nzmode is not implemented: {}rrs r._modezDistribution._modefr r-cp|j|5|jcdddS#1swYyxYw)zMode.N)rrArs r.r zDistribution.modejr,r(cFtj|||jSN)r)r rrrs r._cross_entropyzDistribution._cross_entropyo"  ) ) eT%9%9 ;;r-cr|j|5|j|cdddS#1swYyxYw)aComputes the (Shannon) cross entropy. Denote this distribution (`self`) by `P` and the `other` distribution by `Q`. Assuming `P, Q` are absolutely continuous with respect to one another and permit densities `p(x) dr(x)` and `q(x) dr(x)`, (Shanon) cross entropy is defined as: ```none H[P, Q] = E_p[-log q(X)] = -int_F p(x) log q(x) dr(x) ``` where `F` denotes the support of the random variable `X ~ P`. Args: other: `tfp.distributions.Distribution` instance. name: Python `str` prepended to names of ops created by this function. Returns: cross_entropy: `self.dtype` `Tensor` with shape `[B1, ..., Bn]` representing `n` different calculations of (Shanon) cross entropy. N)rrErrr3s r.rzDistribution.cross_entropyss5,   $ (   '(((-6cFtj|||jSrD)r rrrs r._kl_divergencezDistribution._kl_divergencerFr-cr|j|5|j|cdddS#1swYyxYw)anComputes the Kullback--Leibler divergence. Denote this distribution (`self`) by `p` and the `other` distribution by `q`. Assuming `p, q` are absolutely continuous with respect to reference measure `r`, the KL divergence is defined as: ```none KL[p, q] = E_p[log(p(X)/q(X))] = -int_F p(x) log q(x) dr(x) + int_F p(x) log p(x) dr(x) = H[p, q] - H[p] ``` where `F` denotes the support of the random variable `X ~ p`, `H[., .]` denotes (Shanon) cross entropy, and `H[.]` denotes (Shanon) entropy. Args: other: `tfp.distributions.Distribution` instance. name: Python `str` prepended to names of ops created by this function. Returns: kl_divergence: `self.dtype` `Tensor` with shape `[B1, ..., Bn]` representing `n` different calculations of the Kullback-Leibler divergence. N)rrKrHs r.rzDistribution.kl_divergences52   $ (   '(((rIcTdjt|j|j|jj dj|jnd|j j dj|j nd|jjS)Nzatfp.distributions.{type_name}("{self_name}"{maybe_batch_shape}{maybe_event_shape}, dtype={dtype})z, batch_shape={}rAz, event_shape={}) type_name self_namemaybe_batch_shapemaybe_event_shaper\)rrr(r3rndimsrr\rs r.__str__zDistribution.__str__s  &vt*--))&*&6&6&<&<&H$6#<#)rNrOrrr\)rrr(r3rrr\rs r.rzDistribution.__repr__sN %ft*--)) ,, ,,jjoo %' (r-c#Ktj|j5tj||gn||jz5}|ddddddy#1swYxYw#1swYyxYww)z(Helper function to standardize op scope.Nr)rrr3r)rr3rscopes r.rzDistribution._name_scopesq  " >>$2Vt/B/B B EHM s4 A9)A- A!A- A9!A* &A--A62A9ctj|}|tj|}n/t j ||j j}|jj}|tj|}tjtj|dt jdgtj tj"|}tj$||}||fS|dk(rf|It'j(t j|g|j j|}||fStj$|dg}||fS|dk7r t+d||fS)z-Helper to `sample` which ensures input is 1D.)r\rrrUz#Input is neither scalar nor vector.)rrr reduce_prodnpprodr\as_numpy_dtype get_shaperRrrankr pick_vectorequalarrayrrrrr[ro)rxr3 x_static_valrZrRexpanded_shapes r.rz+Distribution._expand_sample_shape_to_vectorsN--a0L  ! !! $d WW\)?)?)A Bd KKM  E }nnQe'' .. " ((A3bhh ');=n   A~ .a d7N !  !  ! ! HHl^177+A+A+C D  d7N   a! % d7N ! < == d7Nr-ctjtj|}|j j }|j }|j j }|jj }|#|!||||z|z}|jdg|z|H|F|jdg||z z}|j|j j||e|ctjdg||z zj|j}|j|j j|||| | ||z |z }n | |||z |z }|u|stjdg|zj|j jdg|z}|j|j j||S)z+Helper to `sample`; sets static shape info.N) rrrrr\rRrr set_shape concatenate merge_with)rrarrR sample_ndims batch_ndims event_ndimsrs r.rz%Distribution._set_sample_static_shapes ++""<02L KKM  E%%L""((K""((K   [(;6ekk4&5.! \5&&vu|/C'DEekk!++-**512 [4&& &%+% &((3 D4D4D(E kk!++-**512    K$;,{:,  \%= +l:+  !k&=(($ )<=II   )k4&*<=  AKKM,,U34 Hr-cd|j|jdk(S|}|jjI|jjdj"|jj dgk(St j tj|ddS)z;Implementation for `is_scalar_batch` and `is_scalar_event`.r) rRr\dimsr^rr r_rr)rrdynamic_shape_fnrs r.rzDistribution._is_scalar_helpers%   1 $$  E + q!''3__  & & (QC // >>)//%03Q 77r-)NNN)DistributionParamShapes)r)r)r)rr)r,Nr")r)r!)r)r)r)r$)r)r)r2)r%)r#)r)r )r)rNN)Pr(r)r*r+r rrpropertyrsetter classmethodrr staticmethodrr3r\rrrrrrrrrrrrrrrrrr"rrrrr r!rrrrrrrrrr!r#r$r&rr*rr.r0r2r4r%r7r#r>rrAr rErrKrrSr contextlibcontextmanagerrrrrr,r-r.rrszFP;' !44l     ! !--($$L??     7 7  )  ) ! !   $$Q(&*  6  6Q( * 6 6 " " 9  # , # ( #+. #'&" #90 #5*  - ,&#8#:% N ;(2;(8( (:* X 8r-rro)&r+rgrtr8numpyrYtensorflow.python.eagerrtensorflow.python.frameworkrrrrtensorflow.python.opsrr #tensorflow.python.ops.distributionsr r tensorflow.python.utilr r tensorflow.python.util.tf_exportr__all__rjrhr'r?rSr`rbrrexport_constantr(rrr,r-r.rs"2  +.+43+*@4.-6 ($.#++ @), 9DC DCN 567++8+b//FG 3 45EE %'--BC 1 23CC #% +,-S8$0AS8.S8r-