AVh=dZddlZddlmZddlmZddlmZddlmZddlmZddl m Z dd l m Z dd l m Z dd l mZdd lmZdd lmZddlmZdZdZdZdZdZ ddZ ddZedgej8dd ddZdZedgej8dd ddZy) zGradient checker for any ops, graphs. The gradient checker verifies numerically that an op/graph properly computes the gradients N) constant_op)dtypes)indexed_slices)ops)tensor) array_ops) gradients)math_ops) tf_logging) deprecation) numpy_compat) tf_exportcFt|tr|Sd}|D]}||z} |S)N) isinstanceint)tyxs V/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/ops/gradient_checker.py_productr%s33 H A  1fa HcV|s|Si}|j||j||S)N)update)extra_feed_dict new_feedsrs r _extra_feedsr/s-  !((?((9 (rc |jjrt|dz}|jjrdnd}t|}t|dd} t||z} t j || f|jj j} t j ||jj} | jj|jj j} tj}t| D]}d| |<t|tjrt|j!|j"|j$gt'||||| i\}}t)||D]*\}}|| z}|| z}| |||fxx|j*z cc<,n~t|t,j.sJdt1|z|j!|t'||||| i}|jj| j| dd|f<d| |<| s|j!|t'||||| i}|j2|j2k7r%t5d |j2d |j2t j6|r t5d t9j:dd | | S) a1Computes the theoretical Jacobian for dy/dx. Computes the theoretical Jacobian using the ops generated by compute_gradient(). Args: x: the tensor "x". x_shape: the dimensions of x as a tuple or an array of ints. x_data: a numpy parray as the input data for x dy: the tensor "dy". dy_shape: the dimensions of dy as a tuple or an array of ints. dx: Tensor or IndexedSlices representing dx extra_feed_dict: dict that allows fixing specified tensor values during the jacobian calculation. Returns: A 2-d numpy array representing the Jacobian for dy/dx. It has "x_size" rows and "dy_size" columns where "x_size" is the number of elements in x and "dy_size" is the number of elements in dy. Raises: ValueError: If `dy` is empty but the gradient is nonzero. )r rNdtype feed_dictzdx = rz)Empty gradient has wrong shape: expected z, got z#Empty tensor with nonzero gradientszTheoretical Jacobian = %s)r" is_complextuplernpzeros real_dtypeas_numpy_dtyperavelviewrget_default_sessionrangerr IndexedSlicesrunindicesvaluesrzipflatrTensorstrshape ValueErroranyloggingvlog)rx_shapex_datadydy_shapedxr dy_factorx_size x_val_sizedy_sizejacobiandy_data dy_data_flatsesscolbackprop_indicesbackprop_valuesivr_beginr_endbackprops r_compute_theoretical_jacobianrQ8s4WWGnt#G88&&aA) G & $* X  *'XXvw'GG..==?( HHXRXX%<%< ='%%bhh&9&9&H&HI, "$ 7^cL"n223*.(( ::ryy ! 1fb'2JK+3+M'&8/$!Qj.*$#$.$/ FMM *=Gc"g,== * _q&"g6NOQh!)..x~~>hq#vL& xx l?QG4LMOH~~%  hnn6 77 vvh < == ,,q.9 /rcz|jtjk(r$tj|tj }|jtjk(r$tj|tj }|jtjj k(r|jtj }t||jjrdndz}t||jjrdndz}|jjj } |jjj } tj||jj }tjd|z| d} tj||f| } t|D]} |j!}|j!}|j#j%| | xx|z cc<|j't)|||i}|j#j%| | xx|zcc<|j't)|||i}||z | z }|j#j%| | | ddf<t+j,dd| | S)aComputes the numeric Jacobian for dy/dx. Computes the numeric Jacobian by slightly perturbing the inputs and measuring the differences on the output. Args: x: the tensor "x". x_shape: the dimensions of x as a tuple or an array of ints. x_data: a numpy array as the input data for x y: the tensor "y". y_shape: the dimensions of y as a tuple or an array of ints. delta: the amount of perturbation we give to the input extra_feed_dict: dict that allows fixing specified tensor values during the jacobian calculation. Returns: A 2-d numpy array representing the Jacobian for dy/dx. It has "x_size" rows and "y_size" columns where "x_size" is the number of elements in x and "y_size" is the number of elements in y. r rr!r#NzNumeric Jacobian = %s)r"rbfloat16r castfloat32r*astyper'rr%r)r np_asarrayr(r.copyr+r,evalrr:r;)rr<r=ry_shapedeltarrBy_sizex_dtypey_dtypescalerErowx_posx_negy_posy_negdiffs r_compute_numeric_jacobianrgs2WW a(AWW a(A \\V__333 ]]2:: &F G QWW%7%7Q ?& G QWW%7%7Q ?& GG   - -' GG   - -'  " "61G1G H&  ! !!e)7 ;B ?% XXvv&g 6(6]2c KKME KKME KKMw$-$ FF\/Au:FF GE KKMw$-$ FF\/Au:FF GE EMU "Dzz|((1HS!V2 ,,q*H5 /rc*|jj5tjd||j}t j |}dddtj||}t|dk(sJ|dfS#1swY7xYw)z.Returns a node to compute gradient of y wrt x.g?)r7r"Nrr) graph as_defaultrconstantr"ridentityr len)rrr[dy_origr>gradss r_compute_dx_and_dyrps  ww%""3gQWWEG   G $B%   aB '% Uq q7  %%s 8B  Bc Htj|j} tjtjtj tj tjtjg} | j| vsJd| jztj|j} | j| vsJd| jz|3t|j} t|| k(s Jd|d| |} nhtjj|j!| j"} | j$r$tjj|| _t)||| ||||}t+||| ||||}||fS)z0Computes the theoretical and numerical jacobian.zDon't support type %s for xzDon't support type %s for yz x_shape = z, init_data shape = r)ras_dtyper"float16rTrVfloat64 complex64 complex128 base_dtypenamelistr7r'random random_samplerWr*r%imagrQrg)rr<r@rr[r> x_init_valuer\rr allowed_typest2i_shaper=jacob_tjacob_ns r_compute_gradientrs[ ooagg!>>6??FNN>>6#3#3V5F5FH-  &N(E(NN & qww" - 'P)F)PP '<%%&G =G #' # F YY $ $W - 4 4Q5E5E FF||II++G4fk )&"gr? L' %&!We_ N' ' rct|tsJt|Dcgc]}t|||c}\} } |/t|ttfsJ|D]} | j |dgt |z}t||| | |D cgc]\}} } }}t|| | |||||| }}} } }}|Scc}wcc}}} } }w)z)Compute gradients for a list of x values.Nrr)rrzr3rpr&r0rmr)rr<rr[r~r\ init_targetsrxir@r>initx_shapeidxidyi x_init_valueirets r_compute_gradient_listrs At   a@#B73@ A&"b lT5M 22 2 hhj6CF?L7:!Wb":F7H I I 2R3] 2xa#} / C I# I *A Is B7!B< ztest.compute_gradient)v1zUse tf.test.compute_gradient in 2.0, which has better support for functions. Note that the two versions have different usage, so code change is needed.)date instructionsc |i}t|trt||||||||S|/t|ttfsJ|D]}|j t |||\} } t ||| ||| ||| } | S)aComputes and returns the theoretical and numerical Jacobian. If `x` or `y` is complex, the Jacobian will still be real but the corresponding Jacobian dimension(s) will be twice as large. This is required even if both input and output is complex since TensorFlow graphs are not necessarily holomorphic, and may have gradients not expressible as complex numbers. For example, if `x` is complex with shape `[m]` and `y` is complex with shape `[n]`, each Jacobian `J` will have shape `[m * 2, n * 2]` with J[:m, :n] = d(Re y)/d(Re x) J[:m, n:] = d(Im y)/d(Re x) J[m:, :n] = d(Re y)/d(Im x) J[m:, n:] = d(Im y)/d(Im x) Args: x: a tensor or list of tensors x_shape: the dimensions of x as a tuple or an array of ints. If x is a list, then this is the list of shapes. y: a tensor y_shape: the dimensions of y as a tuple or an array of ints. x_init_value: (optional) a numpy array of the same shape as "x" representing the initial value of x. If x is a list, this should be a list of numpy arrays. If this is none, the function will pick a random tensor as the initial value. delta: (optional) the amount of perturbation. init_targets: list of targets to run to initialize model params. extra_feed_dict: dict that allows fixing specified tensor values during the Jacobian calculation. Returns: Two 2-d numpy arrays representing the theoretical and numerical Jacobian for dy/dx. Each has "x_size" rows and "y_size" columns where "x_size" is the number of elements in x and "y_size" is the number of elements in y. If x is a list, returns a list of two numpy arrays. rr)rrzrr&r0rpr) rr<rr[r~r\rrrr@r>rs rcompute_gradientrsdO4 !!Wa,". QQ  tUm 44 4$   1g .FB AwAwL%,; =C Jrct|tr|g}d}|D]X\}}|js |jstj|tj ||z j }Z|S)Nr)rr&sizer'maximumfabsmax)graderrorj_tj_ns r_compute_errorrQsee 6D %:hc3 xx388jjc 2 6 6 89e: ,rztest.compute_gradient_errorc >t||||||||}t|S)aJComputes the gradient error. Computes the maximum error for dy/dx between the computed Jacobian and the numerically estimated Jacobian. This function will modify the tensors passed in as it adds more operations and hence changing the consumers of the operations of the input tensors. This function adds operations to the current session. To compute the error using a particular device, such as a GPU, use the standard methods for setting a device (e.g. using with sess.graph.device() or setting a device function in the session constructor). Args: x: a tensor or list of tensors x_shape: the dimensions of x as a tuple or an array of ints. If x is a list, then this is the list of shapes. y: a tensor y_shape: the dimensions of y as a tuple or an array of ints. x_init_value: (optional) a numpy array of the same shape as "x" representing the initial value of x. If x is a list, this should be a list of numpy arrays. If this is none, the function will pick a random tensor as the initial value. delta: (optional) the amount of perturbation. init_targets: list of targets to run to initialize model params. extra_feed_dict: dict that allows fixing specified tensor values during the Jacobian calculation. Returns: The maximum error in between the two Jacobians. rr)rr) rr<rr[r~r\rrrs rcompute_gradient_errorr[s,Z !Wa,& I$  r)NMbP?N)NrNN) __doc__numpyr'tensorflow.python.frameworkrrrrrtensorflow.python.opsrr r tensorflow.python.platformr r:tensorflow.python.utilr r tensorflow.python.util.tf_exportrrrrQrgrprr deprecatedrrrrSrrrs   3.6+.++*<./6  K\:z *$( &*L)-!%(,+/ 4 &'(  !#'"&%): !) :z ,-.  !)-!%(,+/) !/ )r