BVh)dZddlZddlmZddlmZddlmZddlm Z ddlm Z ddlm Z dd lm Z dd lm Z dd lmZdd lmZdd lmZ ddZdZdZdZdZGddej.Zy)z5The implementation of `tf.data.Dataset.padded_batch`.N) dataset_ops)nest) structure) constant_op)dtypes)ops) smart_cond) tensor_shape) tensor_spec) tensor_util)gen_dataset_opsc|Ltj|}ttj|D]\}}|r t d|dt ||||||S)z/See `tf.data.Dataset.padded_batch` for details.z> "<#4#4#8#=#= ?j)$)D9??*   rc tj|}tj|j Dcgc]}||nd c}t j }t!||std |d |d |Scc}w#ttf$r}tj|t j }|jj?t|jjdk7rtd|d|jd||jt j k7r'td|d |jjd|tj|}Yd}~ d}~wwxYw) aSConverts `padded_shape` to a `tf.Tensor` representing that shape. Args: padded_shape: A shape-like object, which may be a `tf.TensorShape`, a Python sequence, or a 1-D `tf.Tensor` of `tf.int64` elements. input_component_shape: A `tf.TensorShape`, with which `padded_shape` must be compatible. Returns: A 1-D `tf.Tensor` of `tf.int64` elements, representing `padded_shape`. Raises: ValueError: If `padded_shape` is not a shape or not compatible with `input_component_shape`. TypeError: If `padded_shape` is not convertible to a `tf.int64` tensor. Ndtype)preferred_dtypez Padded shape z7 must be a `tf.int64` vector tensor, but its shape was .z> must be a `tf.int64` vector tensor, but its element type was zThe padded shape z" is not compatible with the shape & of the corresponding input component.)r as_shaperconvert_to_tensoras_listrint64 TypeErrorrrr!r"r-rr constant_value_as_shaper))r%r&padded_shape_as_shapedimretes r_padded_shape_to_tensorr<Os~"E(11,?   (002! B&!'-ll  4C( **?*? A ()>(?@22G1HI:; << *5! Z E    fll KC yy~~!c#))..&9Q&>  ,("yyk , -234 yyFLL  ,(..1iinn-=Q @ AFGH(??D Es)6B A<B<BE.CE))E.cd}d}tj|tj|}tj||||S)zGReturns padding values with None elements replaced with default values.cX|jtjk(ry|jtjk(r t d|jtj k(r%t jdtj Stj|jS)NzIUnable to create default padding value for a component of type 'variant'.rr,) base_dtyperstringvariantr6bfloat16rconstantnp zeros_likeas_numpy_dtype)ts r make_zeroz-_padding_values_or_default..make_zerosv||v}}$   ' + ,,  (  ! !!6?? ;; ]]1++- ..rc||S|SN)r$defaults rvalue_or_defaultz4_padding_values_or_default..value_or_defaultsm7..r)r map_structurerget_legacy_output_typesmap_structure_up_to)rrrIrNdefault_paddings r_padding_values_or_defaultrSsN //&&44]CE/ ! !.2B"0/ CCrctj|d}|jjt j gst d|jd|j|k7rtd|jd|d|S)a#Converts the padding value to a tensor. Args: value: The padding value. output_type: Its expected dtype. Returns: A scalar `Tensor`. Raises: ValueError: if the padding value is not a scalar. TypeError: if the padding value's type does not match `output_type`. padding_valuerzMInvalid `padding_values`. `padding_values` values should be scalars, but got r0z7Invalid `padding_values`. `padding_values` values type z does not match type r1) rr3ris_compatible_withr TensorShaperr-r6)r$ output_types r_padding_value_to_tensorrYs   O <%  ' ' (@(@(D E 338;;-qB CC [[K !KK=(=k]K<= >> ,rc6eZdZdZ dfd ZedZxZS)rzEA `Dataset` that batches and pads contiguous elements from its input.c |_d}tj||j|_t j |t jd_t|tj|}tj||} g} ttj|| D] \} } | jt!| | "tj"|| _tj&|r.tj&stjfd|tj(|t*tj,|_t j |t j0d_fd} tj| j$}t5j6tj,j|tj8j_|_t?j@|jBjtjj$Dcgc]'}t j |t j)c}tjj.j2t5jDj:jFjI}tJ||y cc}w) z"See `Dataset.batch()` for details.ct|tjs4t|tjr t dt d|dy)Nz8`padded_batch` is not supported for datasets of datasetsz`padded_batch` is only supported for datasets that produce tensor elements but type spec of elements in the input dataset is not a subclass of TensorSpec: `z`.) isinstancer TensorSpecr DatasetSpecr6)component_specs r check_typesz1_PaddedBatchDataset.__init__..check_typess[  (>(> ? nk&=&= >HJ J++2/0 0 @rr)r-rcSrKrL)_rs rz._PaddedBatchDataset.__init__..sNrrctjtjjrt j jndgjt j|SrK) r rWr smart_constant_value_drop_remainderr constant_value _batch_size concatenater7)sselfs r_padded_shape_to_batch_shapezB_PaddedBatchDataset.__init__.._padded_shape_to_batch_shapes_  % %  , ,T-A-A B  $ $T%5%5 6HL'  [88; <=rr,)rrrr output_shapesmetadataN)'_input_datasetrrO element_specrr3rr5rirSrr flatten_up_tor#rappendr<pack_sequence_as_padded_shapes is_nestedrQrYrP_padding_valuesboolrgrconvert_legacy_structureget_legacy_output_classes _structure_namer padded_batch_dataset_v2_variant_tensorget_flat_tensor_shapes _metadataSerializeToStringsuper__init__)rlrrrrrrra input_shapesflat_padded_shapesflat_padded_shapes_as_tensorsr&r%rmrnrkvariant_tensor __class__s` ` rrz_PaddedBatchDataset.__init__sy(D0 {M$>$>?'D,,&,,\;D/ NN77 FL++L-H$&!/2 \"$608H+|#** !,0E FHH // 0MOD  ~~l#DNN>,J))*B*68n 33.++M:rsk<2,13.+24331 !%!%!& .00 fC02S+22Sr