AVhdZddlmZddlmZddlmZddlmZddlmZddlm Z ddlm Z dd lm Z dd lm Z dd lm Z dd lmZdd lmZddlmZddlmZddlmZddlmZddldZdZdZdZdZdZGddZGddZy)zClustering Operations.) constant_op)dtypes)ops) random_seed) array_ops) check_ops)cond)control_flow_ops)gen_clustering_ops)math_ops)nn_impl) random_ops) state_ops) variable_v1) while_loop)embedding_lookup)*squared_euclideancosinerandomkmeans_plus_pluskmc2clustersceZdZdZeedddddfdZedZed Z edd Z d Z d Z d Z edZdZdZdZdZy)KMeansz)Creates the graph for k-means clustering.Frc tttg} t|tr|| vrt d|d| dt tg} || vrt d|d| dt|tr|n|g|_ ||_ ||_ ||_ ||_ t||_t!j"|d|_||_| |_y)aBCreates an object for generating KMeans clustering graph. This class implements the following variants of K-means algorithm: If use_mini_batch is False, it runs standard full batch K-means. Each step runs a single iteration of K-Means. This step can be run sharded across multiple workers by passing a list of sharded inputs to this class. Note however that a single step needs to process the full input at once. If use_mini_batch is True, it runs a generalization of the mini-batch K-means algorithm. It runs multiple iterations, where each iteration is composed of mini_batch_steps_per_iteration steps. Two copies of cluster centers are maintained: one that is updated at the end of each iteration, and one that is updated every step. The first copy is used to compute cluster allocations for each step, and for inference, while the second copy is the one updated each step using the mini-batch update rule. After each iteration is complete, this second copy is copied back the first copy. Note that for use_mini_batch=True, when mini_batch_steps_per_iteration=1, the algorithm reduces to the standard mini-batch algorithm. Also by setting mini_batch_steps_per_iteration = num_inputs / batch_size, the algorithm becomes an asynchronous version of the full-batch algorithm. Note however that there is no guarantee by this implementation that each input is seen exactly once per iteration. Also, different updates are applied asynchronously without locking. So this asynchronous version may not behave exactly like a full-batch version. Args: inputs: An input tensor or list of input tensors. It is assumed that the data points have been previously randomly permuted. num_clusters: An integer tensor specifying the number of clusters. This argument is ignored if initial_clusters is a tensor or numpy array. initial_clusters: Specifies the clusters used during initialization. One of the following: - a tensor or numpy array with the initial cluster centers. - a function f(inputs, k) that returns up to k centers from `inputs`. - "random": Choose centers randomly from `inputs`. - "kmeans_plus_plus": Use kmeans++ to choose centers from `inputs`. - "kmc2": Use the fast k-MC2 algorithm to choose centers from `inputs`. In the last three cases, one batch of `inputs` may not yield `num_clusters` centers, in which case initialization will require multiple batches until enough centers are chosen. In the case of "random" or "kmeans_plus_plus", if the input size is <= `num_clusters` then the entire batch is chosen to be cluster centers. distance_metric: Distance metric used for clustering. Supported options: "squared_euclidean", "cosine". use_mini_batch: If true, use the mini-batch k-means algorithm. Else assume full batch. mini_batch_steps_per_iteration: Number of steps after which the updated cluster centers are synced back to a master copy. random_seed: Seed for PRNG used to initialize seeds. kmeans_plus_plus_num_retries: For each point that is sampled during kmeans++ initialization, this parameter specifies the number of additional points to draw from the current distribution before selecting the best. If a negative value is specified, a heuristic is used to sample O(log(num_to_sample)) additional points. kmc2_chain_length: Determines how many candidate points are used by the k-MC2 algorithm to produce one new cluster centers. If a (mini-)batch contains less points, one new cluster center is generated from the (mini-)batch. Raises: ValueError: An invalid argument was passed to initial_clusters or distance_metric. z&Unsupported initialization algorithm `z`,must be one of `z`.zUnsupported distance metric `rN) RANDOM_INITKMEANS_PLUS_PLUS_INIT KMC2_INIT isinstancestr ValueErrorSQUARED_EUCLIDEAN_DISTANCECOSINE_DISTANCElist_inputs _num_clusters_initial_clusters_distance_metric_use_mini_batchint_mini_batch_steps_per_iterationrandom_seed_opsget_seed_seed_kmeans_plus_plus_num_retries_kmc2_chain_length) selfinputs num_clustersinitial_clustersdistance_metricuse_mini_batchmini_batch_steps_per_iterationrkmeans_plus_plus_num_retrieskmc2_chain_lengthinitialization_algorithmsdistance_metricss T/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/ops/clustering_ops.py__init__zKMeans.__init__8sV"-.CY O"+3LL  23C2DE67r ; <<3OD.. 66GH**:);2? @@'56F8DL%D-D+D)D+./M+ND( ))+6q9DJ)ED&/Dct|tsJ|tk(r|j||S|tk(r|j ||dSJt |)aComputes distance between each input and each cluster center. Args: inputs: list of input Tensors. clusters: cluster Tensor. distance_metric: distance metric used for clustering Returns: list of Tensors, where each element corresponds to each element in inputs. The value is the distance of each row to all the cluster centers. Currently only Euclidean distance and cosine distance are supported. T)inputs_normalized)r#r(r&_compute_euclidean_distancer'_compute_cosine_distancer$)clsr6rr9s r@_distance_graphzKMeans._distance_graphsh fd ## #44  , ,VX >> O +  ) ) (d*44)C((UrBc g}|D]}tj|d5tjtj|dddtj ||dzz t jtjtj|ddz}|j|ddd|S#1swYxYw)aCComputes Euclidean distance between each input and each cluster center. Args: inputs: list of input Tensors. clusters: cluster Tensor. Returns: list of Tensors, where each element corresponds to each element in inputs. The value is the distance of each row to all the cluster centers. Tignore_existingr)keepdimsr transpose_bN) r colocate_withr reduce_sumsquarematmulr transposeappend)rGr6routputinpsquared_distances r@rEz"KMeans._compute_euclidean_distancesF (   S$ 7 (    4a$ G X4@ @ A   ##OOH-q4A B B   &' ( ( ( M ( (s BCC c g}|s7tj|d5tj|d}ddd|D]e}tj|d5|stj|d}|j dt j ||dz dddg|S#1swYuxYw#1swYxYw)aComputes cosine distance between each input and each cluster center. Args: inputs: list of input Tensor. clusters: cluster Tensor inputs_normalized: if True, it assumes that inp and clusters are normalized and computes the dot product which is equivalent to the cosine distance. Else it L2 normalizes the inputs first. Returns: list of Tensors, where each element corresponds to each element in inp. The value is the distance of each row to all the cluster centers. TrJraxisNrM)rrOr l2_normalizerTr rR)rGr6rrDrUrVs r@rFzKMeans._compute_cosine_distancesF    Xt <:''q9:L   S$ 7L $$Sq1# a(//#xTJJKLLL M::LLsB(AB4(B14B= c t|tsJ|j|||j}g}|jtk(rG|j s7t j|d5tj|d}dddt||D]\}}t j|d5tj||d\}}|jtk(r|dz}|j|tj|dgtj|dgfdddt|S#1swYxYw#1swYxYw)aMaps input to closest cluster and the score. Args: inputs: list of input Tensors. clusters: Tensor of cluster centers. Returns: List of tuple, where each value in tuple corresponds to a value in inp. The tuple has following three elements: all_scores: distance of each input to each cluster center. score: distance of each input to closest cluster center. cluster_idx: index of cluster center closest to the corresponding input. TrJrrYNg?)r#r(rHr,r'_clusters_l2_normalizedrrOr r[zipr nearest_neighborsrTrsqueeze) r5r6rscoresrUrVscoreindices distancess r@ _infer_graphzKMeans._infer_graphsO fd ## # ! !&(D4I4I JF F 0  ( ( *   Xt <:''q9:&&)P U   S$ 7P(::3!L    O 3 s ) I%%i')d,-6->->w-M O P PPP <::PPs,D85A1E8EE ch|jtk(xr|j xs|jdkDS)z5Returns True if clusters centers are kept normalized.r)r,r'r-r/r5s r@r^zKMeans._clusters_l2_normalized s9  ! !_ 4 7%% %6  1 1A 58rBctjgd}tj|td}tjdt j d}|jr|jdkDrtj|dd}tj|jt jd }tjtj|gt j }nK|}d}|jr9tjtj|gt j nd}|||||fS) aCreates variables. Args: num_clusters: an integer Tensor providing the number of clusters. Returns: Tuple with following elements: - cluster_centers: a Tensor for storing cluster centers - cluster_centers_initialized: bool Variable indicating whether clusters are initialized. - cluster_counts: a Tensor for storing counts of points assigned to this cluster. This is used by mini-batch training. - cluster_centers_updated: Tensor representing copy of cluster centers that are updated every step. - update_in_steps: numbers of steps left before we sync cluster_centers_updated back to cluster_centers. N)shapeF)namevalidate_shape initialized)dtyperkrclusters_updatedupdate_in_stepsrn) rplaceholder_with_defaultr VariableV1CLUSTERS_VAR_NAMErboolr-r/int64zerosones)r5r7 init_valuecluster_centerscluster_centers_initializedcluster_centers_updatedrpcluster_countss r@_create_variableszKMeans._create_variabless($33BdCJ!,,*5BO"-"8"8 V[[}#6  D Dq H!, 6 6 -e!E$..  . .  "o #-- //<. =?n!0o ! !  nnl^6<<@ B'+ 8. #_ 66rBcg}|D]H}tj|d5|jtj|ddddJ|S#1swYVxYw)zNormalized the input data.TrJrdimN)rrOrTr r[)rGr6rUrVs r@_l2_normalize_datazKMeans._l2_normalize_data@sbF8   S$ 78 g**3A67888 M88s 'AA c t|jtst|jr,|j}t j |j }n7t j |j}tj|d}|j}|j|\}}}}}t|j|||j|j|j|j||| j!} |} |jt"k(r8|j%|}|j'st)j*| d} |j-|| \} } } |j.rL|j1||||}|Jt j2|g5|j5|| ||}dddn| |k(sJ|j7||| |}| | | || fS#1swYxYw)a%Generate a training graph for kmeans algorithm. This returns, among other things, an op that chooses initial centers (init_op), a boolean variable that is set to True when the initial centers are chosen (cluster_centers_initialized), and an op to perform either an entire Lloyd iteration or a mini-batch of a Lloyd iteration (training_op). The caller should use these components as follows. A single worker should execute init_op multiple times until cluster_centers_initialized becomes True. Then multiple workers may execute training_op any number of times. Returns: A tuple consisting of: all_scores: A matrix (or list of matrices) of dimensions (num_input, num_clusters) where the value is the distance of an input vector and a cluster center. cluster_idx: A vector (or list of vectors). Each element in the vector corresponds to an input row in 'inp' and specifies the cluster id corresponding to the input. scores: Similar to cluster_idx but specifies the distance to the assigned cluster instead. cluster_centers_initialized: scalar indicating whether clusters have been initialized. init_op: an op to initialize the clusters. training_op: an op that runs an iteration of training. rrrN)r#r+r$callablerconvert_to_tensorr*rrjr)r~_InitializeClustersOpFactoryr,r2r3r4opr'rr^r r[rfr-_mini_batch_sync_updates_opcontrol_dependencies_mini_batch_training_op_full_batch_training_op)r5r8r7r6cluster_centers_varr{ total_countsr|rpinit_oprz all_scoresrb cluster_idxsync_updates_op training_ops r@training_graphzKMeans.training_graphIs6 4))3/''(//**4+=+=>l..t/E/EF__%56q9l \\F..|<5|* l$4d6K6K D668O8O4# %&(RT  *O /&&v.f  ) ) +!..AF&*&7&7&P#J  88 .0G o (( (  # #_$5 6A226;3J3?A AA  3 33 3001<1DFk  V-H [ ""AAs 'G((G1cjrZjdkDrKJtjd5fd}t jdk|fdcdddSt j S#1swYyxYw)NrTrJc tjtjjdz g5tj d5j tk(rtjd}n}dddtj d5tjtjg5tj dd5tjtjtjg5tjcdddcdddcdddcdddcdddS#1swYxYw#1swYnxYw dddn #1swYnxYwdddn #1swYnxYwdddn #1swYnxYwdddy#1swYyxYw)NrTrJr) rrrassignr/rOr,r'r r[r zeros_likeidentity)rzr|rr5rrps r@_fz.KMeans._mini_batch_sync_updates_op.._fs''#CCaGI)   ?""'? :&&/9")"6"6+#4#:  :""#6M ?++##$7IJL?&&tTB?//&&|'0';';L'IK1?%--o> ?????? ? ? ? ? : :????????? ? ? ? ? ? ?sG.E"?G+F. F!>F E. 4 F = F F. G"E+ 'G.E73F ; FF F F.F" F.% G.F7 3GG rc0tjdS)Nr)r assign_sub)rpsr@z4KMeans._mini_batch_sync_updates_op..sI((!<rB)r-r/rrOr r no_op)r5rprr|rrs````` r@rz"KMeans._mini_batch_sync_updates_ops  D Dq H  (( (   _d C > ? ?8yy q " <>= > >D # # %%E > >s &BB c rg}t||D]n\}}tj|d5|Jtj|dg}tj |\}} tj |} tj|d5tj||} dddtj|d5tj||} dddtjtj| |j| | } tj|| | }tjtj| dgtjtjtj|dz dgtj gd}|tj"tj| ||j zz}tj$tj" | z|j}tj||}||z}dddt'j(| }t'j(||}|j+||gqt-j.|S#1swYxYw#1swYxYw#1swYyxYw)aCreates an op for training for mini batch case. Args: inputs: list of input Tensors. cluster_idx_list: A vector (or list of vectors). Each element in the vector corresponds to an input row in 'inp' and specifies the cluster id corresponding to the input. cluster_centers: Tensor Ref of cluster centers. total_counts: Tensor Ref of cluster counts. Returns: An op for doing an update of mini-batch k-means. TrJNr]rqrr)r_rrOrreshapeuniquesizegatherr unsorted_segment_sum ones_likernconcatrxrankrint32cast reciprocalr scatter_addextendr group)r5r6cluster_idx_listrzr update_opsrVr unique_ids unique_idxnum_unique_cluster_idx old_countsold_cluster_centers count_updatescluster_center_updatesbroadcast_shape learning_rate update_countsupdate_cluster_centerss r@rzKMeans._mini_batch_training_opsJ(890A[   S$ 7(0''''' bT: "+!1!1+!> J!* !;   |T B B '' jA* B   E N ) 0 0* M  N!55    ,2D2D E .0 "*!>!> 3"5$**   4qc : NN!!).."5"9A3?ll $,  (--   m_ = II#,#- -!++ MM*}4cii @B !))-I -/Q(0T ++L*,9;m(44_5?5K M(>?@a0Ab  ! !: ..Q B B N N(0(0s=A J- J$J-J EJ-J J- J* %J--J6 cg}g}tjd|dj}t||D]\}} t j |d5|j tj|| ||j tjtjtjtjtj|ddgddg| |dddt j |d5tj|tjtj||dj|zz } |jrt!j"| d } dddt%j&| S#1swY~xYw#1swY,xYw) aCreates an op for training for full batch case. Args: inputs: list of input Tensors. num_clusters: an integer Tensor providing the number of clusters. cluster_idx_list: A vector (or list of vectors). Each element in the vector corresponds to an input row in 'inp' and specifies the cluster id corresponding to the input. cluster_centers: Tensor Ref of cluster centers. Returns: An op for doing an update of mini-batch k-means. gư>rrqTrJr]rNr)rconstantrnr_rrOrTr rrrrxrjadd_nrr^r r[rr) r5r6r7rrz cluster_sumsr}epsilonrVrnew_clusters_centerss r@rzKMeans._full_batch_training_opsLN""4vay?G(89 :[   S$ 7:  ) )#{L I K  ) )!!NN!)))//#*>q*AB4HJG*<  9 ::: :  ?D AQ%^^L9 --~6 Q8M8M N    % % '&334HaP Q   O-A BB::QQsB!F)A7F6)F3 6F?N)T)__name__ __module__ __qualname____doc__r r&rA classmethodrHrErFrfr^r~rrrrrrBr@rr5s1 !,9#./,-!$^0@)),42%N8 -6^D"L&&PA/F"CrBrcLeZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z y ) raInternal class to create the op to initialize the clusters. The op performs this algorithm (see constructor args): num_remaining = num_clusters - length(cluster_centers) if num_remaining == 0: assert that cluster_centers_initialized is true else: assert that num_remaining > 0 new_centers = choose up to num_remaining initial centers l2-normalize new_centers if using cosine distance all_centers = concat(cluster_centers, new_centers) cluster_centers := all_centers if there is a cluster_centers_updated variable: cluster_centers_updated := cluster_centers num_now_remaining = num_clusters - length(cluster_centers) if num_now_remaining == 0: cluster_centers_initialized := true c ||_||_||_||_||_||_||_||_| |_| |_ tj|jd|_ |j|jz |_ tj|jD cgc]} tj| dc} |_ycc} w)awCreates an op factory. Args: inputs: See KMeans constructor. num_clusters: An integer Tensor providing the number of clusters. initial_clusters: See KMeans constructor. distance_metric: See KMeans constructor. random_seed: See KMeans constructor. kmeans_plus_plus_num_retries: See KMeans constructor. kmc2_chain_length: See KMeans constructor. cluster_centers: The TF variable holding the initial centers. It may already contain some centers when the op is executed. cluster_centers_updated: A second TF variable to hold a copy of the initial centers, used for full-batch mode. In mini-batch mode, cluster_centers_updated is the same variable as cluster_centers. cluster_centers_initialized: A boolean TF variable that will be set to true when all the initial centers have been chosen. rN)r)r*r+r,r2r3r4_cluster_centers_cluster_centers_updated_cluster_centers_initializedrrj _num_selected_num_remainingr r _num_data) r5r6r7r8r9rr<r=rzr|r{is r@rAz%_InitializeClustersOpFactory.__init__6s.DL%D-D+DDJ)ED&/D+D$;D!(CD%")>)>?BD,,t/A/AAD^^(, 51 A 57DN5s)Cc,tjtj|jdgdt j |jtj|jtj}t|j|dS)Nr]r)minvalmaxvalseedrndiv)partition_strategy) rrandom_uniformrrrr rrrrvr2rr))r5rds r@_randomz$_InitializeClustersOpFactory._random]sf''$--t4}}T^^V\\: ZZll G DLL'e LLrBc$|jd}|jtk(rtj|d}t j |tj|jtj|j|jS)Nrrr)r)r,r'r r[r kmeans_plus_plus_initializationr rrrrvr2r3)r5rVs r@_kmeans_plus_plusz._InitializeClustersOpFactory._kmeans_plus_plusfsm ,,q/C /  ! ,c  = = X]]4.. =tzz ** ,,rBctjdtjd}tj|j z t j}tjtjj|dfd}fd}tj||ddg\}}|S)aqAdds new initial cluster centers using the k-MC2 algorithm. In each call to the op, the provided batch is split into subsets based on the specified `kmc2_chain_length`. On each subset, a single Markov chain of the k-MC2 algorithm is used to add *one* new center cluster center. If there are less than `kmc2_chain_length` points in the subset, a single center is added using one Markov chain on the full input. It is assumed that the provided batch has previously been randomly permuted. Otherwise, k-MC2 may return suboptimal centers. Returns: An op that adds new cluster centers. rrqrc0tj|S)z&Stopping condition for the while loop.)r less)r_ num_to_samples r@_condzB_InitializeClustersOpFactory._kmc2_multiple_centers.._conds ]]1m ,,rBcfd}fd}tjtjjd||}t j j |d}jj ur"t j j|d}dzjtj|dz fS)z5Body that adds a single new center based on a subset.ctjdddg}jtk(rt j |d}|S)z+Returns a random point as a cluster center.rrr]r)rrr,r'r r[) new_center first_shardr5s r@_sample_randomzZ_InitializeClustersOpFactory._kmc2_multiple_centers.._body.._sample_randomsE&&{1~2w?  O 3++JA>*rBc jz}| jz}||}tj| jd\}}tjt j | j}t j||ddg} jtk(rtj|d}t j j|gdS)zEReturns previous centers as well as a new center sampled using k-MC2.rr]rr)r4r r`rkmc2_chain_initializationrrar2rr,r'r r[r) startendsubsetrrenew_center_indexnewly_sampled_centerrrr5s r@_sample_kmc2_chainz^_InitializeClustersOpFactory._kmc2_multiple_centers.._body.._sample_kmc2_chainsD+++d---U3');; D))1. 9.GG   i ($**6 )008H1I23R :  O 3!(!5!5""+ !6!68L M !# #rBrFrlr) r r equalrrrrrr*rrj)rrrr new_centersassigned_centersrr5s` r@_bodyzB_InitializeClustersOpFactory._kmc2_multiple_centers.._bodys#0II ..++Q / k#))   UD & &d.C.C C$++  ) )  "UD&&9I)J1)MM MMrB) r)rrjr rr4rrmaximumminimumrr) r5 batch_size max_to_samplerrr num_remainingrrs ` @@r@_kmc2_multiple_centersz3_InitializeClustersOpFactory._kmc2_multiple_centersps,,q/K-a0JMMT,,,FLLBM$$,,m.sY--dllA>rB)r rrr5samplers` r@_greedy_batch_samplerz2_InitializeClustersOpFactory._greedy_batch_samplers- 99T^^t':'::> rBctjtj|j|j g5|cdddS#1swYyxYwN)rrrassert_greater_equalrrrs r@_single_batch_samplerz2_InitializeClustersOpFactory._single_batch_samplersJ ! !  ' '8K8K LM O Ys AAc*t|jtrI|jtk(r|j |j S|j |jSt|jr&|j|j|jStjtj|jtj |jdg5|jcdddS#1swYyxYwr)r#r+r$r rrrrrr)rrrr assert_equalrrjrhs r@_choose_initial_centersz4_InitializeClustersOpFactory._choose_initial_centerss$((#.  ; .))$,,77))$*@*@AA $(( )  # #DLL$2E2E FF  # #  !4!4!*1G1G!H!K M% &%% &&&s 3 D  Dcjjtk(rtjdt j t jjdfdfd}tjj|d}jjur"tjj|d}jtj|dz S)z>Adds some centers and returns the number of centers remaining.rrrcSrr)rsr@rz?_InitializeClustersOpFactory._add_new_centers..s{rBcHtjjgdSr)rrr)rr5sr@rz?_InitializeClustersOpFactory._add_new_centers..s   $"7"7!EqIrBFr)rr,r'r r[r r rrrrrrr*rrj)r5 all_centersars` @r@_add_new_centersz-_InitializeClustersOpFactory._add_new_centerss..0K /((!.s)""4#D#DdKrB)rrrassert_positiverr+r"rr r r rr r)r5num_now_remainings` r@ _initializez(_InitializeClustersOpFactory._initializes ! !!!$"5"56#  "   9 , 779 113 YY ..*A . K  " " " "s A0B//B8ctjtjjdfdjS)z#Returns the cluster initializer op.rcDtjjdSr )rrrrhsr@rz1_InitializeClustersOpFactory.op..s &&t'H'H$OrB)r r rrrrhs`r@rz_InitializeClustersOpFactory.ops4 99t**A.O  rBN)rrrrrArrrrrrr rrrrBr@rrs=,%7NM,Qf &6" "rBrN)rtensorflow.python.frameworkrrrrr0tensorflow.python.opsrrr r r r r rrrr#tensorflow.python.ops.embedding_opsr(tensorflow.python.ops.gen_clustering_opsr&r'r r!r"rtrrrrBr@rs3.+F++&24*),+-,@7 1 *  gCgCTggrB