Vh>~4dZdZdZddlZ ddlZddlmZddlm Z ddl m Z ddl m Ze jd d d Ze jd d d Zd ZdZdZddZdZdZdZdZdZdZdZdZedk(reyy#e$rYwxYw)aU$ --- module: msk_cluster short_description: Manage Amazon MSK clusters version_added: "2.0.0" description: - Create, delete and modify Amazon MSK (Managed Streaming for Apache Kafka) clusters. - Prior to release 5.0.0 this module was called C(community.aws.aws_msk_cluster). The usage did not change. author: - Daniil Kupchenko (@oukooveu) options: state: description: Create (C(present)) or delete (C(absent)) cluster. choices: ['present', 'absent'] type: str default: 'present' name: description: The name of the cluster. required: true type: str version: description: - The version of Apache Kafka. - This version should exist in given configuration. - This parameter is required when I(state=present). type: str configuration_arn: description: - ARN of the configuration to use. - This parameter is required when I(state=present). type: str configuration_revision: description: - The revision of the configuration to use. - This parameter is required when I(state=present). type: int nodes: description: The number of broker nodes in the cluster. Should be greater or equal to two. type: int default: 3 instance_type: description: - The type of Amazon EC2 instances to use for Kafka brokers. choices: - kafka.t3.small - kafka.m5.large - kafka.m5.xlarge - kafka.m5.2xlarge - kafka.m5.4xlarge - kafka.m5.8xlarge - kafka.m5.12xlarge - kafka.m5.16xlarge - kafka.m5.24xlarge - kafka.m7g.large - kafka.m7g.xlarge - kafka.m7g.2xlarge - kafka.m7g.4xlarge - kafka.m7g.8xlarge - kafka.m7g.12xlarge - kafka.m7g.16xlarge default: kafka.t3.small type: str ebs_volume_size: description: The size in GiB of the EBS volume for the data drive on each broker node. type: int default: 100 subnets: description: - The list of subnets to connect to in the client virtual private cloud (VPC). AWS creates elastic network interfaces inside these subnets. Client applications use elastic network interfaces to produce and consume data. - Client subnets can't be in Availability Zone us-east-1e. - This parameter is required when I(state=present). type: list elements: str security_groups: description: - The AWS security groups to associate with the elastic network interfaces in order to specify who can connect to and communicate with the Amazon MSK cluster. If you don't specify a security group, Amazon MSK uses the default security group associated with the VPC. type: list elements: str encryption: description: - Includes all encryption-related information. - Effective only for new cluster and can not be updated. type: dict suboptions: kms_key_id: description: - The ARN of the AWS KMS key for encrypting data at rest. If you don't specify a KMS key, MSK creates one for you and uses it. default: Null type: str in_transit: description: The details for encryption in transit. type: dict suboptions: in_cluster: description: - When set to true, it indicates that data communication among the broker nodes of the cluster is encrypted. When set to false, the communication happens in plaintext. type: bool default: True client_broker: description: - Indicates the encryption setting for data in transit between clients and brokers. The following are the possible values. TLS means that client-broker communication is enabled with TLS only. TLS_PLAINTEXT means that client-broker communication is enabled for both TLS-encrypted, as well as plaintext data. PLAINTEXT means that client-broker communication is enabled in plaintext only. choices: - TLS - TLS_PLAINTEXT - PLAINTEXT type: str default: TLS authentication: description: - Includes all client authentication related information. - Effective only for new cluster and can not be updated. type: dict suboptions: tls_ca_arn: description: List of ACM Certificate Authority ARNs. type: list elements: str sasl_scram: description: SASL/SCRAM authentication is enabled or not. type: bool sasl_iam: version_added: 5.5.0 description: IAM authentication is enabled or not. type: bool unauthenticated: version_added: 5.5.0 description: Option to explicitly turn on or off authentication type: bool default: True enhanced_monitoring: description: Specifies the level of monitoring for the MSK cluster. choices: - DEFAULT - PER_BROKER - PER_TOPIC_PER_BROKER - PER_TOPIC_PER_PARTITION default: DEFAULT type: str open_monitoring: description: The settings for open monitoring. type: dict suboptions: jmx_exporter: description: Indicates whether you want to enable or disable the JMX Exporter. type: bool default: False node_exporter: description: Indicates whether you want to enable or disable the Node Exporter. type: bool default: False logging: description: Logging configuration. type: dict suboptions: cloudwatch: description: Details of the CloudWatch Logs destination for broker logs. type: dict suboptions: enabled: description: Specifies whether broker logs get sent to the specified CloudWatch Logs destination. type: bool default: False log_group: description: The CloudWatch log group that is the destination for broker logs. type: str required: False firehose: description: Details of the Kinesis Data Firehose delivery stream that is the destination for broker logs. type: dict suboptions: enabled: description: Specifies whether broker logs get send to the specified Kinesis Data Firehose delivery stream. type: bool default: False delivery_stream: description: The Kinesis Data Firehose delivery stream that is the destination for broker logs. type: str required: False s3: description: Details of the Amazon S3 destination for broker logs. type: dict suboptions: enabled: description: Specifies whether broker logs get sent to the specified Amazon S3 destination. type: bool default: False bucket: description: The name of the S3 bucket that is the destination for broker logs. type: str required: False prefix: description: The S3 prefix that is the destination for broker logs. type: str required: False wait: description: Whether to wait for the cluster to be available or deleted. type: bool default: false wait_timeout: description: How many seconds to wait. Cluster creation can take up to 20-30 minutes. type: int default: 3600 notes: - All operations are time consuming, for example create takes 20-30 minutes, update kafka version -- more than one hour, update configuration -- 10-15 minutes; - Cluster's brokers get evenly distributed over a number of availability zones that's equal to the number of subnets. extends_documentation_fragment: - amazon.aws.common.modules - amazon.aws.region.modules - amazon.aws.boto3 - amazon.aws.tags a # Note: These examples do not set authentication details, see the AWS Guide for details. - community.aws.msk_cluster: name: kafka-cluster state: present version: 2.6.1 nodes: 6 ebs_volume_size: "{{ aws_msk_options.ebs_volume_size }}" subnets: - subnet-e3b48ce7c25861eeb - subnet-2990c8b25b07ddd43 - subnet-d9fbeaf46c54bfab6 wait: true wait_timeout: 1800 configuration_arn: arn:aws:kafka:us-east-1:123456789012:configuration/kafka-cluster-configuration/aaaaaaaa-bbbb-4444-3333-ccccccccc-1 configuration_revision: 1 - community.aws.msk_cluster: name: kafka-cluster state: absent a # These are examples of possible return values, and in general should use other names for return values. bootstrap_broker_string: description: A list of brokers that a client application can use to bootstrap. type: complex contains: plain: description: A string containing one or more hostname:port pairs. type: str tls: description: A string containing one or more DNS names (or IP) and TLS port pairs. type: str returned: I(state=present) and cluster state is I(ACTIVE) cluster_info: description: Description of the MSK cluster. type: dict returned: I(state=present) response: description: The response from actual API call. type: dict returned: always sample: {} N)camel_dict_to_snake_dict)AWSRetry)compare_aws_tags)AnsibleCommunityAWSModule)retriesdelaycd|jd}|j|jS)N list_clusters)ClusterNameFilter get_paginatorpaginatebuild_full_result)client cluster_name paginators m/home/dcms/DCMS/lib/python3.12/site-packages/ansible_collections/community/aws/plugins/modules/msk_cluster.pylist_clusters_with_backoffr%s/$$_5I     = O O QQcd|jd}|j|jS)N list_nodes) ClusterArnr )r cluster_arnrs rlist_nodes_with_backoffr+s/$$\2I     5 G G IIrcB t||jdg}r)t|dk7r|jd|d|dSiS#tjjtjj f$r}|j |dYd}~{d}~wwxYw)NClusterInfoListz$Failed to find kafka cluster by namez'Found more than one cluster with name ''msgr) rgetbotocore exceptions BotoCoreError ClientError fail_json_awslen fail_json)rmoduler cluster_listes rfind_cluster_by_namer-1sH1&,GKKL]_ab  |  !   #J<.XY!Z  [A I ))'' H Q FGG HsA 7BBBc |j|d}ddS#|jj$rYytjjtjj f$r}|j |dYd}~pd}~wwxYw)NTr aws_retryDELETEDz!Failed to get kafka cluster state ClusterInfoState)describe_clusterr$NotFoundExceptionr#r%r&r'rr*arnresponser,s rget_cluster_stater9@sE**cT*J M "7 ++    . .))'' E Q CDD EsB 6B .BB c |j|d}ddS#tjjtjjf$r}|j |dYd}~Vd}~wwxYw)NTr/z#Failed to get kafka cluster versionr2CurrentVersion)r4r#r$r%r&r'r6s rget_cluster_versionr<MszG**cT*J M "#3 44 ))'' G Q EFF Gs7A0A++A0c0tj}t|jjd}d} t |||}||k(rytj|z |kDr|j d|d|dtj |\)N wait_timeout<zTimeout waiting for cluster z (desired state is 'z')r )timeintparamsr"r9r)sleep)rr*r7statestarttimeoutcheck_interval current_states rwait_for_cluster_staterIXs IIKE&--##N34GN )&&#> E !  99;  (   #? Nbchbiik!l  m >" rc|jd|jd|jd|jdd|jd|jd|jdd d }|jd r?  &}}W5#]]95"MM/:  H}}&'C >O0P,QUV,V->?P-QR: &' 6}}\"%'!" == & * *< 8$fmmL&A,&O>H% &'9 :#MM,7 EII/[`a|4\BFF|UYZ= !"#89 }}%&+-'( ==) * . .| < N^@_@c@cdn@oD}}-.22<@!*D 1W }}-.22:>($/U 7;H+ ,V 4 ==) * . .| SX/YZ.0C0COUZ0[\ "H  Orci}|jdxsi}|jdrG|jddjd|jddjdd|d<nddi|d<|jd rG|jdd jd|jdd jd d |d <nddi|d <|jd rg|jdd jd|jdd jd|jdd jdd|d<nddi|d<dd|iiS)Nlogging cloudwatchenabled log_group)rrLogGroupCloudWatchLogsrrFfirehosedelivery_stream)rrDeliveryStreamFirehoses3bucketprefix)rrBucketPrefixS3 LoggingInfo BrokerLogsr)r*l_paramsrs rr~r~sjHmmI&,"G{{< }}Y/ =AA)L i0>BB;O& !" '0%7!"{{:}}Y/ ;?? J$mmI6zBFFGXY  !*51{{4}}Y/599)DmmI.t488BmmI.t488B  $U+ L(3 44rc d}i}t|||jd}|sbd}|jrdifSt|} |jd:ddi|}|jjdrt|||dd n|d|d<i|d <|d |jjd d|jjd id|dddd|jjddd|jjddgid|dd|jjdd|jjdid|dd|ddd|jjd|jjd dd!|jjd|jjd d"id|dd#|jjd$d%|jjd$id|d&|jjd'd(t|d)d*|d*it|d(t|d)d+|d+it|d(t|d)d,}|jD]\}} d-| vr|j!| d-s t#|| jd.d/|z} | d2| d3k7sKd}|jrdifcSt'|||d} t)|||d} | d k7r8|jdrt|||dd n|j+d4| d56  d:|d| d7| d8|d |<|jdst|||dd |t-|||dz}||fS#t j jt j jf$r}|j|dYd}~d}~wwxYw#t$$r!}|j|d0|d1Yd}~Ld}~wwxYw#t j jt j jf$r!}|j|d9|d1Yd}~d}~wwxYw);z/ Create new or update existing cluster FrKTr0zFailed to create kafka clusterNwaitrACTIVEr7rDchangesrZrRTargetNumberOfBrokerNodes) current_value target_value update_paramsr[rar_r`r^TargetBrokerEBSVolumeInfoAll)KafkaBrokerNodeId VolumeSizeGBrVrTTargetInstanceTypeCurrentBrokerSoftwareInfoConfigurationArnConfigurationRevision)r7revisionrMrNrYrOrXrLTargetKafkaVersionrrupdate_monitoring)rr update_methodrrr) broker_countbroker_storage broker_typecluster_configurationcluster_kafka_versionrrrbotocore_versionrupdate_z"There is no update method 'update_rrrz?Cluster can be updated only in active state, current state is 'z)'. check cluster state or use wait optionr rr;rz%Failed to update cluster via 'update_)r-rB check_modercreate_clusterr#r$r%r&r'r"rIr|r}r~itemsbotocore_at_leastgetattrAttributeErrorr<r9r)update_cluster_tags) rr*changedr8cluster create_paramsr,msk_cluster_changesmethodoptionsrrLrDs rcreate_or_update_clusterrs' GH"666==3HIG    8O.v6  F,v,,MtM}MH ==  V $ "66x 7MU] ^")!6  "))>!? & 1 1' :"=v}}?P?PQX?Y!Z "))>!? !NO_!`am!n & 1 12C D/.3V]]EVEVWhEij2""))>!?!O & 1 1/ B"6 8I8I/8Z![##>?@RS '(C DE\ ]" "==,,-@A & 1 12J K! (%}}001DE$*MM$5$56N$O*"&""))D!En!U & 1 1) <"6 8I8I)8T!U& "))=!> & 1 12G H!4!DV!L $#3G+BB$$8O.ffgl>ST)&&',:OPH$}}V,.vv7    - -x/B/B/N/N OBQ @AABsA;;7C2C  Ccz tdQidtdddtdddgdd td d td d td dtd ddtgdddtd ddtdddtddddtdttddtdttddtgd d!"#$#d%tdttdddtddtddtddd&'#d(tgd)d*d+d,tdttddtdd-#d.tdttdttddtdd/#tdttddtdd0#tdttddtddtdd1#2#d3tddd4td d5d6tdd7g8d9tdd}t|ddgd:ggd;}|jdkr|jd?@t|j dtt |j dzdAk7r|jdB@t |j ddCkDr2|j|jdD|j ddE@t||\}}n!|j ddk(rt||\}}i}i}jdFr|j ddk(rv |j|dFdGdH}|jdIdJk(rH|j|dFdG}|jdKr|dK|dL<|jdMr|dM|dN<|j'|t)|t)|Py#tjj tjj"f$r"}|j%|dO|dFYd}~|d}~wwxYw)RNrKstrT)typerequiredrDpresentabsent)rchoicesdefaultrL)rrMrNrArR)rrrT)kafka.t3.smallzkafka.m5.largezkafka.m5.xlargezkafka.m5.2xlargezkafka.m5.4xlargezkafka.m5.8xlargezkafka.m5.12xlargezkafka.m5.16xlargezkafka.m5.24xlargezkafka.m7g.largezkafka.m7g.xlargezkafka.m7g.2xlargezkafka.m7g.4xlargezkafka.m7g.8xlargezkafka.m7g.12xlargezkafka.m7g.16xlarger)rrr^drSlist)relementsr\F)rrrrbdictr)ri TLS_PLAINTEXT PLAINTEXTri)rjrh)rr)rdrgrn)rrr)rvrprqryr)r PER_BROKERPER_TOPIC_PER_BROKERPER_TOPIC_PER_PARTITIONr)rrrr)rrr)rr)rr)rrr)rrrrr>ir resource_tags)raliasesr)rLrMrNrS) argument_spec required_ifsupports_check_modekafka)retry_decoratorz.At least two client subnets should be providedr rz\The number of broker nodes must be a multiple of availability zones in the subnets parameter@zCluster name "z" exceeds 64 character limitrr/r2r3rBootstrapBrokerStringplainBootstrapBrokerStringTlstlsz)Can not obtain information about cluster )rr cluster_infor8r)rAnsibleAWSModulerrjittered_backoffrBr(r)rArrr"r4get_bootstrap_brokersr#r$r%r&r' exit_jsonr) module_argsr*rrr8rrbrokersr,s rmainr se ut ,e 8'UU; #'VT#B&*3Xbg&h   ?eXVeeLVe<6E: $&$ O  Yej!   ke~!vu="> eL $&% @"&EE"B $&% @(,%%(H $&% @#?#? MeBvu -CeDud3EeFv'8 9GeHVT2IeKN!y*opq F ]]7H4M4M4O] PF }}W* v}}Y' (1 ,   !Q  R v}}W% &Sy1I-J)K Kq P   r   v}}V$ % *   V--N6==QWCXBYYu2v-w x4VVD w 8 +*66:L ||L!fmmG&< &I !22h|>T`d2efstL(H4 66(?V7W+G4;;9:5<=W5X+E2  7-l;)(3     - -    + +    ;H\ansible_collections.community.aws.plugins.module_utils.modulesrrrrrr-r9r<rIrr|r}r~rrrr __name__rrrrs] ~ . 2  VPXx1A.R/R 1A.J/J  ,5 #=@ 58HV48Vr zFi  sBBB