VhTHdZdZdZddlZ ddlZddlmZddlm Z ddl m Z ddl m Z dd lmZdd lmZd Zd Zd ZdZdZdZedk(reyy#e$rYKwxYw)a --- module: glue_job version_added: 1.0.0 short_description: Manage an AWS Glue job description: - Manage an AWS Glue job. See U(https://aws.amazon.com/glue/) for details. - Prior to release 5.0.0 this module was called C(community.aws.aws_glue_job). The usage did not change. author: - "Rob White (@wimnat)" - "Vijayanand Sharma (@vijayanandsharma)" options: allocated_capacity: description: - The number of AWS Glue data processing units (DPUs) to allocate to this Job. From 2 to 100 DPUs can be allocated; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. type: int command_name: description: - The name of the job command. This must be 'glueetl'. default: glueetl type: str command_python_version: description: - Python version being used to execute a Python shell job. - AWS currently supports C('2') or C('3'). type: str version_added: 2.2.0 command_script_location: description: - The S3 path to a script that executes a job. - Required when I(state=present). type: str connections: description: - A list of Glue connections used for this job. type: list elements: str default_arguments: description: - A dict of default arguments for this job. You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes. type: dict description: description: - Description of the job being defined. type: str glue_version: description: - Glue version determines the versions of Apache Spark and Python that AWS Glue supports. type: str version_added: 1.5.0 max_concurrent_runs: description: - The maximum number of concurrent runs allowed for the job. The default is 1. An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit. type: int max_retries: description: - The maximum number of times to retry this job if it fails. type: int name: description: - The name you assign to this job definition. It must be unique in your account. required: true type: str number_of_workers: description: - The number of workers of a defined workerType that are allocated when a job runs. type: int version_added: 1.5.0 role: description: - The name or ARN of the IAM role associated with this job. - Required when I(state=present). type: str state: description: - Create or delete the AWS Glue job. required: true choices: [ 'present', 'absent' ] type: str timeout: description: - The job timeout in minutes. type: int worker_type: description: - The type of predefined worker that is allocated when a job runs. - Support for instance types C(G.4X( and C(G.8X) was added in community.aws release 7.2.0. choices: [ 'Standard', 'G.1X', 'G.2X', 'G.4X', 'G.8X' ] type: str version_added: 1.5.0 notes: - Support for I(tags) and I(purge_tags) was added in release 2.2.0. extends_documentation_fragment: - amazon.aws.common.modules - amazon.aws.region.modules - amazon.aws.tags - amazon.aws.boto3 a # Note: These examples do not set authentication details, see the AWS Guide for details. # Create an AWS Glue job - community.aws.glue_job: command_script_location: "s3://s3bucket/script.py" default_arguments: "--extra-py-files": s3://s3bucket/script-package.zip "--TempDir": "s3://s3bucket/temp/" name: my-glue-job role: my-iam-role state: present # Delete an AWS Glue job - community.aws.glue_job: name: my-glue-job state: absent a allocated_capacity: description: The number of AWS Glue data processing units (DPUs) allocated to runs of this job. From 2 to 100 DPUs can be allocated; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. returned: when state is present type: int sample: 10 command: description: The JobCommand that executes this job. returned: when state is present type: complex contains: name: description: The name of the job command. returned: when state is present type: str sample: glueetl script_location: description: Specifies the S3 path to a script that executes a job. returned: when state is present type: str sample: mybucket/myscript.py python_version: description: Specifies the Python version. returned: when state is present type: str sample: 3 connections: description: The connections used for this job. returned: when state is present type: dict sample: "{ Connections: [ 'list', 'of', 'connections' ] }" created_on: description: The time and date that this job definition was created. returned: when state is present type: str sample: "2018-04-21T05:19:58.326000+00:00" default_arguments: description: The default arguments for this job, specified as name-value pairs. returned: when state is present type: dict sample: "{ 'mykey1': 'myvalue1' }" description: description: Description of the job being defined. returned: when state is present type: str sample: My first Glue job glue_version: description: Glue version. returned: when state is present type: str sample: 2.0 job_name: description: The name of the AWS Glue job. returned: always type: str sample: my-glue-job execution_property: description: An ExecutionProperty specifying the maximum number of concurrent runs allowed for this job. returned: always type: complex contains: max_concurrent_runs: description: The maximum number of concurrent runs allowed for the job. The default is 1. An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit. returned: when state is present type: int sample: 1 last_modified_on: description: The last point in time when this job definition was modified. returned: when state is present type: str sample: "2018-04-21T05:19:58.326000+00:00" max_retries: description: The maximum number of times to retry this job after a JobRun fails. returned: when state is present type: int sample: 5 name: description: The name assigned to this job definition. returned: when state is present type: str sample: my-glue-job role: description: The name or ARN of the IAM role associated with this job. returned: when state is present type: str sample: my-iam-role timeout: description: The job timeout in minutes. returned: when state is present type: int sample: 300 N)camel_dict_to_snake_dict)is_boto3_error_code)get_aws_account_info)AWSRetry)compare_aws_tags)AnsibleCommunityAWSModulec |jd|dS#td$rYytjjtjj f$r}|j |Yd}~yd}~wwxYw)a Get an AWS Glue job based on name. If not found, return None. :param connection: AWS boto3 glue connection :param module: Ansible module :param glue_job_name: Name of Glue job to get :return: boto3 Glue job dict or None if not found T aws_retryJobNameJobEntityNotFoundExceptionN)get_jobrbotocore exceptions ClientError BotoCoreError fail_json_aws) connectionmodule glue_job_namees j/home/dcms/DCMS/lib/python3.12/site-packages/ansible_collections/community/aws/plugins/modules/glue_job.py _get_glue_jobrsv !!D-!HOO 8 9''))   Q  sA96A9A44A9cd|vrd|d<d|vr t|d<d|vr |d|dk7ryd|vr$|dd|ddk7ry|dd|ddk7ryd |vr |d |d k7ryd|vr |d|dk7ryd|vr |d|dk7ryd |vr|d d |d d k7ryd |vr |d |d k7ryd |vr |d |d k7ryd|vr |d|dk7ryd|vr |d|dk7ryd |vr |d |d k7ryd|vr |d|dk7ryd|vr |d|dk7ryy)a. Compare Glue job params. If there is a difference, return True immediately else return False :param user_params: the Glue job parameters passed by the user :param current_params: the Glue job parameters currently configured :return: True if any parameter is mismatched else False DescriptionDefaultArgumentsAllocatedCapacityTCommandScriptLocation PythonVersion ConnectionsExecutionPropertyMaxConcurrentRuns GlueVersion MaxRetriesRoleTimeout WorkerTypeNumberOfWorkersF)dict) user_paramscurrent_paramss r_compare_glue_job_paramsr/sN*(*}%/-1V)*k)k:M.NR`atRu.uK y !"2 3~i7PQa7b b y !/ 2nY6OP_6` ` # M(BnUbFc(c[([9K-LP^_qPr-r # M(BnUbFc(c{* + ,-@ A - ./B C D # M(BnUbFc(c{"{<'@NS_D`'` V!4v8N!NKK $:nY>W$W # M(BnUbFc(c{"{<'@NS_D`'`K'K8I,Jn]nNo,o c d}|jjdyt|\}}d|d|jd|d|jjd} |j d| jd i}t|jjd|jjd \} } | r#d}|js |jd|| | r%d}|js |jd|| |S|S#t j jt j jf$rI}|jri}n/|j|d |jjd Yd}~ d}~wwxYw#t j jt j jf$r:}|j|d|jjd Yd}~d}~wwxYw#t j jt j jf$r:}|j|d|jjd Yd}~|Sd}~wwxYw)NFtagszarn:z:glue::z:job/nameT)r ResourceArnTagsz Unable to get tags for Glue job )msg purge_tags)r r5 TagsToRemovez Unable to set tags for Glue job )r r5 TagsToAdd)paramsgetrregionget_tagsrrrr check_moderruntag_resource tag_resource) rrglue_jobchanged account_id partitionarn existing_tagsr tags_to_addtags_to_removes r ensure_tagsrJ=sYG }} (08J  6&--*U6==CTCTU[C\B] ^Ch"++d+LPPQWY[\ #3v}}((0&--2C2CL2Q#K   l))DcXf)g   l''$CS^'_ N7N5    + +X-@-@-N-N Oh   M  *J6==K\K\]cKdJe(f g h''33X5H5H5V5VW l$$Q.Nv}}O`O`agOhNi,j$kk l''33X5H5H5V5VW l$$Q.Nv}}O`O`agOhNi,j$kk N lsH#DF:H 7F >F  F7H /HH 7I:/I55I:cl d}t}|jjd|d<|jjd|d<|jjd|jjd|d<|jjd v|jjd |jjd d |d <|jjd !|jjd |d d<|jjd d|jjdi|d<|jjd|jjd|d<|jjd|jjd|d<|jjd|jjd|d<|jjd d|jjdi|d<|jjd|jjd|d<|jjd|jjd|d<|jjd|jjd|d<|jjd|jjd|d<|jjd |jjd |d!<|rQt||rh |dt j |d"}|d#d=|j s|jd(d$d%i|d%}n# |j s|jd(d$d%i|d%}t|||d}|t|||z}|j d(d&|it#|xsidg'y#tjjtjjf$r}|j|Yd}~d}~wwxYw#tjjtjjf$r}|j|Yd}~d}~wwxYw))z Create or update an AWS Glue job :param connection: AWS boto3 glue connection :param module: Ansible module :param glue_job: a dict of AWS Glue job parameters or None :return: Fr4Nameroler(allocated_capacityNrcommand_script_location command_name)rLr!r command_python_versionr" connectionsr#default_argumentsr descriptionr glue_versionr&max_concurrent_runsr%r$ max_retriesr'timeoutr) worker_typer*number_of_workersr+)r JobUpdater[r TrC) ignore_list)r,r;r<r/copydeepcopyr? update_jobrrrrr create_jobrrJ exit_jsonr)rrrBrCr; update_paramsrs rcreate_or_update_glue_jobrdes&G VF]]&&v.F6N]]&&v.F6N }}-.:&,mm&7&78L&M"# }}23?MM%%n5$mm//0IJ y ==  5 6 B171B1BC[1\F9 o . }}'3!. 0A0A-0P Q} }},-9%+]]%6%67J%K!" }}'3 & 1 1- @} }}(4 & 1 1. A} }}./;':FMMN%OG W%##//1D1D1R1RS $   # # $s$=7B4B  Bc @td*idtddtdddtdd tdd td d d tddtddtddtddtddtdddtddtdddtddtdddgddtddg d!tdd"tgd#d$}t|dddd gfgd%}tjd&'}|j d(|)}|j j d}t|||j j d}|dk(rt|||yt|||y)+NrNint)typerPstrglueetl)rjdefaultrQrOrRlist)rjelementsrSr,rTrUrVrWr4T)requiredrjrZr8boolrMstatepresentabsent)rpchoicesrjr2 resource_tags)rjaliasesrXrY)StandardzG.1XzG.2XzG.4XzG.8X)rurj) argument_spec required_ifsupports_check_mode )retriesglue)retry_decoratorr]) r,AnsibleAWSModulerjittered_backoffclientr;r<rrdrg)ryrrrrrrBs rmainrsU+ui8 $/!%% 0  fu5  F+ e$u%!e,e$4e ,E*VT2u D9h*?eL v'8 9!"% #$!MTYZ%M*#y63L*MNO F //;OvGJ MM  g &EZ1B1B61JKH !*fh? FH5r0__main__) DOCUMENTATIONEXAMPLESRETURNr^r ImportError0ansible.module_utils.common.dict_transformationsransible_collections.community.aws.plugins.module_utils.modulesrrrr/rJrdrgr__name__r]r0rrsf P &_ B  V\XPXx (2j%PDtN&,&6R zFI  sAAA