Vh>dZdZdZ ddlZddlmZddlmZddlm Z ddl m Z dd l m Z dd lmZdd lmZd Zd ZdZdZdZdZdZdZedk(reyy#e$rYWwxYw)a --- module: glue_crawler version_added: 4.1.0 short_description: Manage an AWS Glue crawler description: - Manage an AWS Glue crawler. See U(https://aws.amazon.com/glue/) for details. - Prior to release 5.0.0 this module was called C(community.aws.aws_glue_crawler). The usage did not change. author: - 'Ivan Chekaldin (@ichekaldin)' options: database_name: description: - The name of the database where results are written. type: str description: description: - Description of the crawler being defined. type: str name: description: - The name you assign to this crawler definition. It must be unique in your account. required: true type: str recrawl_policy: description: - A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run. suboptions: recrawl_behavior: description: - Specifies whether to crawl the entire dataset again or to crawl only folders that were added since the last crawler run. - Supported options are C(CRAWL_EVERYTHING) and C(CRAWL_NEW_FOLDERS_ONLY). type: str type: dict role: description: - The name or ARN of the IAM role associated with this crawler. - Required when I(state=present). type: str schema_change_policy: description: - The policy for the crawler's update and deletion behavior. suboptions: delete_behavior: description: - Defines the deletion behavior when the crawler finds a deleted object. - Supported options are C(LOG), C(DELETE_FROM_DATABASE), and C(DEPRECATE_IN_DATABASE). type: str update_behavior: description: - Defines the update behavior when the crawler finds a changed schema.. - Supported options are C(LOG) and C(UPDATE_IN_DATABASE). type: str type: dict state: description: - Create or delete the AWS Glue crawler. required: true choices: [ 'present', 'absent' ] type: str table_prefix: description: - The table prefix used for catalog tables that are created. type: str targets: description: - A list of targets to crawl. See example below. - Required when I(state=present). type: dict extends_documentation_fragment: - amazon.aws.common.modules - amazon.aws.region.modules - amazon.aws.tags - amazon.aws.boto3 a # Note: These examples do not set authentication details, see the AWS Guide for details. # Create an AWS Glue crawler - community.aws.glue_crawler: name: my-glue-crawler database_name: my_database role: my-iam-role schema_change_policy: delete_behavior: DELETE_FROM_DATABASE update_behavior: UPDATE_IN_DATABASE recrawl_policy: recrawl_ehavior: CRAWL_EVERYTHING targets: S3Targets: - Path: "s3://my-bucket/prefix/folder/" ConnectionName: my-connection Exclusions: - "**.json" - "**.yml" state: present # Delete an AWS Glue crawler - community.aws.glue_crawler: name: my-glue-crawler state: absent a creation_time: description: The time and date that this crawler definition was created. returned: when state is present type: str sample: '2021-04-01T05:19:58.326000+00:00' database_name: description: The name of the database where results are written. returned: when state is present type: str sample: my_table description: description: Description of the crawler. returned: when state is present type: str sample: My crawler last_updated: description: The time and date that this crawler definition was last updated. returned: when state is present type: str sample: '2021-04-01T05:19:58.326000+00:00' name: description: The name of the AWS Glue crawler. returned: always type: str sample: my-glue-crawler recrawl_policy: description: A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run. returned: when state is present type: complex contains: RecrawlBehavior: description: Whether to crawl the entire dataset again or to crawl only folders that were added since the last crawler run. returned: when state is present type: str sample: CRAWL_EVERYTHING role: description: The name or ARN of the IAM role associated with this crawler. returned: when state is present type: str sample: my-iam-role schema_change_policy: description: The policy for the crawler's update and deletion behavior. returned: when state is present type: complex contains: DeleteBehavior: description: The deletion behavior when the crawler finds a deleted object. returned: when state is present type: str sample: DELETE_FROM_DATABASE UpdateBehavior: description: The update behavior when the crawler finds a changed schema. returned: when state is present type: str sample: UPDATE_IN_DATABASE table_prefix: description: The table prefix used for catalog tables that are created. returned: when state is present type: str sample: my_prefix targets: description: A list of targets to crawl. returned: when state is present type: complex contains: S3Targets: description: List of S3 targets. returned: when state is present type: list JdbcTargets: description: List of JDBC targets. returned: when state is present type: list MongoDBTargets: description: List of Mongo DB targets. returned: when state is present type: list DynamoDBTargets: description: List of DynamoDB targets. returned: when state is present type: list CatalogTargets: description: List of catalog targets. returned: when state is present type: list N)camel_dict_to_snake_dict)snake_dict_to_camel_dict)is_boto3_error_code)get_aws_account_info)AWSRetry)compare_aws_tags)AnsibleCommunityAWSModulec |jd|dS#td$rYytjjtjj f$r}|j |Yd}~yd}~wwxYw)zK Get an AWS Glue crawler based on name. If not found, return None. T aws_retryNameCrawlerEntityNotFoundExceptionN) get_crawlerrbotocore exceptions ClientError BotoCoreError fail_json_aws) connectionmoduleglue_crawler_namees n/home/dcms/DCMS/lib/python3.12/site-packages/ansible_collections/community/aws/plugins/modules/glue_crawler.py_get_glue_crawlerrsw %%;L%MiXX 8 9''))   Q  sA96A9A44A9c>|Dcgc] }t|c}Scc}w)N) _trim_target)targetsts r _trim_targetsr s%, -LO -- -sct|sy|j}|jdds|jdd|S)z Some target types have optional parameters which AWS will fill in and return To compare the desired targets and the current targets we need to ignore the defaults N Exclusions)copygetpop)targetretvals rrrs6  [[]F ::lD ) <& Mc0d|vr |d|dk7ryd|vr |d|dk7ryd|vr |d|dk7ryd|vr |d|dk7ryd|vr |d|dk7ryd|vr |d|dk7ryd|vrd |dvr$t|dd t|dd k7ryd |dvr5|dd |dd k7r$t|dd t|dd k7ryd |dvr|dd |dd k7ryd |dvr|dd |dd k7ryd |dvr|dd |dd k7ryy)zj Compare Glue crawler params. If there is a difference, return True immediately else return False DatabaseNameT Description RecrawlPolicyRoleSchemaChangePolicy TablePrefixTargets S3Targets JdbcTargetsMongoDBTargetsDynamoDBTargetsCatalogTargetsF)r ) user_paramscurrent_paramss r_compare_glue_crawler_paramsr8s$^)DWeHf)f # M(BnUbFc(c+%+o*F.YhJi*i V!4v8N!N + , -@T1U U # M(BnUbFc(cK +i0 0[3K@A]y)+6F [3 3I&}5 9RS`9aa[3MBC}y)-8H  I 6 6I&'78N9F  F7H /HH 7I:/I55I:c d}t}|jjd|d<|jjd|d<|jjd|d<|jjd|jjd|d <|jjd |jjd |d <|jjd )t|jjd d|d<|jjd|jjd|d<|jjd)t|jjdd|d<|jjd|jjd|d<|jjd|jjd|d<|r0t ||rG |j s|j dddi|d}n# |j s|jdddi|d}t|||d}|t|||z}|jdd|it!|xsigdy #tjjtjjf$r}|j|Yd }~d }~wwxYw#tjjtjjf$r}|j|Yd }~d }~wwxYw)z. Create or update an AWS Glue crawler Fr<r roler-rr0 database_nameNr* descriptionr+recrawl_policyT)capitalize_firstr,schema_change_policyr. table_prefixr/r rJ)r.r,r0) ignore_list)dictrCr$rr8rFupdate_crawlerrrrrrcreate_crawlerrrQ exit_jsonr)rrrIrJrCrs rcreate_or_update_glue_crawlerr`Us G VF]]&&v.F6N]]&&v.F6N )))4F9 }})5!'!2!2?!C~ }}'3 & 1 1- @} }})*6":6==;L;LM];^qu"v }} ,**62v }}/0<'? MM  4 5( #$}}(4 & 1 1. A} }}#/"MM--i8y ' = (((-J--GGG $$$) ))CDCFCG%ZHL {:v|<? F vx7P7PY[7\]J MM  g &E$Z9J9J69RSL %j&,GJ =r(__main__) DOCUMENTATIONEXAMPLESRETURNr ImportError0ansible.module_utils.common.dict_transformationsrransible_collections.community.aws.plugins.module_utils.modulesr r{rr rr8rQr`rcr~__name__r[r(rrsK Z 8W r VU\XPXx  . 1h%P2j&" >F zFg  sAA$#A$