Vh$s.dZddlmZddlZddlZddlZddlZddlZddl Z ddl m Z ddl m Z ddlmZddlmZddlmZdd lmZej,rdd lmZmZmZdd lmZdd lmZdd lm Z ddl!m"Z"ddl#m$Z$ddlm%Z%ddl&m'Z'ddl(m)Z)m*Z*m+Z+ddl,m-Z-ddl,m.Z.ddl/m0Z0ddl1m2Z2ddl3m4Z4ddl5m6Z6ddl7m8Z8ddl9m:Z;e6ZdZ?dZ@e-e.dd d!"e$#d-d$ZAd.d%ZB d/d&ZC d/d'ZD d/d(ZEd)ZFd*ZGd+ZHe d,ZIy)0z7Concrete collection candidate management helper module.) annotationsN)contextmanager)sha256)URLError) urldefrag)rmtree)mkdtemp) Candidate Collection Requirement) GalaxyToken)context) AnsibleError) get_collections_galaxy_meta_info)should_retry_error) _GALAXY_YAML) user_agent)to_bytes to_nativeto_text)retry_with_delays_and_condition)generate_jittered_backoff) get_bin_path) yaml_load)open_url)Display)Sentinelz MANIFEST.jsonceZdZdZddZedZedZedZedZ e jdZ d Z d Z d Z d Zd ZdZdZdZdZdZdZee ddZy)ConcreteArtifactsManageraManager for on-disk collection artifacts. It is responsible for: * downloading remote collections from Galaxy-compatible servers and direct links to tarballs or SCM repositories * keeping track of local ones * keeping track of Galaxy API tokens for downloads from Galaxy'ish as well as the artifact hashes * keeping track of Galaxy API signatures for downloads from Galaxy'ish * caching all of above * retrieving the metadata out of the downloaded artifacts Nc||_i|_i|_i|_i|_i|_||_i|_||_||_ ||_ ||_ d|_ y)z;Initialize ConcreteArtifactsManager caches and constraints.TN) _validate_certs_artifact_cache_galaxy_artifact_cache_artifact_meta_cache_galaxy_collection_cache_galaxy_collection_origin_cache_b_working_directory_supplemental_signature_cache_keyringtimeout_required_signature_count_ignore_signature_errors_require_build_metadata)selfb_working_directoryvalidate_certskeyringr*required_signature_countignore_signature_errorss c/home/dcms/DCMS/lib/python3.12/site-packages/ansible/galaxy/collection/concrete_artifact_manager.py__init__z!ConcreteArtifactsManager.__init__@sj .!&(#$&!(*%/1,$7!-/*  )A&(?%'+$c|jSN)r)r.s r4r1z ConcreteArtifactsManager.keyringQs }}r6c|jSr8)r+r9s r4#required_successful_signature_countz)r.values r4r?z/ConcreteArtifactsManager.require_build_metadatads (-$r6c $|jj} |j|d}|j|\}}d|j|j|j||||dS#t$r!}t dj ||d}~wwxYw)Nrz#The is no known source for {coll!s}collz1.0.0)format_version namespacenameversionserver version_url download_url signatures) src api_serverr%r&KeyError RuntimeErrorformatrFrGver)r. collectionrIrKsignatures_urlrLkey_errs r4get_galaxy_artifact_source_infoz8ConcreteArtifactsManager.get_galaxy_artifact_source_infois** 88DQGL)-)M)Mj)Y &NJ&#--OO!~~)($   5J'  s$A%% B.B  Bc H |j|S#t$rYnwxYw |j|\}}}n-#t$r!}tdj ||d}~wwxYwt j dj | t||j||j|}t jdj ||jxsd|j|jjnd n#t$r>}td j t|jt| |d}~wt $r>}td j t|jt| |d}~wwxYw||j|<|S)zGiven a Galaxy-stored collection, return a cached path. If it's not yet on disk, this method downloads the artifact first. z%There is no known source for {coll!s}rCNzFFetching a collection tarball for '{collection!s}' from Ansible GalaxyrS) expected_hashr0tokenz=Collection '{coll!s}' obtained from server {server!s} {url!s}Galaxy)rDrIurlGFailed to download collection tar from '{coll_src!s}': {download_err!s}coll_src download_errzmFailed to download collection tar from '{coll_src!s}' due to the following unforeseen error: {download_err!s})r#rOr%rPrQdisplayvvvv_download_filer'r!vvvrMrNrrr Exception)r.rSr] sha256_hashrZrUb_artifact_patherrs r4get_galaxy_artifact_pathz1ConcreteArtifactsManager.get_galaxy_artifact_paths  ..z: :    &*&C&CJ&O #Ce 7J'     #VzV: # ,)))#33 O6 KK,,2F#JNN,Fh5?^^5O 11-3- ) 8&z~~6!*3  #&z~~6!*3   (3B##J/sD 5 AAA$D F 9E F9F  Fc 6 |j|jS#t$rYnwxYw|jrtj dj |j t|j|jd|j|j}n#t$r>}tdj t|jt||d}~wwxYw|jr,t!|j|j"|j}n_|j$s|j&s |j(rt+|j}n%t-dj |j.||j|j<|S) zGiven a concrete collection pointer, return a cached path. If it's not yet on disk, this method downloads the artifact first. zBCollection requirement '{collection!s}' is a URL to a tar artifactrXN)rYr0r*r^r_z2The artifact is of an unexpected type {art_type!s})art_type)r"rMrOis_urlrbrcrQfqcnrdr'r!r*rfrris_scm_extract_collection_from_gitrRis_fileis_dir is_subdirsrrPtype)r.rSrhris r4get_artifact_pathz*ConcreteArtifactsManager.get_artifact_pathsj  '' 7 7       LL$$*FjooF$F  "0NN--"&#'#7#7 LL # "<F!*:>>!:%.s^   :))O   :#4#4 8M8M&z~~6OD 0  0?Z^^,s$ ''&8B C&(9C!!C&c^|jr|j|S|j|Sr8)is_concrete_artifactrurjr.rSs r4get_artifact_path_from_unknownz7ConcreteArtifactsManager.get_artifact_path_from_unknowns-  * *))*5 5,,Z88r6c*|j|dS)NrFget_direct_collection_metarxs r4 _get_direct_collection_namespacez9ConcreteArtifactsManager._get_direct_collection_namespaces..z:;GGr6c*|j|dS)NrGr{rxs r4_get_direct_collection_namez4ConcreteArtifactsManager._get_direct_collection_name s..z:6BBr6c~|jrydj|j||j|fS)zExtract FQCN from the given on-disk collection artifact. If the collection is virtual, ``None`` is returned instead of a string. N.) is_virtualjoinr}rrxs r4get_direct_collection_fqcnz3ConcreteArtifactsManager.get_direct_collection_fqcn sB  xx  1 1* =  , ,Z 8   r6c*|j|dS)z;Extract version from the given on-disk collection artifact.rHr{rxs r4get_direct_collection_versionz6ConcreteArtifactsManager.get_direct_collection_versions..z:9EEr6c6|j|d}|i}|S)z8Extract deps from the given on-disk collection artifact. dependenciesr{)r.rScollection_dependenciess r4"get_direct_collection_dependenciesz;ConcreteArtifactsManager.get_direct_collection_dependencies"s,#'"A"A*"Mn"] " *&( #&&r6cr |j|jS#t$r|j|}YnwxYw|js |j r t |}n|jrN t||j}n#t$r*}tdjt||d}~wwxYw|jrddt|didd}nF|jr4ddt j#t%t|j&ddd}nt(||j|j<|S)z8Extract meta from the given on-disk collection artifact.z/Failed to find the collection dir deps: {err!s})riN*)rGrFrrH)r$rMrOrurmrq_get_meta_from_tarrr_get_meta_from_dirr? LookupErrorrrQrrorsdictfromkeysmapnamespace_collection_pathsrP)r.rSrhcollection_meta lookup_errs r4r|z3ConcreteArtifactsManager.get_direct_collection_meta*s7 A,,Z^^< < A"44Z@O A    2 20AO    ""4_dFaFa"b ""EFy4F5"" "   !!*?!;S A O  " "! $  :#H#HI! O 4C!!*..1s$88,B B6 %B11B6cJ|||f|j|<||f|j|<y)zStore collection URL, SHA256 hash and Galaxy API token. This is a hook that is supposed to be called before attempting to download Galaxy-based collections with ``get_galaxy_artifact_path()``. N)r%r&)r.rSr]rgrZrTrLs r4save_collection_sourcez/ConcreteArtifactsManager.save_collection_sourceUs058e4K%%j1;I:;U,,Z8r6c#Ktt|d}t|d} ||||||t|y#t|wxYww)aCustom ConcreteArtifactsManager constructor with temp dir. This method returns a context manager that allocates and cleans up a temporary directory for caching the collection artifacts during the dependency resolution process. surrogate_or_stricterrors)dir)r1r2r3N)r rr) cls temp_dir_baser0r1r2r3r? temp_path b_temp_paths r4 under_tmpdirz%ConcreteArtifactsManager.under_tmpdir_s_&/DE y1FG  )A(?   ; F; s%AA A AA)TN<NN)TNNNT)__name__ __module__ __qualname____doc__r5propertyr1r;r3r?setterrVrjruryr}rrrrr|r classmethodrrr6r4rr3s ,"..-- ,,""-#- 0>@<|9 HC F ')VV %)$(#'  r6rcd|vr|jdd\}}n |dk(s|sd}|jdr|dd}n|}t|\}}|jtj j }|jtj j dzr-|jtj j d }n5d |vrd |vr|}n*|jd d }|jdr|dd}||||fS)z>Extract name, version, path and subdir out of the SCM pointer.,rHEADzgit+Nz.gitz://@/)split startswithrstripospathsependswith)rSrHrfragmentrGs r4 parse_scmrs j(..sA6 G CwV$!"~t_ND(~~bggkk*H }}RWW[[6)*zz"''++&r* d s$zz#r" == 9D $ ((r6c tt||\}}}}t|t|d} td}|dk(r|dd |t|g} n|d|t|g} tjd stjr| jd d g tj| |dt|f} tj| ||r)t"j$j'|t|S|S#t$r*} t dj t|| d} ~ wwxYw#tj $r*} t d j t|| d} ~ wwxYw#tj $r4} t dj t|t|| d} ~ wwxYw)Nrrrprefixgitz_Could not find git executable to extract the collection from the Git repository `{repo_url!s}`.)repo_urlrclonez --depth=1 ignore_certsz-czhttp.sslVerify=falsez5Failed to clone a Git repository from `{repo_url!s}`.checkout)cwdz\Failed to switch a cloned Git repo `{repo_url!s}` to the requested revision `{commitish!s}`.) commitishr)rr rr ValueErrorrrQrrrCLIARGSCGALAXY_IGNORE_CERTSextend subprocess check_callCalledProcessErrorrrr) rcoll_verb_pathrGrHgit_urlrb_checkout_pathgit_executableri git_clone_cmdproc_errgit_switch_cmds r4rprps'08'D$D'7H %:;O %e,&'+wP_H`a ''7?;ST ~&!*?*?d$:;<m,$Z1AAN n/B   _hx&89(K  m FIg.F /  $  ( ( C FIg.F /    ( ( 9 F#G,"7+    sG C:D06E0: D-%D((D-0E-%E((E-0F7/F22F7()retries delay_basedelay_threshold)backoff_iteratorrct|jdddd}|dtd }t|dj |df}t j j ||} tjd |d t| tt|d||dn|jd gt| } t| d5} t!| | } ddd|r6tj#dj% ||| k7r td| S#t$r} tt| | d} ~ wwxYw#1swYjxYw)Nrrrrz.tar.gz-r6rz Downloading z to Authorization)r0headersunredirected_headers http_agentr*)orig_excwb)write_tozTValidating downloaded file hash {actual_hash!s} with expected hash {expected_hash!s}) actual_hashrYz+Mismatch artifact hash with downloaded file)rrsplitlenr rrrrbrrrrrrfropen _consume_filercrQ)r]rrYr0rZr*b_tarball_name b_file_name b_tarball_dir b_file_pathrespri download_filers r4rdrdsR 31&;N!!13y>/2K yy+s+,M '',,}n=K OOc7=3IJK 9 c"7 8)!MDu}}"1!2z|   k4 BM#D=A B  . F{-F H K 'LM M  99S>C889BBs$7D&E & E /EE  Ecd}t}|j|}|rH|!|j||j|j ||j|}|rH|j S)Ni)rreadwriteflushupdate hexdigest) read_fromrbufsize sha256_digestdatas r4rrsmGHM >>' "D    NN4 NN T"~~g&   " " $$r6ct}t}t}t}t}t}|D]V} | jddr|j| d||||d| jdd} | j| dXt ||z|z|z|z} t|j } |j | } | rVdt|dd jt| }|r t|tj|t|| j | }t|d kDr3tjd t|d d j||D] }||vsd||<|D]3}|j|d}|g||<t!|t"r.|g||<5|D] }||vsi||<|D]}||vst$||<|jd sd|d <|S)NrequiredFkey)strlistrsentinelrtrThe collection galaxy.yml at 'z+' is missing the following mandatory keys: z, rz0Found unknown keys in collection galaxy.yml at 'z': rHr)rsetgetadd frozensetkeys differencerrsortedrrbwarningrrr isinstancerr) galaxy_ymlb_galaxy_yml_pathr?galaxy_yml_schemamandatory_keys string_keys list_keys dict_keys sentinel_keysinfo key_list_typeall_keysset_keys missing_keysmsg extra_keysoptional_string optional_listlist_val optional_dictoptional_sentinels r4_normalize_galaxy_yml_manifestr!sE )* UN%KIIEM! ' 88J &   tE{ +%  ((65 ! # $u+& '+5 AIMP]]^H:??$%H!,,X6L*+TYYvl7K-L N  "s# #o$$X.J :"#45tyy7LN O'/ * ,*.J '/#3 >>-6  (*J} %Hd+)1 J} % 3#+  *(*J} %++5 J .,4J( )5 >>) $ # 9 r6cP t|S#t$rt||cYSwxYwr8)_get_meta_from_installed_dirr_get_meta_from_src_dir)rr?s r4rr`s2F+F33 F%f.DEEFs %%c tjj|t}tjj |s$t dj t|t|d5} t|} dddttsT|rtdt|dt j#dt|dt%dt|dt'|||S#tjj$r4}tdj t|t||d}~wwxYw#1swYxYw)Nz9The collection galaxy.yml path '{path!s}' does not exist.rrbzRFailed to parse the galaxy.yml at '{path!s}' with the following error: {err_txt!s})rerr_txtrz' is incorrectly formatted.)rrrrisfilerrQrrryamlerror YAMLErrorrr rrbr rr!)rr?r manifest_file_objmanifestyaml_errs r4r$r$jsCfl3J 77>>* % G F *-F .  j$  #4  !23H  h % !!? *@U?VVqrs s8:9N8OOjkl9)J:O9PPklmm )(J@V WW#zz## 4":.%h/    s*4E6 C;;E /EE  EEc tjj|t|d} t |d5}|j }dddtd} tj|}|S#1swY.xYw#t tf$r&tdj|t|wxYw#t$rtdj|wxYw)Nrrr'z;The collection {manifest!s} path '{path!s}' does not exist.)r.rKCollection tar file member {member!s} does not contain a valid json string.member)rrrrrrIOErrorOSErrorrrQrrjsonloadsrr)rfilenameb_json_filepath manifest_fd b_json_text manifest_txtr.s r4_get_json_from_installed_dirr=s ggll68HEZ+[\O  /4 ( -K%**,K -;/DEL ::l+ O- - - W   I F!/        + F(F #   s. BA7 B B;7B<B5B8;%C c t|t}|d}|jd}|s>& !:9V;L:MR P     f % H F &)F *  f3 '!/> #--.?@Fnf 5 /9N':!"((.6G(H #!,L  ::l3-.) / /!/!/ 3"6*"3  0 "3F"3F4  / / /!/!/!/sUF D F+AE:,EE: F3E  F)E77E::F ?FFc#K|j|} ||f||jyy#||jwwxYwwr8) extractfileclose)tarr3tar_objs r4rLrLsM oof%Ggo   MMO 7  MMO sA/AAA)Nrr8)T)Jr __future__rr6rrIrtypingtr* contextlibrhashlibr urllib.errorr urllib.parsershutilrtempfiler TYPE_CHECKING0ansible.galaxy.dependency_resolution.dataclassesr r r ansible.galaxy.tokenr ansibleransible.errorsransible.galaxyransible.galaxy.apirransible.galaxy.user_agentr+ansible.module_utils.common.text.convertersrrransible.module_utils.apirr#ansible.module_utils.common.processr ansible.module_utils.common.yamlransible.module_utils.urlsransible.utils.displayransible.utils.sentinelransible.constants constantsrrbrBrrrprdrr!rr$r=r#rrLrr6r4rps>"  %!"??1';1I0TTD><6.)+ )#L L ^ )61j!.qQXZ[)) )X %$ $Lb $F $XDB*//d  r6