BVh_~ dZddlZddlZddlZddlZddlZddlmZGddeZ GddejdgdZ Gd d ejZ Gd d eZGd deZGddej ZdZy)aControl flow graph (CFG) structure for Python AST representation. The CFG is a digraph with edges representing valid control flow. Each node is associated with exactly one AST node, but not all AST nodes may have a corresponding CFG counterpart. Once built, the CFG itself is immutable, but the values it holds need not be; they are usually annotated with information extracted by walking the graph. Tip: Use `Graph.as_dot` to visualize the CFG using any DOT viewer. Note: the CFG tries to include all code paths that MAY be taken, with a single notable exception: * function calls do not generate edges corresponding to exceptions they may raise (i.e. a function call in the middle of a block does not return or jump to any except or finally block) TODO(mdan): Consider adding the edges above. They'd only add ~O(n) edges. TODO(mdan): Alternatively, consider adding an edge from try to all its excepts. N)annoc"eZdZdZdZdZdZy)NodeaA node in the CFG. Although new instances of this class are mutable, the objects that a user finds in the CFG are typically not. The nodes represent edges in the CFG graph, and maintain pointers to allow efficient walking in both forward and reverse order. The following property holds for all nodes: "child in node.next" iff "node in child.prev". Attributes: next: FrozenSet[Node, ...], the nodes that follow this node, in control flow order prev: FrozenSet[Node, ...], the nodes that precede this node, in reverse control flow order ast_node: ast.AST, the AST node corresponding to this CFG node c.||_||_||_yN)nextprevast_node)selfnext_r r s T/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/autograph/pyct/cfg.py__init__z Node.__init__CsDIDIDMct|j|_tj|j|_yr) frozensetrweakrefWeakSetr r s r freezez Node.freezeHs($))$DI *DIrct|jtjrd|jjzSt|jtj rd|jjzSt|jtj r7tj|jjjStj|jjS)Nzdef %szclass %s) isinstancer gast FunctionDefnameClassDefwithitem astunparseunparse context_exprstriprs r __repr__z Node.__repr__Os$--!1!12  ** ** DMM4== 1 $--,, ,, DMM4== 1    : : ; A A CC   dmm , 2 2 44rN)__name__ __module__ __qualname____doc__rrr!rr rr1s" +5rrceZdZdZdZdZy)GraphaRA Control Flow Graph. The CFG maintains an index to allow looking up a CFG node by the AST node to which it is associated. The index can also be enumerated in top-down, depth first order. Walking the graph in forward or reverse order is supported by double parent-child links. Note: the error nodes are not wired to their corresponding finally guards, because these are shared, and wiring them would create a reverse path from normal control flow into the error nodes, which we want to avoid. The graph also maintains edges corresponding to higher level statements like for-else loops. A node is considered successor of a statement if there is an edge from a node that is lexically a child of that statement to a node that is not. Statement predecessors are analogously defined. Attributes: entry: Node, the entry node exit: FrozenSet[Node, ...], the exit nodes error: FrozenSet[Node, ...], nodes that exit due to an explicitly raised error (errors propagated from function calls are not accounted) index: Dict[ast.Node, Node], mapping AST nodes to the respective CFG node stmt_prev: Dict[ast.Node, FrozenSet[Node, ...]], mapping statement AST nodes to their predecessor CFG nodes stmt_next: Dict[ast.Node, FrozenSet[Node, ...]], mapping statement AST nodes to their successor CFG nodes c"|jSr)as_dotrs r r!zGraph.__repr__|s ;;=rc d}|jjD]}|dt|d|dz }|jjD]1}|jD] }|dt|dt|dz }"3|dz }|S)zPrint CFG in DOT format.zdigraph CFG { z z [label="z"]; z -> z; })indexvaluesidr)r resultnoder s r r*z Graph.as_dots F !!#: "T(D99f: !!#:99:%RXr%y99:: cMF MrN)r"r#r$r%r!r*r&rr r(r(Zs< rr(entryexiterrorr- stmt_prev stmt_nextceZdZdZdZy) _WalkModeN)r"r#r$FORWARDREVERSEr&rr r9r9s ' 'rr9c@eZdZdZdZdZdZdZdZdZ dZ d Z y ) GraphVisitora9Base class for a CFG visitors. This implementation is not thread safe. The visitor has some facilities to simplify dataflow analyses. In particular, it allows revisiting the nodes at the decision of the subclass. This can be used to visit the graph until the state reaches a fixed point. For more details on dataflow analysis, see https://www.seas.harvard.edu/courses/cs252/2011sp/slides/Lec02-Dataflow.pdf Note: the literature generally suggests visiting successor nodes only when the state of the current node changed, regardless of whether that successor has ever been visited. This implementation visits every successor at least once. Attributes: graph: Graph in_: Dict[Node, Any], stores node-keyed state during a visit out: Dict[Node, Any], stores node-keyed state during a visit c2||_|jyr)graphreset)r rAs r rzGraphVisitor.__init__sDJJJLrctd)zState initialization function. Optional to overload. An in/out state slot will be created for each node in the graph. Subclasses must overload this to control what that is initialized to. Args: node: Node Subclasses must implement this.NotImplementedErrorr r1s r init_statezGraphVisitor.init_states ? @@rctd)zVisitor function. Args: node: Node Returns: bool, whether the node should be revisited; subclasses can visit every reachable node exactly once by always returning False rDrErGs r visit_nodezGraphVisitor.visit_nodes ? @@rc4|jjjDcic]}||j|c}|_|jjjDcic]}||j|c}|_ycc}wcc}wr)rAr-r.rHin_outrGs r rBzGraphVisitor.resets~04 0@0@0G0G0I(,dood##DH15 0@0@0G0G0I(,dood##DHs B.Bc|j}tj|tjjryt |t jt jt jt jfS)zFReturns True if the node can safely be assumed not to touch variables.T) r rhasannoBasicSKIP_PROCESSINGrrBreakContinueRaisePass)r r1r s r can_ignorezGraphVisitor.can_ignoresS}}H ||Hdjj889  hzz4==$**diiH JJrcH|tjtjfvsJ|tjk(r|jjg}n2|tjk(rt |jj }t}r|jd}|j||j|}|tjk(r |j}n|tjk(r |j}D]}|s||vs |j||ryy)zVisits the CFG, breadth-first.rN)r9r<r=rAr3listr4setpopaddrJrr append)r modeopen_closedr1should_revisitchildrenr s r _visit_internalzGraphVisitor._visit_internals I%%y'8'89 99 9 y   zz e "" "4::??#e UF  YYq\d jjt,n "" "99 9$$ $99% U&0 ,,u  rcB|jtjyr)rbr9r<rs r visit_forwardzGraphVisitor.visit_forward**+rcB|jtjyr)rbr9r=rs r visit_reversezGraphVisitor.visit_reversererN) r"r#r$r%rrHrJrBrVrbrdrgr&rr r?r?s2* A AJ0,,rr?ceZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z d Zd ZdZdZdZdZdZdZdZdZdZdZdZy) GraphBuilderaBuilder that constructs a CFG from a given AST. This GraphBuilder facilitates constructing the DAG that forms the CFG when nodes are supplied in lexical order (i.e., top-down, depth first). Under these conditions, it supports building patterns found in typical structured programs. This builder ignores the flow generated by exceptions, which are assumed to always be catastrophic and present purely for diagnostic purposes (e.g. to print debug information). Statements like raise and try/catch sections are allowed and will generate control flow edges, but ordinary statements are assumed not to raise exceptions. Finally sections are also correctly interleaved between break/continue/return nodes and their subsequent statements. Important concepts: * nodes - nodes refer to CFG nodes; AST nodes are qualified explicitly * leaf set - since the graph is constructed gradually, a leaf set maintains the CFG nodes that will precede the node that the builder expects to receive next; when an ordinary node is added, it is connected to the existing leaves and it in turn becomes the new leaf * jump nodes - nodes that should generate edges other than what ordinary nodes would; these correspond to break, continue and return statements * sections - logical delimiters for subgraphs that require special edges; there are various types of nodes, each admitting various types of jump nodes; sections are identified by their corresponding AST node c2|j||_yr)rBparent)r parent_ast_nodes r rzGraphBuilder.__init__sJJL!DKrcBd|_t|_i|_t|_t|_i|_t|_i|_i|_ i|_ t|_ i|_ i|_ i|_i|_i|_i|_y)z!Resets the state of this factory.N)headrYerrors node_indexleaves active_stmtsowners forward_edgesfinally_sectionsfinally_section_subgraphsfinally_section_has_direct_flowpending_finally_sectionsexits section_entry continuesraises cond_entry cond_leavesrs r rBzGraphBuilder.resetsDI%DKDO%DKDDKDD&D",.D($'ED!DJDDNDKDODrct|trT|jj||jj||j j||fy|D]}|j ||y)zConnects nodes to signify that control flows from first to second. Args: first: Union[Set[Node, ...], Node] second: Node N)rrrr[r rt_connect_nodes)r firstsecondr1s r rzGraphBuilder._connect_nodesFse% jjnnV kkooe eV_-*$ D&)*rc||jvrtd|zttt j |}||j|<t |j|j|<|j||_ |jD]}|j|||jD]}||j|d<t|_ |S)zBGrows the graph by adding a CFG node following the current leaves.z%s added twice)r r r r)rp ValueErrorrrYrrrrrrsrnrqrrxrv)r r r1leaf section_ids r _add_new_nodezGraphBuilder._add_new_nodeUs4??" '(2 33 ce'//"3h GD $DOOH!$"3"34DKK yydi & $%&33; 6:d$$Z03;$'ED! Krc:|jj|y)zMarks the beginning of a statement. Args: stmt: Hashable, a key by which the statement can be identified in the CFG's stmt_prev and stmt_next attributes N)rrr[r stmts r begin_statementzGraphBuilder.begin_statementms $rc:|jj|y)zMarks the end of a statement. Args: stmt: Hashable, a key by which the statement can be identified in the CFG's stmt_prev and stmt_next attributes; must match a key previously passed to begin_statement. N)rrremovers r end_statementzGraphBuilder.end_statementvs T"rcJ|j|}t|f|_|S)zGrows the graph by adding an ordinary CFG node. Ordinary nodes are followed by the next node, in lexical order, that is, they become the new leaf set. Args: ast_node: ast.AST Returns: Node )rrYrq)r r r1s r add_ordinary_nodezGraphBuilder.add_ordinary_nodes&   h 'Dtg,DK Krcd|j|}t|_||j|<|S)aGrows the graph by adding a jump node. Jump nodes are added to the current leaf set, and the leaf set becomes empty. If the jump node is the last in a cond section, then it may be added back to the leaf set by a separate mechanism. Args: ast_node: ast.AST guards: Tuple[ast.AST, ...], the finally sections active for this node Returns: Node )rrYrqru)r r guardsr1s r _add_jump_nodezGraphBuilder._add_jump_nodes2   h 'D%DK"(D$ Krct|f}||jvr|S|j|D](}|j|\}}|j|||}*|j|=|S)z;Connects a jump node to the finally sections protecting it.)rYrurvr)r r1cursorguard_section_id guard_begin guard_endss r !_connect_jump_to_finally_sectionsz.GraphBuilder._connect_jump_to_finally_sectionss{ $\F 4((( m 11$7 $ > >?O Pk: &+.f d# Mrcf|j||}|j|j||S)aTGrows the graph by adding an exit node. This node becomes an exit for the current section. Args: ast_node: ast.AST section_id: Hashable, the node for which ast_node should be considered to be an exit node guards: Tuple[ast.AST, ...], the finally sections that guard ast_node Returns: Node )rryr[r r rrr1s r add_exit_nodezGraphBuilder.add_exit_nodes2   x 0DJJzt$ Krcd|j||}|j|j|y)aPGrows the graph by adding a reentry node. This node causes control flow to go back to the loop section's entry. Args: ast_node: ast.AST section_id: Hashable, the node for which ast_node should be considered to be an exit node guards: Tuple[ast.AST, ...], the finally sections that guard ast_node N)rr{r[rs r add_continue_nodezGraphBuilder.add_continue_nodes-   x 0DNN:""4(rc|D]?}||jvr|j|j|0|g|j|<Ay)zAdds extra connection between a raise node and containing except guards. The node is a graph node, not an ast node. Args: node: Node except_guards: Tuple[ast.AST, ...], the except sections that guard node N)r|r\)r r1 except_guardsguards r connect_raise_nodezGraphBuilder.connect_raise_nodesF$ $++  E!!$'"V E $rcR||jvsJt|j|<y)zEnters a regular section. Regular sections admit exit jumps, which end the section. Args: section_id: Hashable, the same node that will be used in calls to the ast_node arg passed to add_exit_node N)ryrYr rs r enter_sectionzGraphBuilder.enter_sections& TZZ '' ' UDJJzrc|j|D]&}|xj|j|zc_(|j|=y)zExits a regular section.N)ryrqr)r rexit_s r exit_sectionzGraphBuilder.exit_sectionsEJ'C kkT;;EBBkC :rc||jvsJ||jvsJt|j|<|j|}||j|<y)aEnters a loop section. Loop sections define an entry node. The end of the section always flows back to the entry node. These admit continue jump nodes which also flow to the entry node. Args: section_id: Hashable, the same node that will be used in calls to the ast_node arg passed to add_continue_node entry_node: ast.AST, the entry node into the loop (e.g. the test node for while loops) N)rzr{rYr)r r entry_noder1s r enter_loop_sectionzGraphBuilder.enter_loop_sectionsZ T// // / T^^ ++ +!$DNN:  ! !* -D%)Dz"rcN|j|j|j||j|D]2}|j |}|j||j|4t |j|f|_|j|=|j|=y)zExits a loop section.N)rrqrzr{rrY)r rreentryrs r exit_loop_sectionzGraphBuilder.exit_loop_sections T%7%7 %CD>>*-F99'Bj *d&8&8&DEF t))*578DK z" :&rcb||jvsJ||jvsJg|j|<y)zEnters a conditional section. Conditional sections define an entry node, and one or more branches. Args: section_id: Hashable, the same node that will be used in calls to the section_id arg passed to new_cond_branch N)r}r~rs r enter_cond_sectionzGraphBuilder.enter_cond_sections: T__ ,, , T-- -- -#%DZ rc||jvsJ||jvr=|j|j|j|j||_y|j|j|<y)z&Begins a new branch in a cond section.N)r~r}r\rqrs r new_cond_branchzGraphBuilder.new_cond_branch%se )) )) )T__$ z"))$++6OOJ/dk%)KKdooj!rc|j|D]}|xj|zc_|j|=|j|=y)zExits a conditional section.N)r~rqr})r rsplits r exit_cond_sectionzGraphBuilder.exit_cond_section3sD!!*- kkUk  # $rcr||jvr)|jj|j|yy)zEnters an except section.N)r|rqupdaters r enter_except_sectionz!GraphBuilder.enter_except_section:s.T[[  kkZ01!rcddg|j|<|jrd|j|<nd|j|<|jj |y)zEnters a finally section.NTF)rvrqrwrxr[rs r enter_finally_sectionz"GraphBuilder.enter_finally_section?sS37D"":. {{9=d**:69>d**:6!!%%j1rc||jvsJd|j|j|d<|j|st |_|j|=y)zExits a finally section.zEmpty finally?r:N)rxrqrvrwrYrs r exit_finally_sectionz!GraphBuilder.exit_finally_sectionIs\ T:: :L>m#$$E"# #3!)D/2io33!)D/2io3ii [[kkoo F JJL MrN)r"r#r$r%rrBrrrrrrrrrrrrrrrrrrrrrr&rr riris|F"'R *0 # ( $ ) $ #*& ' & 0%2 29-rriceZdZdZfdZdZdZdZdZdZ d%dZ d Z d Z d Z d Zd ZdZdZdZdZdZdZdZdZdZdZdZdZdZdZdZdZdZ dZ!d Z"d!Z#d"Z$d#Z%d$Z&xZ'S)&AstToCfgzTConverts an AST to CFGs. A separate CFG will be constructed for each function. cbtt| g|_d|_i|_g|_yr)superrr builder_stackbuildercfgslexical_scopes)r __class__s r rzAstToCfg.__init__s. (D"$DDLDIDrc:|jj|yr)rr\rGs r _enter_lexical_scopezAstToCfg._enter_lexical_scopest$rcF|jj}||k(sJyr)rrZ)r r1 leaving_nodes r _exit_lexical_scopezAstToCfg._exit_lexical_scopes$&&**,L <  rcg}t|jD]K}t|tjr|j r|j |t||sG||fcSd|fSr)reversedrrrTry finalbodyr\r stop_atincludedr1s r _get_enclosing_finally_scopesz&AstToCfg._get_enclosing_finally_scopessaH,,- D$(( # D' "X~  >rcg}t|jD]R}t|tjr'|j r|j |j t||sQ|S|Sr)rrrrrhandlersextendrs r _get_enclosing_except_scopesz%AstToCfg._get_enclosing_except_scopess^H,,- D$(( #  & D' " O  Orc\|j||jj|yr) generic_visitrrrGs r _process_basic_statementz!AstToCfg._process_basic_statements"tLL""4(rc|j||j|\}}|Jdj|||jj |||}|r.|j |}|jj ||yy)Nz${} that is not enclosed by any of {})rrformatrrrr)r r1exits_nodes_of_typemay_exit_via_excepttry_noderrs r _process_exit_statementz AstToCfg._process_exit_statements t99:MNHf  #!G!N!N !"##  << % %dHf =D778KLm ll%%dM:rc|jt|\}}|t|d||jj |||y)Nz that is not enclosed by any of )rtuplerrr)r r1loops_to_nodes_of_typerrs r _process_continue_statementz$AstToCfg._process_continue_statementsT99 $%'Hf 46 77LL""46:rc|j|j|y|jj||jj |jt ||_|j ||j||j||jj|_yr) rrrrr\rirrrrZrGs r visit_ClassDefzAstToCfg.visit_ClassDefs ||  LL""4(dll+%DLd#!!$'T"%%))+DLrc|j|jj||jj|jt ||_|j ||jj ||j|j|r,|j|jtjfn"|jD]}|j||jj||j||jj!|j"|<|jj%|_yr)rrrr\rirrrargsrbodyrLambdavisitrrrrrZ)r r1 is_lambdars r _process_function_defzAstToCfg._process_function_defs || ll$$T*dll+%DLd#LLt$!!$)), ""499t{{n=))$ 4 LLd#T"ll((*DIIdO%%))+DLrc*|j|dy)NFrrrGs r visit_FunctionDefzAstToCfg.visit_FunctionDefstu5rc*|j|dy)NTrrrGs r visit_LambdazAstToCfg.visit_Lambdastt4rcF|j|tjfyr)rrrrGs r visit_ReturnzAstToCfg.visit_Returns  (8(8':;rc&|j|yrrrGs r visit_ImportzAstToCfg.visit_Import!!$'rc&|j|yrrrGs r visit_ImportFromzAstToCfg.visit_ImportFrom rrc&|j|yrrrGs r visit_ExprzAstToCfg.visit_Expr rrc&|j|yrrrGs r visit_NamedExprzAstToCfg.visit_NamedExprs !!$'rc&|j|yrrrGs r visit_AssignzAstToCfg.visit_Assignrrc&|j|yrrrGs r visit_AnnAssignzAstToCfg.visit_AnnAssignrrc&|j|yrrrGs r visit_AugAssignzAstToCfg.visit_AugAssignrrc&|j|yrrrGs r visit_PasszAstToCfg.visit_Passrrc&|j|yrrrGs r visit_GlobalzAstToCfg.visit_Global rrc&|j|yrrrGs r visit_NonlocalzAstToCfg.visit_Nonlocal#rrc&|j|yrrrGs r visit_PrintzAstToCfg.visit_Print&rrc|j|tjfd|jjj |y)NT)r)rrrrror[rGs r visit_RaisezAstToCfg.visit_Raise)s=   t!t!=LLD!rc&|j|yrrrGs r visit_AssertzAstToCfg.visit_Assert.s!!$'rc&|j|yrrrGs r visit_DeletezAstToCfg.visit_Delete2rrc|jj||jj||j|j|jj ||j D]}|j||jj ||jD]}|j||jj||jj|yr) rrrrtestrrrorelserrr r1rs r visit_IfzAstToCfg.visit_If5s  LL  &LL##D)!!$)),LL  &  jj LL  &  jj LL""4(LLt$rc`|jj||j||jj||j |j |jj ||j |jD]}|j||jj||j||jD]}|j||jj||jj|yr)rrrrrrrrrrrrrrrs r visit_WhilezAstToCfg.visit_WhileKsLL  &d#LLt$tyy!LL##D$))4  jjLL""4(  T"  jj LLd#LLt$rc6|jj||j||jj||j |j |jj ||j tj|tjjr=|jtj|tjj|jD]}|j||jj||j!||j"D]}|j||jj%||jj'|yr)rrrrriterrrrOrPEXTRA_LOOP_TESTrgetannorrrrrrrrs r visit_ForzAstToCfg.visit_Forbs&LL  &d#LLt$  tyy!LL##D$))4 ||D$**445 ## ,,tTZZ77 8:  jjLL""4(  T"  jj LLd#LLt$rcd|j|tjtjfyr)rrWhileForrGs r visit_BreakzAstToCfg.visit_Breaks%    (rcd|j|tjtjfyr)rrr$r%rGs r visit_ContinuezAstToCfg.visit_Continues%$$T  ,rc|jj||jj||j|j |j|j |j |j |j D]}|j ||jj|yr)rrrtyperrrrrs r visit_ExceptHandlerzAstToCfg.visit_ExceptHandlersLL  &LL%%d+ yy jj yy jj  jj LLt$rcd|jj||j||jD]}|j ||j r|j d}|jj ||jj||j D]}|j ||jj||jj||j||jr|jd}|jj ||jD].}|jj||j |0|jj||jj||jrX|jj||jD]}|j ||jj||jj|y)Nr)rrrrrrrrrrrrrrr)r r1rblock_representativeblocks r visit_TryzAstToCfg.visit_TrysLL  &d#  jj {{![[^ ll%%&:; ll""#78++$ 4 ll""#78 ll$$%9:T" }}"]]1- ll%%&:;==% $$%9: 5 ll""#78 ll$$%9: ~~ ll((...$ 4 ll''-LLt$rc|jD]}|j||jD]}|j|yr)itemsrrr)r r1itemrs r visit_WithzAstToCfg.visit_WithsB * ##D)*  jjr)F)(r"r#r$r%rrrrrrrrrrrrrrrrrrrrr r rrrrrrrr"r&r(r+r/r3 __classcell__)rs@r rrs % )38; ;,8,265<(((( (((((((" ((%,%.%>  %(%TrrcPt}|j||jSr)rrr)r1visitors r rrs J' -- r)r% collectionsenumrrr tensorflow.python.autograph.pyctrobjectr namedtupler(Enumr9r?ri NodeVisitorrrr&rr r>s0  1&56&5R.KEG.b `,6`,FK6K\ GtGT r