Vh$dZddlZddlddlddlmZddlmZejdZ ejdZ ejdZ ejd Z Gd d eZGd d eeZGddeZy)zO Corpus reader for corpora that consist of parenthesis-delineated parse trees. N)*)map_tag)Treez\((\d+) ([^\s()]+) ([^\s()]+)\)z\(([^\s()]+) ([^\s()]+)\)z\([^\s()]+ ([^\s()]+)\)z \s*\(\s*\(c@eZdZdZ d dZdZdZdZd dZdZ y) BracketParseCorpusReaderz Reader for corpora that consist of parenthesis-delineated parse trees, like those found in the "combined" section of the Penn Treebank, e.g. "(S (NP (DT the) (JJ little) (NN dog)) (VP (VBD barked)))". Nc^tj||||||_||_||_y)a :param root: The root directory for this corpus. :param fileids: A list or regexp specifying the fileids in this corpus. :param comment_char: The character which can appear at the start of a line to indicate that the rest of the line is a comment. :param detect_blocks: The method that is used to find blocks in the corpus; can be 'unindented_paren' (every unindented parenthesis starts a new parse) or 'sexpr' (brackets are matched). :param tagset: The name of the tagset used by this corpus, to be used for normalizing or converting the POS tags returned by the ``tagged_...()`` methods. N)SyntaxCorpusReader__init__ _comment_char_detect_blocks_tagset)selfrootfileids comment_char detect_blocksencodingtagsets P/home/dcms/DCMS/lib/python3.12/site-packages/nltk/corpus/reader/bracket_parse.pyr z!BracketParseCorpusReader.__init__"s/, ##D$B)+ c v|jdk(rt||jS|jdk(r t|S|jdk(r_t |d}|jrD|Dcgc]9}t j dt j|jzd|;}}|SJd cc}w) Nsexpr)r blanklineunindented_parenz^\()start_rez (?m)^%s.*zbad block type)r read_sexpr_blockr read_blankline_blockread_regexp_blockresubescape)rstreamtokstoks r _read_blockz$BracketParseCorpusReader._read_block=s   ' )#F9K9KL L  K /'/ /  $6 6$Vf=D!! $FF;43E3E)FFCPK && &1 s->B6cbtjdd|}tjdd|}|S)Nz\((.)\)z(\1 \1)z"\(([^\s()]+) ([^\s()]+) [^\s()]+\)(\1 \2)r r!rts r _normalizez#BracketParseCorpusReader._normalizeOs, FF:z1 - FF8*a Hrc n tj|j|}|jdk(rt |dk(r|dS|S#t $r}t jjd|jdk(rktddD]\} t|j|d|zz}t jjd|z|ccYd}~S#t $rYZwxYwt jjd td |j|cYd}~Sd}~wwxYw) Nrrz(Bad tree detected; trying to recover... )zmismatched parens)z( Recovered by adding %d close paren(s) z' Recovered by returning a flat parse. S) r fromstringr,labellen ValueErrorsysstderrwriteargsrange_tag)rr+treeenvs r_parsezBracketParseCorpusReader._parseVs +??4??1#56Dzz|r!c$i1nAw  + JJ  H Ivv//q!A S1W!=> ((JQN !% JJ  G HTYYq\* *! +sOA A A D4=D/AC D/D4 C,)D/+C,,=D/)D4/D4c tj|j|Dcgc] \}}||f }}}|r:||jk7r+|Dcgc]\}}|t |j||f }}}|Scc}}wcc}}wN)TAGWORDfindallr,r r)rr+rpw tagged_sents rr;zBracketParseCorpusReader._tagqs,3OODOOA cbtj||tj|g|i|y)at Initialize the corpus reader. Categorization arguments (C{cat_pattern}, C{cat_map}, and C{cat_file}) are passed to the L{CategorizedCorpusReader constructor }. The remaining arguments are passed to the L{BracketParseCorpusReader constructor }. N)CategorizedCorpusReaderr r)rr9kwargss rr z,CategorizedBracketParseCorpusReader.__init__s, ((v6 ))$@@@rcDt||j|||SrB)super tagged_words_resolverr categoriesr __class__s rrXz0CategorizedBracketParseCorpusReader.tagged_words w#DMM':$FOOrcDt||j|||SrB)rW tagged_sentsrYrZs rr_z0CategorizedBracketParseCorpusReader.tagged_sentsr]rcDt||j|||SrB)rW tagged_parasrYrZs rraz0CategorizedBracketParseCorpusReader.tagged_parasr]rcBt||j||SrB)rW parsed_wordsrYrrr[r\s rrcz0CategorizedBracketParseCorpusReader.parsed_wordsw#DMM':$FGGrcBt||j||SrB)rW parsed_sentsrYrds rrgz0CategorizedBracketParseCorpusReader.parsed_sentsrercBt||j||SrB)rW parsed_parasrYrds rriz0CategorizedBracketParseCorpusReader.parsed_parasrer)NNN)NN) rLrMrNrOr rXr_rarcrgri __classcell__)r\s@rrRrR}s8 APPPHHHHrrRc.eZdZdZddZddZd dZdZy) AlpinoCorpusReadera Reader for the Alpino Dutch Treebank. This corpus has a lexical breakdown structure embedded, as read by `_parse` Unfortunately this puts punctuation and some other words out of the sentence order in the xml element tree. This is no good for `tag_` and `word_` `_tag` and `_word` will be overridden to use a non-default new parameter 'ordered' to the overridden _normalize function. The _parse function can then remain untouched. Nc:tj||dd||y)Nz alpino\.xmlr)rrr)rr )rrrrs rr zAlpinoCorpusReader.__init__s' ))   % * rc2|dddk7rytjdd|}|rtjdd|}ntjd d |}tjd d |}tjd d|}tjdd|}|S)aNormalize the xml sentence element in t. The sentence elements , although embedded in a few overall xml elements, are separated by blank lines. That's how the reader can deliver them one at a time. Each sentence has a few category subnodes that are of no use to us. The remaining word nodes may or may not appear in the proper order. Each word node has attributes, among which: - begin : the position of the word in the sentence - pos : Part of Speech: the Tag - word : the actual word The return value is a string with all xml elementes replaced by clauses: either a cat clause with nested clauses, or a word clause. The order of the bracket clauses closely follows the xml. If ordered == True, the word clauses include an order sequence number. If ordered == False, the word clauses only have pos and word parts. N z z(\1z> z (\1 \2 \3)z- r(z r0z.*zr))rr+ordereds rr,zAlpinoCorpusReader._normalizes" Sb6\ ! FF0&! < QA GUVWA FF<q ) FF-sA 6 FF%sA .rc tj|j|dDcgc]\}}}t|||f}}}}|j |r?||j k7r0|Dcgc]\}}}|t |j ||f!}}}}|S|Dcgc] \}}}||f }}}}|Scc}}}wcc}}}wcc}}}w)NT)rp) SORTTAGWRDrDr,intsortr r)rr+rorErFrGs rr;zAlpinoCorpusReader._tags(//40PQ  AqVQN    f ,GR:C1aGDLL&!45K 4???iq!QAq6?K? @sB4/$B;CcZ|j|}|Dcgc]\}}| c}}Scc}}w)z(Return a correctly ordered list if words)r;)rr+rGrFrEs rrJzAlpinoCorpusReader._words'iil +,fq!,,,s ')z ISO-8859-1N)FrB)rLrMrNrOr r,r;rJrPrrrlrls  D -rrl)rOr6nltk.corpus.reader.apinltk.corpus.reader.utilnltk.tagr nltk.treerr compilerrrCrIEMPTY_BRACKETSr rrTrRrlrPrrr}s $%ZZ: ; **1 2 zz,-M*`01`0F%H5%HPH-1H-r