Vh,|dZdZddlddlmZmZddlmZGddeZdd Z Gd d e Z Gd d eZ y)z& Corpus reader for the SemCor Corpus. z epytext en)*)XMLCorpusReader XMLCorpusView)TreecleZdZdZddZddZddZddfdZddZdd Z ddfd Z d Z d Z e d Zy)SemcorCorpusReadera Corpus reader for the SemCor Corpus. For access to the complete XML data structure, use the ``xml()`` method. For access to simple word lists and tagged word lists, use ``words()``, ``sents()``, ``tagged_words()``, and ``tagged_sents()``. cNtj|||||_||_yN)r__init___lazy_wordnet)selfrootfileidswordnetlazys I/home/dcms/DCMS/lib/python3.12/site-packages/nltk/corpus/reader/semcor.pyr zSemcorCorpusReader.__init__s"  tW5  Nc,|j|ddddS)zr :return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str) wordF_itemsrrs rwordszSemcorCorpusReader.words s {{7FE5%@@rc,|j|ddddS)z :return: the given file(s) as a list of chunks, each of which is a list of words and punctuation symbols that form a unit. :rtype: list(list(str)) chunkFrrs rchunkszSemcorCorpusReader.chunks's{{7GUE5AArposc8|j|dd|dk7|dk7S)ac :return: the given file(s) as a list of tagged chunks, represented in tree form. :rtype: list(Tree) :param tag: `'pos'` (part of speech), `'sem'` (semantic), or `'both'` to indicate the kind of tags to include. Semantic tags consist of WordNet lemma IDs, plus an `'NE'` node if the chunk is a named entity without a specific entry in WordNet. (Named entities of type 'other' have no lemma. Other chunks not in WordNet have no semantic tag. Punctuation tokens have `None` for their part of speech tag.) rFsemrrrrtags r tagged_chunksz SemcorCorpusReader.tagged_chunks0s#{{7GUC5L#,OOrc,|j|ddddS)z :return: the given file(s) as a list of sentences, each encoded as a list of word strings. :rtype: list(list(str)) rTFrrs rsentszSemcorCorpusReader.sents?s {{7FD%??rc,|j|ddddS)z :return: the given file(s) as a list of sentences, each encoded as a list of chunks. :rtype: list(list(list(str))) rTFrrs r chunk_sentszSemcorCorpusReader.chunk_sentsGs {{7GT5%@@rc8|j|dd|dk7|dk7S)a :return: the given file(s) as a list of sentences. Each sentence is represented as a list of tagged chunks (in tree form). :rtype: list(list(Tree)) :param tag: `'pos'` (part of speech), `'sem'` (semantic), or `'both'` to indicate the kind of tags to include. Semantic tags consist of WordNet lemma IDs, plus an `'NE'` node if the chunk is a named entity without a specific entry in WordNet. (Named entities of type 'other' have no lemma. Other chunks not in WordNet have no semantic tag. Punctuation tokens have `None` for their part of speech tag.) rTr rrr!s r tagged_sentszSemcorCorpusReader.tagged_sentsOs#{{7GT3%<NNrc |dk(r|sfd}njrtn j}tj |Dcgc]}||||||j c}Scc}w)Nrcdtjr t|Sj|Sr )LazyConcatenationr SemcorWordView_words)argsrs rz+SemcorCorpusReader._items..bs0/>4::F37;;Fr)r r-r.concatabspathsr )rrunit bracket_sentpos_tagsem_tag_fileids` rrzSemcorCorpusReader._items^sl 6>,A#'**$++A#mmG4 &$ gw N    sA*c |dvsJg}tj|j}|jdD]}g} t |D]M} t j | ||||j} |dk(r| j| =| j| O|r)|jt|jd| |j| d|vsJ|S)a] Helper used to implement the view methods -- returns a list of tokens, (segmented) words, chunks, or sentences. The tokens and chunks may optionally be tagged (with POS and sense information). :param fileid: The name of the underlying file. :param unit: One of `'token'`, `'word'`, or `'chunk'`. :param bracket_sent: If true, include sentence bracketing. :param pos_tag: Whether to include part-of-speech tags. :param sem_tag: Whether to include semantic tags, namely WordNet lemma and OOV named entity status. )tokenrrz.//srsnumN) ElementTreeparsegetrootfindall_all_xmlwords_inr_wordr extendappendSemcorSentenceattrib) rr8r3r4r5r6resultxmldocxmlsentsentxmlworditms rr.zSemcorCorpusReader._wordsns1111""6*224~~f- $GD+G4 %(..T7GT]]6>KK$KK$ % nW^^F-CTJK d# $ 6!!! rc|j}|sd}|jd|}|jd}|,|dz|z}dt|jdddz } ndx}} |jd |} |jd } d |j v} |jd } |d k(r |s|s|}|S|f|r| fndz|r|| | | fndz}|S|jd}|dk(r|S|  |j |}|r t| |gn|}|r'| r%| ttd|gStd|S|r| t|S|r|dS|S#t $r3 d|| t| fz}n#t$r|dz| zdz| z}YnwxYwYwxYw)Nlemmalexsn%)nvars:rrdfwnsnpnrr:r7rz %s.%s.%02d.NE) textgetintsplitkeyslemma_from_key Exception ValueErrorr)rJr3r5r6rtknrNrO sense_keywnposredefsensenum isOOVEntityrrKwwsensebottoms rrAzSemcorCorpusReader._wordsllC GS) G$   e+I-EKK$Q'(1,E!% $I 3 ;;v&glln, kk   7?7J F!(vb2@Guh +?@@#D&11!5v..!!9$!M;% $0 % % #H 4%E * % e 3c 9H D"s6,E F E,+F ,FF FF  F )Tr )__name__ __module__ __qualname____doc__r rrr#r%r'r)rr. staticmethodrAr[rrrrs_ AB%)u P@A$(e O #JJ"J"rrNct|g}|D].}|jdvr|j|#t||0|S)Nwfpunc)r"rCr@)eltrFchilds rr@r@sD ~, 99 & MM% UF + , MrceZdZdZdZy)rDz A list of words, augmented by an attribute ``num`` used to record the sentence identifier (the ``n`` attribute from the XML). c>||_tj||yr )numlistr )rr|itemss rr zSemcorSentence.__init__s dE"rN)rorprqrrr r[rrrDrDs  #rrDc(eZdZdZdZdZdZdZy)r-zN A stream backed corpus view specialized for use with the BNC corpus. c|rd}nd}||_||_||_||_||_t j |||y)a{ :param fileid: The name of the underlying file. :param unit: One of `'token'`, `'word'`, or `'chunk'`. :param bracket_sent: If true, include sentence bracketing. :param pos_tag: Whether to include part-of-speech tags. :param sem_tag: Whether to include semantic tags, namely WordNet lemma and OOV named entity status. z.*/sz.*/s/(punc|wf)N)_unit_sent_pos_tag_sem_tagr rr )rr8r3r4r5r6rtagspecs rr zSemcorWordView.__init__sF G&G !    tVW5rc^|jr|j|S|j|Sr )r handle_sent handle_word)rrxcontexts r handle_eltzSemcorWordView.handle_elts+ ::##C( (##C( (rctj||j|j|j|j Sr )rrArrrr )rrxs rrzSemcorWordView.handle_words0!'' T]]DMM4==  rcg}|D]k}|jdvrD|j|}|jdk(r|j|C|j |Ut d|jzt |jd|S)NrurzUnexpected element %sr;)r"rrrBrCrerDrE)rrxrIryrKs rrzSemcorWordView.handle_sents FEyyN*&&u-::'KK$KK$ !8599!DEE Fcjj0$77rN)rorprqrrr rrrr[rrr-r-s6,)  8rr-r ) rr __docformat__nltk.corpus.reader.apinltk.corpus.reader.xmldocsrr nltk.treerrr@r}rDr-r[rrrsF $EK"K"\#T#18]18r