Vh- J d Z ddlZddl ddl ej d Z ej d Z ej d Z ej d Z ej d Z ej d Z
ej d
Z G d d Z G d
de
Zy)a
CorpusReader for the Comparative Sentence Dataset.
- Comparative Sentence Dataset information -
Annotated by: Nitin Jindal and Bing Liu, 2006.
Department of Computer Sicence
University of Illinois at Chicago
Contact: Nitin Jindal, njindal@cs.uic.edu
Bing Liu, liub@cs.uic.edu (https://www.cs.uic.edu/~liub)
Distributed with permission.
Related papers:
- Nitin Jindal and Bing Liu. "Identifying Comparative Sentences in Text Documents".
Proceedings of the ACM SIGIR International Conference on Information Retrieval
(SIGIR-06), 2006.
- Nitin Jindal and Bing Liu. "Mining Comprative Sentences and Relations".
Proceedings of Twenty First National Conference on Artificial Intelligence
(AAAI-2006), 2006.
- Murthy Ganapathibhotla and Bing Liu. "Mining Opinions in Comparative Sentences".
Proceedings of the 22nd International Conference on Computational Linguistics
(Coling-2008), Manchester, 18-22 August, 2008.
N)*z^\*+$z z z
zz (\d)_((?:[\.\w\s/-](?!\d_))+)z
\(([^\(]*)\)$c * e Zd ZdZ ddZd Zy)
ComparisonzN
A Comparison represents a comparative sentence and its constituents.
Nc X || _ || _ || _ || _ || _ || _ y)a]
:param text: a string (optionally tokenized) containing a comparison.
:param comp_type: an integer defining the type of comparison expressed.
Values can be: 1 (Non-equal gradable), 2 (Equative), 3 (Superlative),
4 (Non-gradable).
:param entity_1: the first entity considered in the comparison relation.
:param entity_2: the second entity considered in the comparison relation.
:param feature: the feature considered in the comparison relation.
:param keyword: the word or phrase which is used for that comparative relation.
N)text comp_typeentity_1entity_2featurekeyword)selfr r r r
r r s T/home/dcms/DCMS/lib/python3.12/site-packages/nltk/corpus/reader/comparative_sents.py__init__zComparison.__init__8 s. & "
c dj | j | j | j | j | j
| j S )Nz]Comparison(text="{}", comp_type={}, entity_1="{}", entity_2="{}", feature="{}", keyword="{}"))formatr r r r
r r )r
s r __repr__zComparison.__repr__R s@
*
& II NN MM MM LL LL
r )NNNNNN)__name__
__module__ __qualname____doc__r r r r r r 3 s% 4
r r c j e Zd ZdZeZ e ddfdZddZddZ d Z
ddZ dd Z d
Z
d Zd Zd
Zy) ComparativeSentencesCorpusReadera
Reader for the Comparative Sentence Dataset by Jindal and Liu (2006).
>>> from nltk.corpus import comparative_sentences
>>> comparison = comparative_sentences.comparisons()[0]
>>> comparison.text # doctest: +NORMALIZE_WHITESPACE
['its', 'fast-forward', 'and', 'rewind', 'work', 'much', 'more', 'smoothly',
'and', 'consistently', 'than', 'those', 'of', 'other', 'models', 'i', "'ve",
'had', '.']
>>> comparison.entity_2
'models'
>>> (comparison.feature, comparison.keyword)
('rewind', 'more')
>>> len(comparative_sentences.comparisons())
853
Nutf8c ^ t j | ||| || _ || _ d| _ y)a
:param root: The root directory for this corpus.
:param fileids: a list or regexp specifying the fileids in this corpus.
:param word_tokenizer: tokenizer for breaking sentences or paragraphs
into words. Default: `WhitespaceTokenizer`
:param sent_tokenizer: tokenizer for breaking paragraphs into sentences.
:param encoding: the encoding that should be used to read the corpus.
z
README.txtN) CorpusReaderr _word_tokenizer_sent_tokenizer_readme)r
rootfileidsword_tokenizersent_tokenizerencodings r r z)ComparativeSentencesCorpusReader.__init__t s/ " dD'8< - - # r c |
| j }nt |t r|g}t | j |dd D cg c]$ \ }}}| j || j | & c}}} S c c}}}w )a
Return all comparisons in the corpus.
:param fileids: a list or regexp specifying the ids of the files whose
comparisons have to be returned.
:return: the given file(s) as a list of Comparison objects.
:rtype: list(Comparison)
Tr% )_fileids
isinstancestrconcatabspaths
CorpusView_read_comparison_blockr
r" pathencfileids r comparisonsz,ComparativeSentencesCorpusReader.comparisons sz ?mmG
% iG ,0==$+M
'T3 d&A&ACP
s )A1 c t | j |dd D cg c]$ \ }}}| j || j | & c}}} }|D ch c] }|s|j }}|S c c}}}w c c}w )a&
Return a set of all keywords used in the corpus.
:param fileids: a list or regexp specifying the ids of the files whose
keywords have to be returned.
:return: the set of keywords and comparative phrases used in the corpus.
:rtype: set(str)
Tr' )r+ r, r- _read_keyword_blocklower)r
r" r0 r1 r2 all_keywordsr keywords_sets r keywordsz)ComparativeSentencesCorpusReader.keywords s ,0==$+M
'T3 d&>&>M
8DOGw
O O
Ps )A1 A8A8c g }| j d 5 }|j }ddd j d D ]5 }|r|j d r|j |j 7 |S # 1 sw Y TxY w)z
Return the list of words and constituents considered as clues of a
comparison (from listOfkeywords.txt).
zlistOfkeywords.txtN
z//)openreadsplit
startswithappendstrip)r
r9 fpraw_textlines r keywords_readmez0ComparativeSentencesCorpusReader.keywords_readme sz
YY+
, !wwyH ! NN4( *D4??40 OODJJL ) *
! !s A88Bc t | j |dd D cg c]$ \ }}}| j || j | & c}}} S c c}}}w )ac
Return all sentences in the corpus.
:param fileids: a list or regexp specifying the ids of the files whose
sentences have to be returned.
:return: all sentences of the corpus as lists of tokens (or as plain
strings, if no word tokenizer is specified).
:rtype: list(list(str)) or list(str)
Tr' )r+ r, r- _read_sent_blockr/ s r sentsz&ComparativeSentencesCorpusReader.sents s[ ,0==$+M
'T3 d&;&;cJ
)A c t | j |dd D cg c]$ \ }}}| j || j | & c}}} S c c}}}w )a)
Return all words and punctuation symbols in the corpus.
:param fileids: a list or regexp specifying the ids of the files whose
words have to be returned.
:return: the given file(s) as a list of words and punctuation symbols.
:rtype: list(str)
Tr' )r+ r, r- _read_word_blockr/ s r wordsz&ComparativeSentencesCorpusReader.words s[ ,0==$+M
'T3 d&;&;cJ
rI c |j }|sg S t j t | }|rt j t | }t j t
| }|j j
}| j r| j j | }|j g }|r|D ] }t t j d| j d } t || }
|j }t j | } | r[| D ]V \ } }
| dk( r|
j
|
_ !| dk( r|
j
|
_ <| dk( sB|
j
|
_ X t" j | }|r
|d |
_ |j' |
|rS|D ]N }t t j d| j d } t || }
|j' |
P |S
)Nz )r r 123r )readlinerefindall
COMPARISONGRAD_COMPARISONNON_GRAD_COMPARISONrA r tokenizeintmatchgroupr ENTITIES_FEATSr r
r KEYWORDr r@ )r
streamrD comparison_tagsgrad_comparisonsnon_grad_comparisonscomparison_textcomparison_bundlecompr
comparisonentities_featscode entity_featr s r r. z7ComparativeSentencesCorpusReader._read_comparison_block s ??$D jjT :O #%::ot#D ')zz2Et'L$"(//"3"9"9";''&*&:&:&C&CO&TO! %'!# 0 =$' t(D(J(J1(M$N %/!0I&
& 0)7)?)?)E)5C M 1k#'3;:E:K:K:MJ$7%)S[:E:K:K:MJ$7%)S[9D9J9J9LJ$6
M #*//$"7"18J .)00<%=* ( 4 =$' t(D(J(J1(M$N %/!0I&
*00<
= )(e r c l g }| j | D ] }|j |j |S N)r. r@ r )r
r^ r9 re s r r5 z4ComparativeSentencesCorpusReader._read_keyword_block s8 55f= 0J OOJ . . / 0r c ( |j }t j t | r. |j }t j t | rn,Yt j t
| st j | st j t | sj| j rB| j j | D cg c] }| j j | c}S | j j | gS
c c}w rj ) rR rS rZ STARSrT rU r\ CLOSE_COMPARISONr rX r )r
r^ rD sents r rG z1ComparativeSentencesCorpusReader._read_sent_block s ??$Dxxt$!??,Dxxt, JJz40&..t4
#3T:'' %)$8$8$A$A$$G ,,55d;
! 0 0 9 9$ ?@@' s "Dc X g }| j | D ] }|j | |S rj )rG extend)r
r^ rL rn s r rK z1ComparativeSentencesCorpusReader._read_word_block1 s2 ))&1 D LL r rj )r r r r StreamBackedCorpusViewr- WhitespaceTokenizerr r3 r9 rE rH rL r. r5 rG rK r r r r r ` sP " (J +,
$,
( &
"
3)j A,r r )r rS nltk.corpus.reader.api
nltk.tokenizecompilerl rU rm rV rW r\ r] r r r r r r rv s 8
$
8
RZZ
'
2::o. "**]+ bjj+ <=
"**%
&*
*
ZU| Ur