NLTK Vader SentimentIntensityAnalyzer Bigram

Question

对于Python中的维德SentimentIntensityAnalyzer，有没有办法添加二元语法规则？我尝试使用两个单词的输入来更新词典，但它没有改变极性分数。提前感谢！

from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer

analyser = SentimentIntensityAnalyzer()

#returns a compound score of -0.296
print(analyser.polarity_scores('no issues'))

analyser.lexicon['no issues'] = 0.0
#still returns a compound score of -0.296
print(analyser.polarity_scores('no issues'))

Answer 1

没有直接的方法可以将二元语法添加到维德词典中。这是因为维德考虑了单个标记来进行情感分析。但是，您可以使用以下步骤来完成此操作：

1. 将二元语法创建为令牌。例如，您可以将二元语法(“无问题”)转换为新创建的令牌的极性字典中的令牌("noissues").
2. Maintain。{"noissues“：2}然后，
3. 在传递文本进行情感评分计算之前执行额外的文本处理。

下面的代码实现了上述功能：

allowed_bigrams = {'noissues' : 2} #add more as per your requirement
    
def process_text(text):
    tokens = text.lower().split() # list of tokens
    bigrams = list(nltk.bigrams(tokens)) # create bigrams as tuples of tokens
    bigrams = list(map(''.join, bigrams)) # join each word without space to create new bigram
    bigrams.append('...') # make length of tokens and bigrams list equal
     
    #begin recreating the text
    final = ''
    for i, token in enumerate(tokens):
        b = bigrams[i]
        
        if b in allowed_bigrams:
          join_word = b # replace the word in text by bigram
          tokens[i+1] = '' #skip the next word
        else:
            join_word = token
        final += join_word + ' '
    return final
text  = 'Hello, I have no issues with you'
print (text)
print (analyser.polarity_scores(text))
final = process_text(text)
print (final)
print(analyser.polarity_scores(final))

输出：

Hello, I have no issues with you
{'neg': 0.268, 'neu': 0.732, 'pos': 0.0, 'compound': -0.296}
hello, i have noissues  with you 
{'neg': 0.0, 'neu': 0.625, 'pos': 0.375, 'compound': 0.4588}

请注意，在输出中，"no“和"issues”这两个单词是如何添加在一起的，形成了二元语法"noissues“。