在Pandas中运行NLTK sentence_bleu

Question

我正在尝试将sentence_bleu应用于Pandas中的一个专栏，以评估一些机器翻译的质量。但是它输出的分数是不正确的。有人能看到我的错误吗？

import pandas as pd
from nltk.translate.bleu_score import sentence_bleu

translations = {
    'reference': [['this', 'is', 'a', 'test'],['this', 'is', 'a', 'test'],['this', 'is', 'a', 'test']],
    'candidate': [['this', 'is', 'a', 'test'],['this', 'is', 'not','a', 'quiz'],['I', 'like', 'kitties', '.']]
}
df = pd.DataFrame(translations)

df['BLEU'] = df.apply(lambda row: sentence_bleu(row['reference'],row['candidate']), axis=1)
df

它输出以下内容：

Index   reference   candidate   BLEU
0   [this, is, a, test] [this, is, a, test] 1.288230e-231
1   [this, is, a, test] [this, is, not, a, quiz]    1.218332e-231
2   [this, is, a, test] [I, like, kitties, .]   0.000000e+00

行0应等于1.0，行1应小于1.0。大概在0.9左右。我做错了什么？

Answer 1

您当前正在比较列表中的字符串。由于这些字符串只包含单个单词，因此得分将所有n>1的n-gram直接评分为0。

相反，您希望引用是['this is a test'] (基本事实引用的列表)，候选者是'this is a test' (单个候选者)。

from nltk.translate.bleu_score import sentence_bleu

translations = {
    'reference': [['this is a test'],['this is a test'],['this is a test']],
    'candidate': ['this is a test','this is not a test','I like kitties']
}
df = pd.DataFrame(translations)

df['BLEU'] = df.apply(lambda row: sentence_bleu(row['reference'],row['candidate']), axis=1)
df

其结果是：

          reference           candidate           BLEU
0  [this is a test]      this is a test   1.000000e+00
1  [this is a test]  this is not a test   7.037906e-01
2  [this is a test]      I like kitties  6.830097e-155