如何修复NLTK分块错误？

Question

我正在尝试使用教程http://streamhacker.com/2008/12/29/how-to-train-a-nltk-chunker/训练我自己的NLTK分块程序

我把代码写成，

>>> import nltk
>>> import nltk.chunk
>>> def conll_tag_chunks(chunk_sents):
    tag_sents = [nltk.chunk.tree2conlltags(tree) for tree in chunk_sents]
    return [[(t, c) for (w, t, c) in chunk_tags] for chunk_tags in tag_sents]

>>> import nltk.corpus, nltk.tag
>>> from nltk.metrics import accuracy
>>> def ubt_conll_chunk_accuracy(train_sents, test_sents):
    train_chunks = conll_tag_chunks(train_sents)
        test_chunks = conll_tag_chunks(test_sents)

        u_chunker = nltk.tag.UnigramTagger(train_chunks)
        print 'u:', accuracy(u_chunker, test_chunks)

        ub_chunker = nltk.tag.BigramTagger(train_chunks, backoff=u_chunker)
        print 'ub:', accuracy(ub_chunker, test_chunks)

        ubt_chunker = nltk.tag.TrigramTagger(train_chunks, backoff=ub_chunker)
        print 'ubt:', accuracy(ubt_chunker, test_chunks)

        ut_chunker = nltk.tag.TrigramTagger(train_chunks, backoff=u_chunker)
        print 'ut:', accuracy(ut_chunker, test_chunks)

        utb_chunker = nltk.tag.BigramTagger(train_chunks, backoff=ut_chunker)
        print 'utb:', accuracy(utb_chunker, test_chunks)


>>> conll_train = nltk.corpus.conll2000.chunked_sents('train.txt')
>>> conll_test = nltk.corpus.conll2000.chunked_sents('test.txt')
>>> ubt_conll_chunk_accuracy(conll_train, conll_test)

但在这里，我得到的错误是，

>>> ubt_conll_chunk_accuracy(conll_train, conll_test)
u:

Traceback (most recent call last):
  File "<pyshell#10>", line 1, in <module>
    ubt_conll_chunk_accuracy(conll_train, conll_test)
  File "<pyshell#7>", line 6, in ubt_conll_chunk_accuracy
    print 'u:', accuracy(u_chunker, test_chunks)
  File "C:\Python27\lib\site-packages\nltk\metrics\scores.py", line 38, in accuracy
    if len(reference) != len(test):
TypeError: object of type 'UnigramTagger' has no len()
>>> treebank_sents = nltk.corpus.treebank_chunk.chunked_sents()
>>> ubt_conll_chunk_accuracy(treebank_sents[:2000], treebank_sents[2000:])
u:

Traceback (most recent call last):
  File "<pyshell#12>", line 1, in <module>
    ubt_conll_chunk_accuracy(treebank_sents[:2000], treebank_sents[2000:])
  File "<pyshell#7>", line 6, in ubt_conll_chunk_accuracy
    print 'u:', accuracy(u_chunker, test_chunks)
  File "C:\Python27\lib\site-packages\nltk\metrics\scores.py", line 38, in accuracy
    if len(reference) != len(test):
TypeError: object of type 'UnigramTagger' has no len()
>>> 

如果任何人可以建议，我可以如何修复这个错误？提前谢谢。我在MS-Windows10上使用NLTK 3.1，Python2.7.11。

Answer 1

查看有关nltk包的accuracy方法的文档

nltk.metrics.scores.accuracy(reference，测试)

参考值和相应的测试值列表，返回相等的对应值的分数。特别是，返回索引0的分数参数：

- reference ( list ) -参考值的有序列表。

-测试列表()-要与相应的参考值进行比较的值列表。