使用stanford corenlp库标记单词和句子失败

Question

 //tagger
      MaxentTagger tagger = new MaxentTagger(args[0]);
      TokenizerFactory<CoreLabel> ptbTokenizerFactory = PTBTokenizer.factory(new CoreLabelTokenFactory(),
                                   "untokenizable=noneKeep");
      BufferedReader r = new BufferedReader(new InputStreamReader(new FileInputStream(args[1]), "utf-8"));
      PrintWriter pw = new PrintWriter(new OutputStreamWriter(System.out, "utf-8"));
      DocumentPreprocessor documentPreprocessor = new DocumentPreprocessor(r);
      documentPreprocessor.setTokenizerFactory(ptbTokenizerFactory);
      for (List<HasWord> sentence : documentPreprocessor) {
        List<TaggedWord> tSentence = tagger.tagSentence(sentence);
        pw.println(Sentence.listToString(tSentence, false));
      }

从C:\work\development\workspace\stanfordnlp\sample.txt读取POS机标记器模型时出现以下异常...

C:\work\development\workspace\stanfordnlp\sample.txtException in thread "main" edu.stanford.nlp.io.RuntimeIOException: Error while loading a tagger model (probably missing model file)
    at edu.stanford.nlp.tagger.maxent.MaxentTagger.readModelAndInit(MaxentTagger.java:869)
    at edu.stanford.nlp.tagger.maxent.MaxentTagger.readModelAndInit(MaxentTagger.java:767)
    at edu.stanford.nlp.tagger.maxent.MaxentTagger.<init>(MaxentTagger.java:298)
    at edu.stanford.nlp.tagger.maxent.MaxentTagger.<init>(MaxentTagger.java:263)
    at phoenix.TokenizerDemo.main(TokenizerDemo.java:42)
Caused by: java.io.StreamCorruptedException: invalid stream header: 416E6F74
    at java.io.ObjectInputStream.readStreamHeader(Unknown Source)
    at java.io.ObjectInputStream.<init>(Unknown Source)
    at edu.stanford.nlp.tagger.maxent.TaggerConfig.readConfig(TaggerConfig.java:748)
    at edu.stanford.nlp.tagger.maxent.MaxentTagger.readModelAndInit(MaxentTagger.java:804)
    ... 4 more

Answer 1

日志应该清楚地指出问题所在：

正在从C:\work\development\workspace\stanfordnlp\sample.txt读取POS标记器模型的

...

您错误地实例化了MaxentTagger实例。如果向构造函数提供单个字符串参数，则该字符串将提供标记器模型文件的路径。

有关详细信息，请参阅documentation for MaxentTagger。