udpipe (keywords_rake)如何将关键字链接到从中提取关键字的文档

Question

我使用udpipe包中的函数keywords_rake (针对R)从一堆文档中提取关键字。

udmodel_en <- udpipe_load_model(file = dl$file_model)
x <- udpipe_annotate(udmodel_en, x = data$text)
x <- as.data.frame(x)

keywords <- keywords_rake(x = x, term = "lemma", group = "doc_id", 
                          relevant = x$xpos %in% c("NN", "JJ"), ngram_max = 2)

其中数据如下所示

  Text
  "cats are nice but dogs are better..."
  "I really like dogs..."
  "red flowers are pretty, especially roses..."
  "once I saw a blue whale ..."
  ....

(每行是一个单独的文档)

但是，输出不包括关键字的来源，而是提供所有文档的关键字列表

如何将这些关键字链接到它们所取自的相应文档？(即每个文档都有一个关键字列表)

如下所示：

      keywords
doc1   dog, cat, blue whale
doc2   dog 
doc3   red flower, tower, Donald Trump 

Answer 1

您可以结合使用txt_recode_ngram和keywords_rake的结果来实现这一点。优点是一切都回到了原始的data.frame，然后你可以选择你需要的东西。请参见下面的示例，使用随udpipe提供的数据集。

免责声明:在udpipe的github页面上，代码从jwijffels在issue 41中的答案中复制。

data(brussels_reviews_anno)
x <- subset(brussels_reviews_anno, language == "nl")
keywords <- keywords_rake(x = x, term = "lemma", group = "doc_id", 
                          relevant = x$xpos %in% c("NN", "JJ"), sep = "-")
head(keywords)

            keyword ngram freq     rake
1  openbaar-vervoer     2   19 2.391304
2         heel-fijn     2    2 2.236190
3  heel-vriendelijk     2    3 2.131092
4 herhaling-vatbaar     2    6 2.000000
5  heel-appartement     2    2 1.935450
6 steenworp-afstand     2    4 1.888889

x$term <- txt_recode_ngram(x$lemma, compound = keywords$keyword, ngram = keywords$ngram, sep = "-")
x$term <- ifelse(!x$term %in% keywords$keyword, NA, x$term)
head(x[!is.na(x$term), ])

        doc_id language sentence_id token_id       token      lemma xpos                term
67039 19991431       nl        4379       11         erg        erg   JJ        erg-centraal
67048 19991431       nl        4379       20        leuk       leuk   JJ          leuk-adres
67070 21054450       nl        4380        6       goede       goed   JJ        goed-locatie
67077 21054450       nl        4380       13    Europese   europees   JJ       europees-wijk
67272 23542577       nl        4393       84 uitstekende uitstekend   JJ uitstekend-gastheer
67299 40676307       nl        4396       25   gezellige   gezellig   JJ      gezellig-buurt