R中同义词的识别与分组

Question

我试图识别和聚合给定数据集的同义词。请参阅下面的样本数据。

library(tm)
library(SnowballC)

dataset <- c("dad glad accept large admit large accept dad big large big accept big accept dad dad Happy dad accept glad papa dad Happy dad glad dad dad papa admit Happy big accept accept big accept dad Happy admit Happy Happy glad Happy dad accept accept large daddy large accept large large large big daddy accept admit dad admit daddy dad admit dad admit Happy accept accept Happy daddy accept admit")

docs <- Corpus(VectorSource(dataset))
dtm <- TermDocumentMatrix(docs)
m <- as.matrix(dtm)
sort(rowSums(m),decreasing=TRUE)

结果：

accept    dad  happy  admit  large    big  daddy   glad   papa 
    15     14      9      8      8      6      4      4      2 

我想用我下载和安装的wordnet软件包找到上面每个单词的同义词。例如，要获得“接受”的同义词，我可以这样做：

library(wordnet)
setDict("C:/Program Files (x86)/WordNet/2.1/dict")

filter <- getTermFilter("ExactMatchFilter", "accept", TRUE)
terms <- getIndexTerms("VERB", 1, filter)
getSynonyms(terms[[1]])

结果：

 [1] "accept"    "admit"     "assume"    "bear"      "consent"   "go for"    "have"      "live with"
 [9] "swallow"   "take"      "take on"   "take over"

现在，我想将这两个结果集组合起来，以便按以下方式对同义词进行分组。对给定的组和组的最常用的单词(1级)后面用这些词标记，类似于这样：

id  word    word_count  syn_group   rank
1   accept  15          1           1
5   admit   8           1           2
2   dad     14          2           1
8   daddy   4           2           2
9   papa    2           2           3
3   happy   9           3           1
7   glad    4           3           2
4   large   8           4           1
6   big     6           4           2

这样就可以聚合成这样

id  word    word_count
1   accept  15+8
2   dad     14+4+2
3   happy   9+4
4   large   8+6

最终的结果就是

id  word    word_count
1   accept  23
2   dad     20
3   large   14
4   happy   13

我遇到了几个问题，包括让GetIndexTerms循环遍历这些单词是否是名词、动词等等。希望这一切都有意义吗？任何帮助都将不胜感激。谢谢。

Answer 1

我们可以使用dplyr执行以下操作

library(dplyr)
df %>% 
  group_by(syn_group) %>%
  mutate(sum_word_count = sum(word_count)) %>% 
  filter(rank == 1)

数据：

df <- read.table(text = "id  word    word_count  syn_group   rank
1   accept  15          1           1
5   admit   8           1           2
2   dad     14          2           1
8   daddy   4           2           2
9   papa    2           2           3
3   happy   9           3           1
7   glad    4           3           2
4   large   8           4           1
6   big     6           4           2", header = T)

请下次发布dput的输出。

编辑:这是一些代码，可以帮助您开始遍历单词和词类，并存储同义词。剩下的是确定当前术语是否是前一个同义词的同义词，在这种情况下，您已经拥有了同义词，并且您可以指定一个唯一的同义词组。接下来，您需要存储一些结果。最后，您需要计算排名，也就是seq_along (同义词)和一个grep来确定排名位置。这些注释是提示您可能希望包含这些提示的代码的位置。

d <- data.frame(Term = row.names(m), word_count = m[,1])
all_pos <- c("ADJECTIVE", "ADVERB", "NOUN","VERB")
syns <- vector("list", length(all_pos))
for(w in seq(nrow(d))){
  # if sysns of (d$Term[w]) has been calculated skip over current w 
  emf <- getTermFilter("ExactMatchFilter", d$Term[w], TRUE)  
  for(i in seq_along(syns)){
    terms <- getIndexTerms(all_pos[i], 1, emf)
    if(is.null(terms)){
      syns[i] <- NA
    } else{
      syns[[i]] <-  getSynonyms(terms[[1]])
    }
  }
  # store the results of syns for current w 
}