我有我的临时名单,我想用它从文本中删除特定的短语:
#dummy text
df2 <- c("hi my name is Ann and code code all the time! However not after that I would like")
mystopwords <- c("hi", "code code", "not after that")我使用这个选项:
myDfm <- df2 %>%
tokens(remove_punct = TRUE, remove_numbers = TRUE, remove_symbols = TRUE) %>%
tokens_remove(pattern = c(stopwords(source = "smart"), mystopwords)) %>%
tokens_wordstem() %>%
tokens_ngrams(n = c(1, 3)) %>%
dfm()但是,当我检查双克或三联图的频率时,它们并没有被删除,只是词干而已。
语法上有什么问题吗?
发布于 2019-02-06 15:39:38
当您使用停止短语列表时,您可以通过使用phrase()函数来实现这一点。
它的工作方式如下:
library(quanteda)
df2 <- c("hi my name is Ann and code code all the time! However not after that I would like")
mystopwords <- c("hi", "code code", "not after that")
df2 %>% tokens %>%
tokens_remove(pattern = phrase(mystopwords), valuetype = 'fixed')
## tokens from 1 document.
## text1 :
## [1] "my" "name" "is" "Ann" "and" "all" "the" "time" "!" "However" "I" "would"
## [13] "like" 您可以在quanteda中获得有关如何使用多个单词表达式的详细信息:https://quanteda.io/articles/pkgdown/examples/phrase.html
https://stackoverflow.com/questions/54550990
复制相似问题