Python中的文本清理

Question

我是Python新手，无法找到删除无用文本的方法。其主要目的是保留我想要的单词，并移除其余的单词。在这个阶段，我可以检查我的in_data并找到我想要的单词。如果sentence.find(wordToCheck)是阳性的，那么保持它。in_data是每一行的句子，但是当前的输出是每行一个单词。我想要的是保留格式，在每一行中找到单词，然后删除其余的。

import Orange
import orange

word = ['roaming','overseas','samsung']
out_data = []

for i in range(len(in_data)):
    for j in range(len(word)):
        sentence = str(in_data[i][0])
        wordToCheck = word[j]
        if(sentence.find(wordToCheck) >= 0):
            print wordToCheck

输出

roaming
overseas
roaming
overseas
roaming
overseas
samsung
samsung

in_data的句子类似于

contacted vodafone about going overseas and asked about roaming charges. The customer support officer says there isn't a charge but while checking my usage overseas.

我希望看到输出就像

overseas roaming overseas

Answer 1

您可以为此使用regex：

>>> import re
>>> word = ['roaming','overseas','samsung']
>>> s =  "Contacted vodafone about going overseas and asked about roaming charges. The customer support officer says there isn't a charge but while checking my usage overseas."
>>> pattern = r'|'.join(map(re.escape, word))
>>> re.findall(pattern, s)
['overseas', 'roaming', 'overseas']
>>> ' '.join(_)
'overseas roaming overseas'

非正则表达式方法是将str.join与str.strip和生成器表达式结合使用。为了去掉标点符号(如'.'、','等)，需要带条()调用。

>>> from string import punctuation
>>> ' '.join(y for y in (x.strip(punctuation) for x in s.split()) if y in word)
'overseas roaming overseas'

Answer 2

你可以做的更简单，就像这样：

for w in in_data.split():
    if w in word:
        print w

这里，我们首先将in_data拆分为空格，它返回一个单词列表。然后，我们循环遍历in数据中的每个单词，并检查该单词是否等于您正在寻找的单词中的一个。如果是的话，我们就打印出来。

而且，为了更快地查找，请将word-list改为一个集合。快多了。

此外，如果要处理标点符号和符号，则需要使用regex或检查字符串中的所有字符是否为字母。因此，要获得所需的输出：

import string
in_words = ('roaming','overseas','samsung')
out_words = []

for w in in_data.split():
    w = "".join([c for c in w if c in string.letters])
    if w in in_words:
        out_words.append(w)
" ".join(out_words)

Answer 3

这里有一个更简单的方法：

>>> import re
>>> i
"Contacted vodafone about going overseas and asked about roaming charges. The customer support officer says there isn't a charge but while checking my usage overseas."
>>> words
['roaming', 'overseas', 'samsung']
>>> [w for w in re.findall(r"[\w']+", i) if w in words]
['overseas', 'roaming', 'overseas']