我有两个字符串数据,每个数据都被命名为数据集和口语。
数据口语表:
|**wrong** | **correct** |
|sheis | she is |
|taht | that |
|diedwhen | died when | 第二个dataframe是具有特定列的dataset,我使用的列如下所示:
数据集“‘review”:
review
[shewas, wrong, but, her, intentions, are, good]
[is, taht, you]
[he, diedwhen, he, was, young] 我想要改变数据集‘’review‘中的错误单词为正确的单词根据数据口语。我用这个:
normalizad_word = pd.read_excel("colloquial.xlsx")
normalizad_word_dict = {}
for index, row in normalizad_word.iterrows():
if row[0] not in normalizad_word_dict:
normalizad_word_dict[row[0]] = row[1]
def normalized_term(document):
return [normalizad_word_dict[term] if term in normalizad_word_dict else term for term in document]
dataset['review_normalized'] = dataset['review'].apply(normalized_term)
dataset['review_normalized'] 但结果是:
review_normalized
[she was, wrong, but, her, intentions, are, good]
[is, that, you]
[he, died when, he, was, young] 我想要这样:
review_normalized
[she, was, wrong, but, her, intentions, are, good]
[is, that, you]
[he, died, when, he, was, young] 发布于 2022-10-30 06:54:39
更改normalized_term函数:
def normalized_term(document):
result = []
for term in document:
if term in normalizad_word_dict:
for word in normalizad_word_dict[term].split(' '):
result.append(word)
else:
result.append(term)
return result或者如果您想使用内联循环:
import itertools
def normalized_term(document):
return list(itertools.chain(*[normalizad_word_dict[term].split() if term in normalizad_word_dict else term.split() for term in document]))https://stackoverflow.com/questions/74250905
复制相似问题