我有一个包含一列文章标题的csv,我使用SpaCy从其中提取出现在标题中的任何人的姓名。当尝试使用由SpaCy提取的名称向csv添加新列时,它们不会与从中提取它们的行对齐。
我相信这是因为SpaCy结果有自己的索引,独立于原始数据的索引。
我尝试将索引添加到新的列行中,但得到"ValueError:传递的值的长度是2,索引意味着10“。
如何将SpaCy输出与它们所在的行对齐?
下面是我的代码:
import pandas as pd
from pandas import DataFrame
df = (pd.read_csv(r"C:\Users\Admin\Downloads\itsnicethat (5).csv", nrows=10,
usecols=['article_title']))
article = [_ for _ in df['article_title']]
import spacy
nlp = spacy.load('en_core_web_lg')
doc = nlp(str(article))
ents = list(doc.ents)
people = []
for ent in ents:
if ent.label_ == "PERSON":
people.append(ent)
import numpy as np
df['artist_names'] = pd.Series(people)
print(df.head())这是生成的数据帧:
article_title artist_names
0 “They’re like, is that? Oh it’s!” – ... (Hannah, Ward)
1 Billed as London’s biggest public festival of ... (Dylan, Mulvaney)
2 Transport yourself back to the dusky skies and... NaN
3 Turning to art at the beginning of quarantine ... NaN
4 Dylan Mulvaney, head of design at Gretel, expl... NaN这就是我所期待的:
article_title artist_names
0 “They’re like, is that? Oh it’s!” – ... (Hannah, Ward)
1 Billed as London’s biggest public festival of ... NaN
2 Transport yourself back to the dusky skies and... NaN
3 Turning to art at the beginning of quarantine ... NaN
4 Dylan Mulvaney, head of design at Gretel, expl... (Dylan, Mulvaney)您可以看到artist_names列中的第5个值与第5个文章标题相关。我如何才能使它们对齐?
谢谢你的帮助。
发布于 2021-03-24 20:21:30
我将遍历文章,分别检测每个文章中的实体,并将检测到的实体放在一个列表中,每个文章一个元素:
nlp = spacy.load('en_core_web_lg')
article = [_ for _ in df['article_title']]
entities_by_article = []
for doc in nlp.pipe(article):
people = []
for ent in doc.ents:
if ent.label_ == "PERSON":
people.append(ent)
entities_by_article.append(people)
df['artist_names'] = pd.Series(entities_by_article)注意:for doc in nlp.pipe(article)是spaCy更高效的遍历文本列表的方法,可以替换为:
for a in article:
doc = nlp(a)
## rest of code within loop发布于 2021-03-24 19:57:15
if ent.label_ == "PERSON":
people.append(ent)
else:
people.append(np.nan) # if ent.label_ is not a PERSON包括else语句,因此如果PERSON不是PERSON,则将其视为NaN。
https://stackoverflow.com/questions/66780391
复制相似问题