如何在没有html类的情况下从单行文本中提取信息？

Question

我正在尝试使用scrapy和python.The抓取我的第一个网站(https://news.ycombinator.com/jobs)，我需要提取的信息如下：-正在招聘的公司的名称-公司的位置-广告招聘的职位

页面中的这些字段没有单独的标记，html.And文本没有特定的模式。例如，ZeroCater (YC W11)正在聘请科幻小说《必须热爱食物》的首席工程师

仅有正则表达式不足以提取此信息。这个问题有没有简单有效的解决方案？

我尝试过python regex。我还学习了NLP和nltk的文本分类。但是nltk会增加代码的复杂度，而且很耗时。

Answer 1

在这种情况下，我要做的是尝试找到任何模式来帮助我提取这些数据，例如，我可以看到这些词是常见的"is hiring|is looking for|is looking to hire|hiring"和公司名称在前，位置在in之后

这只是一个小的试验，你可以扩展它来获得你需要的东西

import re
text = """ZeroCater (YC W11) Is Hiring a Principal Engineer in SF: Must Love Food (zerocater.com)
OneSignal Is Hiring Full Stack Engineers in San Mateo (onesignal.com)
Faire (YC W17) Is Looking to Hire Business Operations Leads (greenhouse.io)
InsideSherpa (YC W19) Is Hiring Software Engineers in Sydney (workable.com)
Jerry (YC S17) Is Hiring Senior Software Dev, Data Engineer (Toronto/Remote) (getjerry.com)
Iris Automation Is Hiring an Account Executive for B2B Flying Vehicle Software (irisonboard.com)"""

data = text.lower().splitlines()

for i, line in enumerate(data):
    # getting company name
    data[i] = re.split(r'is hiring|is looking for|is looking to hire|hiring', line)

    # job title and location if present
    data[i][1] = re.split(r' in ', data[i][1])

print('company --- Job Title --- Location')
for c in data:
    print(f'{c[0]} --- {c[1][0]} --- {c[1][1] if len(c[1])>1 else ""}')

输出：

company --- Job Title --- Location
zerocater (yc w11)  ---  a principal engineer --- sf: must love food (zerocater.com)
onesignal  ---  full stack engineers --- san mateo (onesignal.com)
faire (yc w17)  ---  business operations leads (greenhouse.io) --- 
insidesherpa (yc w19)  ---  software engineers --- sydney (workable.com)
jerry (yc s17)  ---  senior software dev, data engineer (toronto/remote) (getjerry.com) --- 
iris automation  ---  an account executive for b2b flying vehicle software (irisonboard.com) --- 

当然，这段代码需要进行大量修改才能获得可靠的结果，但至少这是一个开始