需要为大型语料库应用基于规则的算法来查找元素数组中存在的相似/相关关键字

Question

目前正在NLP工作空间中各自处理文本数据。我想通过基于搜索的方式找出基于列的实际给定的基于关键字的领域字典。

developer_position=['software engineer','florida','highest pay','startups']
analyst_position=['qa', 'testing','plsql']
data_science_position=['analytics lead','lead','python','R']
architect_position=['mongodb','technical architect','sql','java','kafka']
manager_position=['pmp certified','sixsigma', 'belt','delivery manager']

corpus=["software engineer positions are high demand in California",
   "qa average salary in USA is $120K-$150K",
   "Django & reactjs are minimum requirements for lead positions"]

输出应根据每个类别中的高概率关键字预测哪个类别位置将落入特定行

Answer 1

您可以在Python中使用基于spaCy规则的匹配，也可以在Javascript中使用winkNLP自定义实体或coreNLP的令牌正则表达式。