如何使用soup.find，soup.find_all

Question

这是我的代码和输出

从bs4导入BeautifulSoup导入请求

res = requests.get("https://www.jobberman.com/jobs")
soup = BeautifulSoup(res.text, "html.parser")
job = soup.find("div", class_ = "relative inline-flex flex-col w-full text-sm font-normal pt-2")
company_name = job.find('a[href*="jobs"]')
print(company_name)

输出为零

None

但是当我使用select方法时，我得到了想要的结果，但不能在上面使用.text。

import requests
from bs4 import BeautifulSoup

res = requests.get("https://www.jobberman.com/jobs")
soup = BeautifulSoup(res.text, "html.parser")
job = soup.find("div", class_ = "relative inline-flex flex-col w-full text-sm font-normal pt-2")
company_name = job.select('a[href*="jobs"]').text
print(company_name)

输出

AttributeError: ResultSet object has no attribute 'text'. You're probably treating a list of elements like a single element. Did you call find_all() when you meant to call find()?

Answer 1

改变你的选择策略--因为这里的主要问题是，并不是所有的公司名称都有关联：

job.find('div',{'class':'search-result__job-meta'}).text.strip()

或

job.select_one('.search-result__job-meta').text.strip()

示例

还以结构化的方式存储您的信息，以便进行后处理：

import requests
from bs4 import BeautifulSoup

res = requests.get("https://www.jobberman.com/jobs")
soup = BeautifulSoup(res.text, "html.parser")
data = []
for job in soup.select('div:has(>.search-result__body)'):
    data.append({
        'job':job.h3.text,
        'company':job.select_one('.search-result__job-meta').text.strip()
    })
data

输出

[{'job': 'Restaurant Manager', 'company': 'Balkaan Employments service'},
 {'job': 'Executive Assistant', 'company': 'Nolla Fresh & Frozen ltd'},
 {'job': 'Portfolio Manager/Instructor 1', 'company': 'Fun Science World'},
 {'job': 'Microbiologist', 'company': "NEIMETH INT'L PHARMACEUTICALS PLC"},
 {'job': 'Data Entry Officer', 'company': 'Nkoyo Pharmaceuticals Ltd.'},
 {'job': 'Chemical Analyst', 'company': "NEIMETH INT'L PHARMACEUTICALS PLC"},
 {'job': 'Senior Front-End Engineer', 'company': 'Salvo Agency'},...]

Answer 2

你的搜索策略的问题已经被先前发布的评论和答案所涵盖。我为您的问题提供了一个解决方案，包括使用regex库以及find_all()函数调用：

    import requests
    from bs4 import BeautifulSoup
    import re
    
    res = requests.get("https://www.jobberman.com/jobs")
    soup = BeautifulSoup(res.text, "html.parser")
    company_name = soup.find_all("a", href=re.compile("/jobs\?"), rel="nofollow")
    for i in range(len(company_name)):
        print(company_name[i].text)

输出：

GRATIAS DEI NIGERIA LIMITED

Balkaan Employments service

Fun Science World

NEIMETH INT'L PHARMACEUTICALS PLC

Nkoyo Pharmaceuticals Ltd.

...