我正在做一些精美的汤。在从下一页抓取值时,我使用for循环。一切都很好,但是当我列出抓取值的列表时,我只得到了最后一页的值。下面是我的代码。
from bs4 import BeautifulSoup as bs
import requests
params = []
for page_number in range(0, 4):
p = page_number*10
params.append(p)
print(params)
gymname_list = []
gymratings_list =[]
gymnumreviews_list = []
gymcat_list = []
for i in params:
headers = {'User-Agent':'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/601.3.9 (KHTML, like Gecko) Version/9.0.2 Safari/601.3.9'}
url= f'https://www.yelp.com.au/search?find_desc=gyms&find_loc=Berlin%2C%20Germany&start={i}'
response=requests.get(url,headers=headers)
page_soup=bs(response.content,'lxml')
mains = page_soup.find_all("div", {"class": "mainAttributes__09f24__26-vh arrange-unit__09f24__3IxLD arrange-unit-fill__09f24__1v_h4 border-color--default__09f24__1eOdn"})
for main in mains:
try:
gymname = main.find("a", {"class": "css-166la90"}).text
print(gymname)
except:
print(None)
gymname_list.append(gymname)在上面的代码中,正如你所看到的,我试图抓取前四页,但当我打印“gym name”时,我得到的只是最后一页的健身房名称,即第四页的结果。我希望所有的结果都在我的列表中。gymname_list。请帮帮忙
发布于 2021-07-10 13:56:28
在最后一个for循环中,缩进应该在这个for循环中
for main in mains:
try:
gymname = main.find("a", {"class": "css-166la90"}).text
print(gymname)
except:
print(None)
gymname_list.append(gymname)https://stackoverflow.com/questions/68316682
复制相似问题