我正在学习Python3进行网络抓取,下面是我的代码:
import requests
url = 'https://beijing.douban.com/events/week-all'
req = requests.get(url)
from bs4 import BeautifulSoup
soup = BeautifulSoup(req.text,'html.parser')
events = soup.find('ul',{'class':'events-list'}).findAll('li')
for event in events:
event_details = dict()
event_details['title']= event.find('div',{'class':'title'})
f = open('douban8.txt','a',encoding='utf-8')
print(event_details,file=f)我得到的结果如下:
**{'title': <div class="title">
<a href="https://www.douban.com/event/31568343/" itemprop="url" title="年度动作奇幻大片《海王》北京IMAX 3D零点场免费观影来了!">
<span itemprop="summary">年度动作奇幻大片《海王》北京IMAX 3D零点场免费观影来了!</span>
</a>
</div>}
{'title': None}
{'title': None}
{'title': None}
{'title': None}**我想要的只是标签之间的字符串,但是我不能从整个文本中去掉标记。我尝试了.string或.text以及其他一些方法,都失败了。有人能给我一些建议吗?
感谢@sohan-das,我在这里更新代码:
import requests
url = 'https://beijing.douban.com/events/week-all'
req = requests.get(url)
from bs4 import BeautifulSoup
soup = BeautifulSoup(req.text,'html.parser')
events = soup.find('ul',{'class':'events-list'}).findAll('li')
for event in events:
event_details=dict()
try:
event_details['title'] = event.find('div'{'class','title'}).text.strip()
f = open('douban9.txt', 'a', encoding='utf-8')
print(event_details, file=f)
except AttributeError:
pass
continue它现在起作用了
发布于 2018-12-05 06:49:29
event_details需要列表,而不是dict,并将其置于循环之外。元素在events列表中有超过10个元素,您需要对其进行筛选。
.....
events = soup.find('ul', {'class':'events-list'}).findAll('li')
event_details = []
for event in events:
event_title = event.find('div', {'class':'title'})
# if element not have div.title skip
if event_title:
event_details.append({'title' : event_title.text.strip()})
f = open('douban8.txt','a',encoding='utf-8')
print(event_details,file=f)结果
[
{'title': '年度动作奇幻大片《海王》北京IMAX 3D零点场免费观影来了!'},
....,
{'title': '在被认可前,先让自己更专业!「42时」纪录片导演制作班第二季'}
]https://stackoverflow.com/questions/53626020
复制相似问题