用python lxml抓取airbnb

Question

我正试图在airbnb列表上找到一个节点。节点是

< div class="col-md-3 text-muted" data-reactid=".2e7if3twveo.0.0.0.0.1.6.0">< span data-reactid=".2e7if3twveo.0.0.0.0.1.6.0.0">The Space< /span> /div> 

import mechanize

br = mechanize.Browser()

url ='https://www.airbnb.com/rooms/5711344'
tree = html.fromstring(br.open(url).get_data())
els = tree.xpath('//div[@class="row"]/div[@class="col-md-3 text-muted"]')
for element in els:
    if element.text.find('The Space') >= 0:

不知何故，“太空”是无法挽回的。

Answer 1

这对我来说是可行的:我使用BeautifulSoup通过类属性获取div，然后循环得到正确的div。

import requests
from bs4 import BeautifulSoup

url = 'https://www.airbnb.com/rooms/5711344'
html = requests.get(url)
soup = BeautifulSoup(html.text, 'html.parser')
divs = soup.find_all('div', attrs={'class': 'col-md-3 text-muted'})
for div in divs:
    space = div.find('span').text.strip()
    if space == "The Space":
        print(space)