为什么在python中使用.get_text()和漂亮的汤时会出现错误？

Question

我正试图在youtube上跟随这个turorial来制作一个网络刮刀，我到达了他试图获取产品标题并将其打印到屏幕上的某个部分。然而，当我尝试这样做的时候，我得到了一个关于.get_text()的错误，我不明白当我复制他的所有东西时，我哪里出了错。

另一件事是，当他输入soup.时，它会为他显示一个课程列表，而不是给我，为什么会这样呢？我是在visual studio代码btw中这样做的。

import requests 
from bs4 import BeautifulSoup

URL = "https://www.amazon.com/Nintendo-Console-Resolution-802-11ac-Surround/dp/B07RGFF98S/ref=sr_1_2?crid=213XSEHLOFP4W&dchild=1&keywords=nintendo+switch&qid=1599079468&sprefix=nintendo%2Caps%2C395&sr=8-2"

headers = {"User-Agent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36'}

page = requests.get(URL, headers=headers)

soup = BeautifulSoup(page.content, 'html.parser')

title = soup.find(id="productTitle").get_text()

print(title.strip())

Answer 1

要从亚马逊服务器(而不是验证码页面)获得正确的响应，还需要指定Accept-Language HTTP header：

import requests
from bs4 import BeautifulSoup

URL = "https://www.amazon.com/Nintendo-Console-Resolution-802-11ac-Surround/dp/B07RGFF98S/ref=sr_1_2?crid=213XSEHLOFP4W&dchild=1&keywords=nintendo+switch&qid=1599079468&sprefix=nintendo%2Caps%2C395&sr=8-2"

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36',
    'Accept-Language': 'en-US,en;q=0.5'
}

page = requests.get(URL, headers=headers)

soup = BeautifulSoup(page.content, 'html.parser')

title = soup.find(id="productTitle").get_text()
print(title.strip())

打印：

Nintendo Switch 32GB Console Video Games w/ 32GB Memory Card | Neon Red/Neon Blue Joy-Con | 1080p Resolution | 802.11ac WiFi | HDMI | Surround Sound | IR Motion Camera