如何使用Python登录亚马逊子公司Audible.com

Question

我想要刮声音网站使用Python美丽的汤。有一些数据我无法访问，除非我登录到我的音频帐户。它是Amazon.com的子公司。我一直没有成功。我只想使用Python登录并抓取html。

我已经尝试了各种代码，比如这个How to login to Amazon using BeautifulSoup。有人会认为，在这段代码中简单地替换我的凭据就可以了。

Answer 1

不幸的是，在Python中这不再是简单的自动化。这是我用Audible AU所能达到的最大值。POST需要一堆标头，其中大多数都可以提取，除了metadata1 (在底部有更多信息)：

"""load packages"""
import requests
from bs4 import BeautifulSoup
from urllib.parse import urlsplit, parse_qs

"""define URL where login form is located"""
site = "https://www.audible.com.au/signin"

"""initiate session"""
session = requests.Session()

"""define session headers"""
session.headers = {
    "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
    "accept-encoding": "gzip, deflate, br",
    "accept-language": "en-US,en;q=0.9,cs;q=0.8",
    "sec-fetch-dest": "document",
    "sec-fetch-mode": "navigate",
    "sec-fetch-site": "none",
    "upgrade-insecure-requests": "1",
    "user-agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36",
    "metadata1": "",
}

"""get login page"""
resp = session.get(site)
html = resp.text

"""extract clientContext from the login page"""
query = urlsplit(resp.url).query
params = parse_qs(query)
clientContext = params["clientContext"]
new_login_url = "https://www.amazon.com.au/ap/signin/" + str(clientContext[0])

"""get BeautifulSoup object of the html of the login page"""
soup = BeautifulSoup(html, "lxml")

"""scrape login page to get all the needed inputs required for login"""
data = {}
form = soup.find("form", {"name": "signIn"})
for field in form.find_all("input"):
    try:
        data[field["name"]] = field["value"]
    except:
        pass

"""add username and password to the data for post request"""
data[u"email"] = "EMAIL"
data[u"password"] = "PASSWORD"

"""display: redirect URL, appActionToken, appAction, siteState, openid.return_to, prevRID, workflowState, create, email, password"""
print(new_login_url, data)

"""submit post request with username / password and other needed info"""
post_resp = session.post(new_login_url, data=data, allow_redirects=True)
post_soup = BeautifulSoup(post_resp.content, "lxml")

"""check the captcha"""
warning = post_soup.find("div", id="auth-warning-message-box")
if warning:
    print("Warning:", warning)
else: print(post_soup)

session.close()

在48和49行上添加您的电子邮件地址和密码。另外，使用浏览器登录并检查流量，以查看计算机上的metadata1，并将其添加到22行。如果你很幸运，你不会被检测到是机器人，你会进入，否则你会得到验证码图像。

metadata1是base64中的一个巨大的有效负载，它由浏览器收集的数据组成，这些数据唯一地识别你，并将你与机器人区分开来(鼠标点击、输入延迟、页面脚本、浏览器信息和兼容性&扩展、flash版本、用户代理、脚本性能、硬件图形处理器、本地存储、画布大小等)。