为下拉表发送具有指定值的request.get

Question

我需要为网页链接使用requests.get：

https://www.investing.com/crypto/bitcoin/btc-usd-historical-data

默认值是bitfinex，但我需要Kraken的页面，可以从下拉菜单中选择。我在下面包含了html代码。我没有看到一个可以选择然后作为数据参数传递的选项值，那么我该如何使用requests.get设置这个下拉列表的值呢？

<div class="clear"></div><div class="exchangeDropdownContainer             float_lang_base_1 instDropDown" id="DropDownContainer">
<a id="DropdownBtn" class="newBtnDropdown inlineblock ">
                <span class="ceFlags bitcoin">&nbsp;</span>
            <i class="btnTextDropDwn arial_12 bold">Bitfinex</i>
                <i class="bottunImageDoubleArrow buttonWhiteImageDownArrow">&nbsp;</i>
        </a>
        <table class="genTbl closedTbl exchangeDropdownTbl displayNone" id="DropdownSiblingsTable">
        <thead>
        <tr>
            <th class="center">&nbsp;</th>
            <th class="left">Symbol</th>
            <th class="left">Exchange</th>
            <th class="center">Currency</th>
            <th class="left">&nbsp;</th>
        </tr>
        </thead>
        <tbody>
                        <tr data-href="/crypto/bitcoin/btc-usd-historical-data" class="selected">
                <td class="center"><i class="ceFlags bitcoin">&nbsp;</i></td>
                <td class="left bold"><a href="/crypto/bitcoin/btc-usd-historical-data">BTC/USD</a></td>
                <td class="left">Bitfinex</td>
                <td class="center">USD</td>
                <td class="left lightgrayFont bold">Real-time</td>
            </tr>
                        <tr data-href="/indices/investing.com-btc-usd-historical-data" >
                <td class="center"><i class="ceFlags bitcoin">&nbsp;</i></td>
                <td class="left bold"><a href="/indices/investing.com-btc-usd-historical-data">BTC/USD</a></td>
                <td class="left">Investing.com</td>
                <td class="center">USD</td>
                <td class="left lightgrayFont bold">Real-time</td>
            </tr>
                        <tr data-href="/crypto/bitcoin/btc-usd-historical-data?cid=53082" >
                <td class="center"><i class="ceFlags bitcoin">&nbsp;</i></td>
                <td class="left bold"><a href="/crypto/bitcoin/btc-usd-historical-data?cid=53082">BTC/USD</a></td>
                <td class="left">OKCoin</td>
                <td class="center">USD</td>
                <td class="left lightgrayFont bold">Real-time</td>
            </tr>
                        <tr data-href="/crypto/bitcoin/btc-usd-historical-data?cid=1010780" >
                <td class="center"><i class="ceFlags bitcoin">&nbsp;</i></td>
                <td class="left bold"><a href="/crypto/bitcoin/btc-usd-historical-data?cid=1010780">BTC/USD</a></td>
                <td class="left">Poloniex</td>
                <td class="center">USD</td>
                <td class="left lightgrayFont bold">Real-time</td>
            </tr>
                        <tr data-href="/crypto/bitcoin/btc-usd-historical-data?cid=1010796" >
                <td class="center"><i class="ceFlags bitcoin">&nbsp;</i></td>
                <td class="left bold"><a href="/crypto/bitcoin/btc-usd-historical-data?cid=1010796">BTC/USD</a></td>
                <td class="left">GDAX</td>
                <td class="center">USD</td>
                <td class="left lightgrayFont bold">Real-time</td>
            </tr>
                        <tr data-href="/crypto/bitcoin/btc-usd-historical-data?cid=53073" >
                <td class="center"><i class="ceFlags bitcoin">&nbsp;</i></td>
                <td class="left bold"><a href="/crypto/bitcoin/btc-usd-historical-data?cid=53073">BTC/USD</a></td>
                <td class="left">ANX</td>
                <td class="center">USD</td>
                <td class="left lightgrayFont bold">Real-time</td>
            </tr>
                        <tr data-href="/crypto/bitcoin/btc-usd-historical-data?cid=49799" >
                <td class="center"><i class="ceFlags bitcoin">&nbsp;</i></td>
                <td class="left bold"><a href="/crypto/bitcoin/btc-usd-historical-data?cid=49799">BTC/USD</a></td>
                <td class="left">Kraken</td>
                <td class="center">USD</td>
                <td class="left lightgrayFont bold">Real-time</td>
            </tr>

我需要将下拉表设置为Kraken选项

Answer 1

将请求传递到以下url

https://www.investing.com/crypto/bitcoin/btc-usd-historical-data?cid=49799

只需更改基本url页面中的下拉列表，您就可以看到cid参数已添加到其中。在做网页抓取的时候，与你将要抓取的网站打交道是很重要的。

import requests
headers = {'User-Agent' : 'Mozilla/5.0 (Macintosh; Intel Mac OS X x.y; rv:42.0) Gecko/20100101 Firefox/42.' }
url="https://www.investing.com/crypto/bitcoin/btc-usd-historical-data?cid=49799"
response=requests.get(url,headers=headers)
print(response.text)