网络抓取表

Question

我一直在拉年度收入表，而我本应该每季度拉一次。有人能解释一下我做错了什么吗？(代码如下)

url='https://www.macrotrends.net/stocks/charts/TSLA/tesla/revenue'

html_data=requests.get(url).text

soup=BeautifulSoup(html_data, 'html.parser')

tesla_revenue=pd.DataFrame(columns=['Date', 'Revenue'])
for row in soup.find('tbody').find_all('tr'):
    col=row.find_all('td')
    date=col[0]
    revenue=col[1]
    tesla_revenue=tesla_revenue.append({'Date':date,'Revenue':revenue}, ignore_index=True)

tesla_revenue.head()

Answer 1

季度数据位于第二个表([1])中：

url = "https://www.macrotrends.net/stocks/charts/TSLA/tesla/revenue"

html_data = requests.get(url).text

soup = BeautifulSoup(html_data, "html.parser")

table = soup.select("table")[1]

all_data = []
for row in table.find("tbody").find_all("tr"):
    col = row.find_all("td")
    date = col[0].text
    revenue = col[1].text
    all_data.append({"Date": date, "Revenue": revenue})

tesla_revenue = pd.DataFrame(all_data)
print(tesla_revenue.head())

打印：

         Date  Revenue
0  2021-06-30  $11,958
1  2021-03-31  $10,389
2  2020-12-31  $10,744
3  2020-09-30   $8,771
4  2020-06-30   $6,036

Answer 2

我认为你应该提取文本。您在代码中编写的代码如下所示。

date=col[0]
revenue=col[1]

你应该像下面这样修改它。

date=col[0].text.strip()
revenue=col[1].text.strip()