在BeautifulSoup和Git Bash上进行网络抓取并传输到CSV

Question

所以我一直在网上抓取一个有表格的网站，理想情况下，我想在网上抓取成excel表格，并将其保存在表格中，我将输入我所拥有的，我已经使用了scrapy和BeautifulSoup，我对这两个都有问题。如果能帮上忙就太好了！

import requests
import csv
from bs4 import BeautifulSoup

url = 'https://pcpartpicker.com/products/video-card/'
r = requests.get(url)
html = r.text

soup = BeautifulSoup(html, 'lxml')

name = soup.find('tbody', {"id":"category_content"})

print(name.text)
~

Answer 1

学习在Splash中使用Selenium或Scrapy，我的小任务推荐使用Selenium，你可以在一天内学会基础知识。

from selenium import webdriver
from bs4 import BeautifulSoup as bs
import time
options = webdriver.ChromeOptions()
#install chrome if none and download chromedriver and add path to it
driver = webdriver.Chrome(executable_path="D:/Python/chromedriver", options=options)
driver.get("https://pcpartpicker.com/products/video-card/")
time.sleep(2)
soup = bs(driver.page_source,'lxml')
name = soup.find('tbody', {"id":"category_content"})
for i in name:
        print(i.find('a').text)