写入selenium数据时空CSV

Question

我第一次体验Selenium，并在一个著名的视频平台上做了一些教程。大多数情况下，它的工作非常可靠。但是，我在创建CSV但没有导出数据的几个页面上遇到了问题。CSV被“触摸”，但它不导出正常打印中显示的数据。

有人能帮我找到这个脚本的问题所在吗？

#_*_coding: utf-8_*_


from selenium import webdriver
import selenium
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
import csv
import os

os.chdir("C:\Selenium")
PATH = "chromedriver.exe"
driver = webdriver.Chrome(PATH)

driver.get("https://twitter-trends.iamrohit.in/")

try: 
    main = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.CLASS_NAME, "panel-body"))
    )
 
    main = (main.text)
 
    f = open('twitter.csv', 'wb')
    print(main, file = f)
    f.close()
    
    #print(main)

except:
    driver.quit()

driver.quit()

Python版本3.7.4，Selenium Versionm 3.141.0，Windows 10

Answer 1

调试您的代码，您将正确地获取数据。

   main = (main.text)
   print(main)
   f = open('twitter.csv', 'wb')

因此，错误是当您写入输出文件时。将代码替换为

main = (main.text)

with open('twitter.txt', 'wb', encoding='utf-8') as file1:
# Writing data to a file
    file1.writelines(main)

将工作，如果您检查打印您的汉字，将使写入输出文件失败。

Answer 2

要刮掉推特趋势-全球表，可以使用Pandas中的DataFrame并使用下面的定位器策略将其写入csv文件

代码块：

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
import pandas as pd

driver.get("https://twitter-trends.iamrohit.in/")
driver.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//b[text()='Note:']"))))
headers = [my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "thead > tr > th")))]
ranks = [my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//table[@id='twitter-trends']//tbody//tr//descendant::th[1]")))]
topics = [my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//table[@id='twitter-trends']//tbody/tr//descendant::th[2]/a")))]
volumes = [my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//table[@id='twitter-trends']//tbody/tr//descendant::th[3]")))]
df = pd.DataFrame(data=list(zip(ranks, topics, volumes)), columns=headers)
df.to_csv(r'C:\Data_Files\output_files\twitter.csv', index=False)
driver.quit()

CSV快照：

​

​

参考文献

您可以在以下网站找到几个相关的详细讨论：

• 熊猫DF.output写入列(当前数据全部写入一行或一列)
• Selenium:从Coincodex中抓取历史数据并转化为Pandas Dataframe
• 抓取表并将其写入dataframe向我展示TypeError

Answer 3

从硒进口selenium.webdriver.chrome.options进口工具从selenium.webdriver.chrome.service进口选项从selenium.webdriver.support.ui进口WebDriverWait从selenium.webdriver.common.by进口从selenium.webdriver.support进口expected_conditions进口熊猫作为pd Driver.get(“https://twitter-trends.iamrohit.in/"”)driver.execute_script(“返回arguments.scrollIntoView(True)”；"，20).until(EC.visibility_of_element_located((By.XPATH，(driver，WebDriverWait“//btext()=‘Note：’”)( headers = my_elem.text for my_elem in WebDriverWait(driver，20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR，"thead > tr >th“))20).until(EC.visibility_of_all_elements_located((By.XPATH，"//table@id='twitter-trends'//tbody//tr//descendant::th1")))]主题= [my_elem.text for my_elem in WebDriverWait(驱动程序，20).until(EC.visibility_of_all_elements_located((By.XPATH，)"//table@id='twitter-trends'//tbody/tr//descendant::th2/a")))] volumes = [my_elem.text for my_elem in WebDriverWait(驱动程序，20).until(EC.visibility_of_all_elements_located((By.XPATH，"//table@id='twitter-trends'//tbody/tr//descendant::th3")))] df =pd.DataFrame)(data=list(等级、主题、卷))，( df.to_csv(r'C:\Data_Files\output_files\twitter.csv'，index=False) driver.quit()可以提供这个有用的推特趋势