从亚马逊搜索中抓取书目进入终端

Question

我正在尝试修改我的代码，以便它将亚马逊搜索中的标题打印到终端上。我该怎么做呢？我只是在最近才真正让它打印出至少一些来自亚马逊的东西到终端中，在经历了大量的尝试之后，这是一个错误。

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
import time

PATH = "/Users/fuadhafiz/Documents/Python Projects/Selenium Automation /Web Scraping (1)/chromedriver"
driver = webdriver.Chrome(PATH)
driver.get("https://www.amazon.co.uk/")

search = driver.find_element_by_id("twotabsearchtextbox")
search.send_keys("usb c to hdmi")
search.send_keys(Keys.RETURN)

try:
    main = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, ".s-main-slot")))
    print(main.text)
except TimeoutException:
    print("Timeout")

print(main)

driver.quit()

到目前为止，它将整个页面打印到终端上。下面是打印到终端上的结果中的一小部分：

USB C to HDMI
Shop BENFEI 
USB C to HDMI Cable(4K@60Hz), Benfei USB Type-C to HDMI Cable [Thunderbolt 3] Compatible for MacBook Pro 2019/2018/2017, Samsung Galaxy S9/S8, Surface Book 2, Dell XPS 13/15 and More - 1.8M, Grey
268
USB C to HDMI Adapter 4K@60Hz, BENFEI USB Type-C to HDMI Adapter [Thunderbolt 3 Compatible] for MacBook Pro 2018/2017, MacBook Air/iPad Pro 2018, Samsung Galaxy S10/S9, Surface Book 2 and More - Slim
133
USB C Hub, BENFEI 5 in 1 Type C to HDMI 4K Adapter with 2 USB Ports, SD/Micro SD Card Reader, for MacBook Pro 2019/2018/2017, iPad Pro 2019/2018, Pixelbook, XPS, and More
142
Sponsored
Sponsored
iVANKY USB C to HDMI Cable, [2020 Updated Version] 4K@60Hz Nylon Braided Cable, Type C [Thunderbolt 3 Compatible] to HDMI, for MacBook Pro/Air, iPad, Surface Book, Samsung Galaxy and More - 6.6ft, 2M
352
£14
99
£15.99

Answer 1

为什么不使用CSS选择器选择所有结果并将其打印出来？

下面是操作步骤：

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

from selenium.webdriver.chrome.options import Options

options = Options()
options.headless = False
driver = webdriver.Chrome(options=options)

driver.get("https://www.amazon.co.uk/")

search = driver.find_element_by_id("twotabsearchtextbox")
search.send_keys("usb c to hdmi")
search.send_keys(Keys.RETURN)

search_results = driver.find_elements_by_css_selector('h1:last-child, h2:last-child, h3:last-child, h4:last-child')

for result in search_results[1:]:
    print(result.text)

driver.quit()

输出：

iVANKY USB C to HDMI Cable, [2020 Updated Version] 4K@60Hz Nylon Braided Cable, Type C [Thunderbolt 3 Compatible] to HDMI, for MacBook Pro/Air, iPad, Surface Book, Samsung Galaxy and More - 6.6ft, 2M
AmazonBasics Premium Aluminium USB-C to HDMI Cable Adapter (Thunderbolt 3 Compatible) 4K@60Hz, 0.9 metres
USB C to HDMI Adapter, Type c to HDMI 4K Adapter (Thunderbolt 3 Compatible) with Video Audio Output for MacBook Pro 2018/2017/2016, Samsung Note 9/S9/Note 8/S8, Huawei Mate 20 and More (Black)
USB C to HDMI Cable 4K, uni【2020 New Updated】USB Type C to HDMI Cable(Thunderbolt 3 Compatible), Compatible with iPad Pro 2020, MacBook Pro, Samsung S20, Huawei Mate40, XPS and More - 6ft/1.8m
USB C to HDMI Cable【4K@60Hz High Refresh Rate】uni Type-C [Thunderbolt 3] to HDMI Cable Compatible with MacBook Pro, MacBook Air, iPad Pro, Surface Book 2, Galaxy S20, Dell XPS and More - 0.9m/3ft
QGeeM USB C to HDMI Cable Adapter 6ft 4K@30Hz, USB Type C to HDMI Cable Thunderbolt 3 Compatible with MacBook Pro 2018 IPad pro,Samsung S9 S10,Surface Book 2,Dell XPS 13/15,Pixelbook More
USB C to Dual HDMI Adapter, Thunderbolt 3 USB Type C to 2 HDMI Monitor Adapter for Surface Book 2/Go/Pro 7,Dell XPS 13/15, Lenovo Yoga 720/900/910/920,Macbook Pro Air, iPad Pro and more
USB C to HDMI Adapter 4K for MacBook Pro, HDMI Adapter Hub Dock with USB 3.0 Port, Type-C PD Charging Port Compatible w/Nintendo Switch/MacBook Air 2018/Chromebook Pixel/Dell XPS13 - Space Grey
and so on ...

Answer 2

从搜索结果中获取每个项目的标题。诱导WebDriverWait()并等待presence_of_all_elements_located()和随后的css selector。

allprodtitle =WebDriverWait(driver,10).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR,".s-main-slot h2>a>span")))
for item in allprodtitle:
    print("Title: " + item.get_attribute("innerHTML"))

控制台输出：

Title: iVANKY USB C to HDMI Cable, [2020 Updated Version] 4K@60Hz Nylon Braided Cable, Type C [Thunderbolt 3 Compatible] to HDMI, for MacBook Pro/Air, iPad, Surface Book, Samsung Galaxy and More - 6.6ft, 2M
Title: AmazonBasics Premium Aluminium USB-C to HDMI Cable Adapter (Thunderbolt 3 Compatible) 4K@60Hz, 0.9 metres
Title: USB C to HDMI Adapter, Type c to HDMI 4K Adapter (Thunderbolt 3 Compatible) with Video Audio Output for MacBook Pro 2018/2017/2016, Samsung Note 9/S9/Note 8/S8, Huawei Mate 20 and More (Black)
Title: USB C to HDMI Cable 4K, uni【2020 New Updated】USB Type C to HDMI Cable(Thunderbolt 3 Compatible), Compatible with iPad Pro 2020, MacBook Pro, Samsung S20, Huawei Mate40, XPS and More - 6ft/1.8m
Title: USB C to HDMI Cable【4K@60Hz High Refresh Rate】uni Type-C [Thunderbolt 3] to HDMI Cable Compatible with MacBook Pro, MacBook Air, iPad Pro, Surface Book 2, Galaxy S20, Dell XPS and More - 0.9m/3ft
Title: QGeeM USB C to HDMI Cable Adapter 6ft 4K@30Hz, USB Type C to HDMI Cable Thunderbolt 3 Compatible with MacBook Pro 2018 IPad pro,Samsung S9 S10,Surface Book 2,Dell XPS 13/15,Pixelbook More
Title: USB C to Dual HDMI Adapter, Thunderbolt 3 USB Type C to 2 HDMI Monitor Adapter for Surface Book 2/Go/Pro 7,Dell XPS 13/15, Lenovo Yoga 720/900/910/920,Macbook Pro Air, iPad Pro and more
Title: USB C to HDMI Adapter 4K for MacBook Pro, HDMI Adapter Hub Dock with USB 3.0 Port, Type-C PD Charging Port Compatible w/Nintendo Switch/MacBook Air 2018/Chromebook Pixel/Dell XPS13 - Space Grey
Title: USB C Hub for iPad Pro 2018 2019 2020, Baseus 6 in 1 USB C to 4K HDMI Adapter, PD 100W Power Delivery, USB3.0, SD/TF Card Reader, 3.5mm Headphone Jack Compatible with MacBook Pro/iPad Pro - Grey
Title: USB C Hub, Aceele 10 in 1 USB Ultra Adapter with 4K HDMI, Power Delivery, 6 USB3.0 Ports, SD/TF Card Reader for MacBook Pro/Air, iPad Pro, Surface Pro 7, Samsung S20+ and More Type C Devices
Title: USB C to HDMI Adapter, Type c to HDMI 4K Adapter (Thunderbolt 3 Compatible) with Video Audio Output for MacBook Pro 2018/2017/2016, Samsung Note 9/S9/Note 8/S8, Huawei Mate 20 and More (Black)
Title: USB C to HDMI Cable for Home Office 6ft, Anker Type C to HDMI Adapter Cable 4K 60Hz for MacBook Pro 2020, iPad Pro 2020, Samsung Galaxy S20/ S10, Dell XPS 13/15, and More [Thunderbolt 3 Compatible]
Title: USB C to HDMI Adapter, Syntech Thunderbolt 3 to HDMI Adapter Compatible with MacBook Pro 2019/2018/2017, MacBook Air, iPad Pro 2020, iPad Air 2020, Samsung Galaxy S10/S9,Dell XPS 13/15 and More
Title: Apple USB-C Digital AV Multi-port Adaptor - White (Latest Model)
Title: Syntech USB C to HDMI Cable (4K@60Hz),Thunderbolt 3 to HDMI Cable Compatible with MacBook Pro 2019/2018/2017, MacBook Air, iPad Pro 2020,Samsung Galaxy S10/S9, Dell XPS and More - 6ft/1.8m
Title: USB C to HDMI Cable(4K@60Hz), Bauihr Type-C to HDMI Cable (Thunderbolt 3) Compatible for iPad Pro 2018, MacBook Pro/Air, Galaxy S10/Note 8/S9/S8, Huawei P20/Pro/P30/Pro, Mate20/30/30 pro - 6ft/1.8m
Title: USB C to HDMI Adapter 4K for MacBook Pro, HDMI Adapter Hub Dock with USB 3.0 Port, Type-C PD Charging Port Compatible w/Nintendo Switch/MacBook Air 2018/Chromebook Pixel/Dell XPS13 - Space Grey
Title: USB C to HDMI Adapter, Syntech Thunderbolt 3 to HDMI Adapter Compatible with MacBook Pro 2019/2018/2017, MacBook Air, iPad Pro 2020, iPad Air 2020, Samsung Galaxy S10/S9,Dell XPS 13/15 and More
Title: COVVY USB-C to HDMI Adapter USB C Charging Port Converter Adapter USB C Hub Compatible with Chromebook/Samsung Galaxy S8/S9, MacBook Air, iPad Pro 2020,Dell XPS 13/15 and More (Silver)
Title: USB C to HDMI Cable,CHOETECH 4K@60Hz (Thunderbolt 3 Compatible) HDMI Cable Compatible with Apple iPad Pro,2018/2017/2016 MacBookPro,iMac,MacBook,ChromeBookl,Note10/S10/Note 9/S9,Huawei P30/P20/Mate20
Title: USB C to HDMI Adapter, 3 in 1 Multiport USB Type C to 4K HDMI, USB3.0 and USB C Power Delivery Port Converter Compatible with MacBook/Chromebook Pixel/Dell XPS13/Samsung S10/S10+ and More
Title: USB Type C(Thunderbolt 3) to HDMI 4K UHD 1.8M Cable, Benfei USB 3.1 (USB-C) to HDMI Adapter Male to Male Gold-Plated Cord
Title: USB C to HDMI Cable [3m/10Ft], JSAUX 4K@60Hz Type C to HDMI Cord Thunderbolt 3 Compatible for Apple iPad Pro 2019/2018, MacBook Pro/Air, iMac, Samsung S20/S10/S9/S8 Plus/Note 10, Huawei P30/P20 - Red
Title: Belkin USB-C to HDMI Adapter + Charge (Supports 4K UHD Video, Passthrough Power up to 60W for Connected Devices) MacBook Pro HDMI Adapter
Title: USB C to HDMI Cable for Home Office 6ft, Anker Type C to HDMI Adapter Cable 4K 60Hz for MacBook Pro 2020, iPad Pro 2020, Samsung Galaxy S20/ S10, Dell XPS 13/15, and More [Thunderbolt 3 Compatible]
Title: USB C to HDMI Adapter, Thunderbolt 3 Type C to HDMI Adapter 4K for MacBook Pro 2019/2018/2017/2016, MacBook Air, Samsung Note 9/S9/Note 8/S8, Huawei P30/Mate 30
Title: QGeeM USB C to HDMI Cable Adapter 6ft 4K@30Hz, USB Type C to HDMI Cable Thunderbolt 3 Compatible with MacBook Pro 2018 IPad pro,Samsung S9 S10,Surface Book 2,Dell XPS 13/15,Pixelbook More
Title: USB C to HDMI Adapter, Aluminum Type C/Thunderbolt 3 to HDMI 4K Adapter for MacBook Pro 2017-2019, MacBook Air, Samsung Note 9/S9/Note 8/S8, Huawei Mate 20 and More (Black)
Title: USB C to HDMI Cable, uni USB Type C to HDMI Cable[Thunderbolt 3 Compatible] for Home Office, 4K, Compatible for iPad Air 4, iPad Pro 2020, MacBook Pro, Galaxy S20, XPS and More -6ft/1.8m
Title: USB C to HDMI Cable(4K@60Hz), Benfei USB Type-C to HDMI Cable [Thunderbolt 3] Compatible for MacBook Pro 2019/2018/2017, Samsung Galaxy S9/S8, Surface Book 2, Dell XPS 13/15 and More - 1.8M, Grey
Title: YEHUA USB C/Micro USB to HDMI Cable 2-in-1 Support Netflix, 6.6ft MHL to HDMI Adapter 1080P HD HDTV TikTok Mirroring for Android All Smartphone to TV/Projector/Monitor.