如何使用ChromeDriver Chrome和Selenium在页面上通过Python打印类别链接？

Question

使用Python3，我试图获得，以识别网页www.jtinsight.com上的各种“分类”类别，并从中打印出分类标题。到目前为止，使用下面的代码，我能做的是让它打印出前两种-“所有类别”和“汽车(私人)”。我已经确定了这两种代码的html是不同的，并且尝试了我在代码注释中列出的一些不同的代码行，但是无法识别正确的标记/class/xpath等等。如果有任何帮助，我们将不胜感激。

from selenium import webdriver
from selenium.webdriver.common.by import By

# Creating the WebDriver object using the ChromeDriver
driver = webdriver.Chrome()

# Directing the driver to the defined url
driver.get("https://www.jtinsight.com/JTIRA/JTIRA.aspx#!/main")

# Locate the categories

# Each code line runs but only returns the first two categories
# categories = driver.find_elements_by_xpath('//div[@class="col-md-3 col-sm-4 col-xs-6"]')
# categories = driver.find_elements_by_xpath('//div[@class="mainCatEntry"]')
# categories = driver.find_elements_by_xpath('//div[@class="Description"]')

# Process ran but finished with exit code 0
# categories = driver.find_elements_by_xpath('//*[@class="col-md-3 col-sm-4 col-xs-6 ng-scope"]')
# categories = driver.find_elements_by_xpath('//div[@class="col-md-3 col-sm-4 col-xs-6 ng-scope"]')
# categories = driver.find_elements_by_partial_link_text('//href[@class="divLink"]')
# categories = driver.find_elements_by_tag_name('bindonce')
# categories = driver.find_elements_by_xpath('//div[@class="divLink"]')

# Error before finished running
# categories = driver.find_elements(By.CLASS_NAME, "col-md-3 col-sm-4 col-xs-6 ng-scope")
# categories = driver.find_elements(By.XPATH, '//div bindonce[@class="col-md-3 col-sm-4 col-xs-6 ng-scope"]')
# categories = driver.find_elements_by_class_name('//div bindonce[@class="col-md-3 col-sm-4 col-xs-6 ng-scope"]')

# Print out all categories on current page
num_page_items = len(categories)
print(num_page_items)
for i in range(num_page_items):
    print(categories[i].text)

# Clean up (close browser once task is completed.)
driver.close()

Answer 1

这确实是一个时间问题。如果我在收集类别之前加了一个“睡眠(5)”，它就会找到所有的24个。有趣的是，当我使用WebDriverWait时，它仍然只会拉出两个项目。因此，为了迫使驱动程序做更多的工作，我扩展了xpath。以下几点对我有用：

categories = WebDriverWait(driver, 10).until(
EC.visibility_of_all_elements_located((By.XPATH, '//div[@class="mainCatEntry"]/div[@class="Description"]')))

Answer 2

要在网页https://www.jtinsight.com/JTIRA/JTIRA.aspx#!/main上识别各种分类类别并打印类别标题，例如All类别、Cars (私有)等，您需要向下滚动并为visibility_of_all_elements_located()导出WebDriverWait，您可以使用以下解决方案：

• 代码块： 从selenium导入from驱动程序从driver=webdriver.Chrome(chrome_options=options，executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')导入从selenium.webdriver.support.ui导入WebDriverWait从selenium.webdriver.support导入selenium.webdriver.support选项= webdriver.ChromeOptions() options.add_argument(“启动-最大化”)options.add_argument(‘disable’)selenium.webdriver.support.ui driver.get(“https://www.jtinsight.com/JTIRA/JTIRA.aspx#!/main"”)( "，30).until(EC.visibility_of_element_located((By.XPATH，"//span@class='ng-scope‘和text()=’分类‘))；打印([elem.get_attribute(“innerHTML”)表示WebDriverWait中的elem (driver，30).until(EC.visibility_of_all_elements_located((By.XPATH，"//div@class='mainCatEntry'//div@class='Description'")))]) )