用RSelenium抓取Fangraphs中的动态数据

Question

我正在尝试自动从fangraphs下载动态数据。目标是模拟点击下载csv的"Export Data“链接。或者，如果可能的话，简单地抓取数据。无论哪种获取数据的方法，我都可以接受。我已经在RSelenium中尝试了以下代码，但都没有用：

remDr <- remoteDriver(port = 4445L,
            browserName = 'firefox')
remDr$open()
remDr$navigate("https://www.fangraphs.com/leaders/splits-leaderboards?splitArr=1,7&splitArrPitch=&position=B&autoPt=false&splitTeams=false&statType=player&statgroup=1&startDate=2017-03-01&endDate=2021-11-01&players=&filter=&groupBy=career&sort=-1,1")
remDr$findElement(using = "class", value = "data-export")

此URL的数据是使用javascript呈现的。尽管您可以检查页面上的元素并看到class = "data-export“元素，但remDr$findElement返回一个错误：

Selenium message:Unable to locate element: {"method":"class name","selector":"data-export"}
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Build info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'
System info: host: '7d65bd0674cb', ip: '172.17.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-80-generic', java.version: '1.8.0_91'
Driver info: driver.version: unknown

Error:   Summary: NoSuchElement
     Detail: An element could not be located on the page using the given search parameters.
     class: org.openqa.selenium.NoSuchElementException
     Further Details: run errorDetails method

Answer 1

尝试xpath，而不是class

library(RSelenium)
driver = rsDriver(port = 4941L, browser = c("firefox"))
remDr <- driver[["client"]]
remDr$navigate("https://www.fangraphs.com/leaders/splits-leaderboards?splitArr=1,7&splitArrPitch=&position=B&autoPt=false&splitTeams=false&statType=player&statgroup=1&startDate=2017-03-01&endDate=2021-11-01&players=&filter=&groupBy=career&sort=-1,1")

#Click Exporrt Data 

remDr$findElement(using = "xpath",'//*[@id="react-drop-test"]/div[2]/a') -> downloaddata
downloaddata$clickElement()