rvest返回NA

Question

我正在使用"rvest“进行网页抓取，但我无法从页面提取模型的价格:- https://www.motorola.com/us/products/moto-z-force-droid-edition。我需要从页面中提取"$720.00“。我的代码是：

library(rvest)

data<-read_html("https://www.motorola.com/us/products/moto-z-force-droid-edition")

price<-data%>%
html_nodes(".price-amount")%>%
html_text()
print(price)

我不断地从价格中得到字符(0)。

请帮帮忙。

Answer 1

url<-"https://www.motorola.com/us/js/motorola_blc_catalog/price/1281"
page<-html_session(url)
text<-XML::xmlToList(XML::xmlParse(httr::content(page$response)$price))
price<-text$tbody$tr$td$span$text

Answer 2

将rvest与RSelenium和seleniumPipes一起使用。确保下载phantomJS并将.exe文件放在当前R工作目录中。详情请参见my other answser。

library(tidyverse)
library(rvest)
library(RSelenium) # start a server with utility function
library(seleniumPipes)
#start server
rD <- rsDriver (browser = 'chrome',chromever = "latest",port = 4445L)
#open browser
remDr <- remoteDr(browserName = "chrome",port = 4445L)
web_url <- "https://www.motorola.com/us/products/moto-z-force-droid-edition"

remDr %>% go(web_url)
price <- remDr %>% getPageSource() %>% html_nodes(".price-amount") %>% html_text()

print(price)    
[1] "$30.00/mo" "$720.00"   "$720.00" 

remDr %>% deleteSession() # delete session
rD[["server"]]$stop() # close server

简而言之，您需要使用无头浏览器来呈现javascript生成的值。在此之后，您可以照常使用rvest。