使用Rvest进行Web抓取--如果找不到节点，则返回NA？

Question

我有点被困在这里了。我想从一个网站抓取数据，并提取一些东西，如用户评分，评论等。我试图添加到一个数据框架的数据。

下面是我到目前为止的代码：

# Read html and select the URLs for each game review. 

library(rvest)
library(dplyr)
library(plyr)

# Read the webpage and the number of ratings.

getGame <- function(metacritic_game) {

total_ratings<- metacritic_game %>%
  html_nodes("strong") %>%
  html_text()

total_ratings <- ifelse(length(total_ratings) == 0, NA, 
as.numeric(strsplit(total_ratings, " ") [[1]][1]))

# Get the game title and the platform.

game_title <- metacritic_game %>%
  html_nodes("h1") %>%
  html_text()

game_platform <- metacritic_game %>%
  html_nodes(".platform a") %>%
  html_text()

game_platform <- strsplit(game_platform," ")[[1]][57:58]
game_platform <- gsub("\n","", game_platform)
game_platform<- paste(game_platform[1], game_platform[2], sep = " ")

game_publisher <- metacritic_game %>%
  html_nodes(".publisher a:nth-child(1)") %>%
  html_attr("href") %>%
  strsplit("/company/")%>%
  unlist() 

game_publisher <- gsub("\\W", " ", game_publisher)
game_publisher <- strsplit(game_publisher,"\\t")[[2]][1]

release_date <- metacritic_game %>%
  html_nodes(".release_data .data") %>%
  html_text()


user_ratings <- metacritic_game %>%
  html_nodes("#main .indiv") %>%
  html_text() %>%
  as.numeric()


user_name <- metacritic_game %>%
  html_nodes(".name a") %>%
  html_text()



review_date <- metacritic_game %>%
  html_nodes("#main .date") %>%
  html_text()


user_comment <- metacritic_game %>%
  html_nodes("#main .review_section .review_body") %>%
  html_text()



record_game <- data.frame(game_title = game_title,
                      game_platform = game_platform,
                      game_publisher = game_publisher,
                      username = user_name,
                      ratings =  user_ratings,
                      date = review_date,
                      comments = user_comment)

}

metacritic_home <-read_html("https://www.metacritic.com/browse/games/score/metascore/90day/all/filtered")

game_urls <- metacritic_home %>%
  html_nodes("#main .product_title a") %>%
  html_attr("href")

get100games <- function(game_urls) {
  data <- data.frame()
  i = 1
  for(i in 1:length(game_urls)) {
    metacritic_game <- read_html(paste0("https://www.metacritic.com", 
game_urls[i], "/user-reviews"))
    record_game <- getGame(metacritic_game)
    data <-rbind.fill(data, record_game)
    print(i)
  }
  data
}

df100games <- get100games(game_urls)

然而，一些链接没有任何用户评论，因此rvest无法找到节点，我得到了以下错误: Error in data.frame(game_title = game_title，game_platform = game_platform，：arguments暗示不同的行数: 1，0。

我尝试包含ifelse语句，例如：

username = ifelse(length(user_name) !=0 , user_name, NA),
                      ratings =  ifelse(length(user_ratings) != 0, 
user_ratings, NA),
                      date = ifelse(length(review_date) != 0, 
review_date, NA),
                      comments = ifelse(length(user_comment) != 0, 
user_comment, NA))

但是，数据框在每场比赛中只返回一条评论，而不是返回所有评论。对此有什么想法吗？

谢谢..！

Answer 1

您可以在purrr包中使用函数运算符possibly：

df100games <- purrr::map(game_urls, purrr::possibly(get100games, NULL)) %>%
  purrr::compact() %>% 
  dplyr::bind_rows()

我相信这将返回您想要的输出。