具有多个子标头的HTML表的刮除

Question

我试图使用以下代码在data.frame中导入核试验站点列表(从维基百科的页面)：

library(RCurl)
library(XML)

theurl <- "https://en.wikipedia.org/wiki/List_of_nuclear_test_sites"
webpage <- getURL(theurl)
webpage <- readLines(tc <- textConnection(webpage)); close(tc)

pagetree <- htmlTreeParse(webpage, error=function(...){}, useInternalNodes = TRUE)

# Find XPath (go the webpage, right-click inspect element, find table then right-click copyXPath) 
myxpath <- "//*[@id='mw-content-text']/table[2]"

# Extract table header and contents
tablehead <- xpathSApply(pagetree, paste(myxpath,"/tr/th",sep=""), xmlValue)
results <- xpathSApply(pagetree, paste(myxpath,"/tr/td",sep=""), xmlValue)

# Convert character vector to dataframe
content <- as.data.frame(matrix(results, ncol = 5, byrow = TRUE))
names(content) <- c("Testing country", "Location", "Site", "Coordinates", "Notes")

然而，有多个子标头阻止data.frame一致填充。我怎么才能解决这个问题？

Answer 1

请看一下htmltab包。它允许您使用子标题填充新列：

library(htmltab)
tab <- htmltab("https://en.wikipedia.org/wiki/List_of_nuclear_test_sites",
           which = "/html/body/div[3]/div[3]/div[4]/table[2]",
           header = 1 + "//tr/th[@style='background:#efefff;']",
           rm_nodata_cols = F)

Answer 2

我发现Carson的这个例子对我很有帮助：

library(rvest)
theurl <- "https://en.wikipedia.org/wiki/List_of_nuclear_test_sites"
# First, grab the page source
content <- html(theurl) %>%
  # then extract the first node with class of wikitable
  html_node(".wikitable") %>% 
  # then convert the HTML table into a data frame
  html_table()

Answer 3

你试过这个吗？

l.wiki.url <- getURL( url = "https://en.wikipedia.org/wiki/List_of_nuclear_test_sites" )
l.wiki.par <- htmlParse( file = l.wiki.url )

l.tab.con <- xpathSApply( doc  = l.wiki.par
                        , path = "//table[@class='wikitable']//tr//td"
                        , fun  = xmlValue
                        )