data.table没有按组返回正确的样条乐趣

Question

我们最近将data.table从1.12.0版本更新为1.12.8，R从3.5.3更新到3.6.3。该示例在Windows操作系统上。

我们有一个data.table，在其中循环一个类别列并创建一个splinefun对象供以后使用。我们将这个splinefun函数输出存储到list中，在data.table列中。它在我们的旧规范上发挥了预期的作用，根据分段数据为每个类别级别生成了一个唯一的splinefun。但是，现在它似乎只保留最后类别的值，并将其解析到所有条目中。

设置数据

创建一些显示问题的假数据

# R version: 3.6.3 (2020-02-29)
library(data.table) # data.table_1.12.8
library(ggplot2)
library(stats) 

# mimic our data in simpler format
set.seed(1)
dt <- data.table(cat = rep(letters[1:3], each = 10),
                 x = 1:10)
dt[, y := x^0.5 * rnorm(.N, mean=runif(1, 1, 100), sd=runif(1, 1, 10)), by=cat]

# can see that each line is different
pl0 <- ggplot(data=dt, aes(x=x, y=y, col=cat)) + geom_line()
pl0

拟合样条

通过我们目前的方法拟合样条，并使用lapply进行比较。lapply按预期工作，data.table不工作。

# fit spline, segment the data by category
mod_splines <- dt[, .(Spline = list(splinefun(x=x, y=y, method = "natural"))),
                  by = c("cat")]

# splinefun works such that you provide new values of x and it gives an output
# y from a spline fitted to y~x
# Can see they are all the same, which seems unlikely
mod_splines$Spline[[1]](5)
mod_splines$Spline[[2]](5)
mod_splines$Spline[[3]](5)

# alternative approach
alt_splines <-  lapply(unique(dt$cat), function(x_cat){
  splinefun(x=dt[cat==x_cat, ]$x, 
            y=dt[cat==x_cat, ]$y, 
            method = "natural")
})

# looks more realistic
alt_splines[[1]](5)
alt_splines[[2]](5)
alt_splines[[3]](5) # Matches the mod_splines one!

splinefun 检查是否适合

当我们从splinefun循环中打印出来时，data.table的数据和输出看起来是正确的，但是它没有被正确地存储。

# check the data is segmenting
mod_splines2 <- dt[, .(Spline = list(splinefun(x=x, y=y, method = "natural")),
                      x=x, y=y),
                  by = c("cat")]
mod_splines2[] # the data is definitely segmenting ok

# try catching and printing the data
splinefun_withmorefun <- function(x, y){

  writeLines(paste(x, collapse =", "))
  writeLines(paste(round(y, 0), collapse =", "))

  foo <- splinefun(x=x, 
            y=y, 
            method = "natural")
  writeLines(paste(foo(5), collapse =", "))
  writeLines("")
  return(foo)
}

# looks like its in the function ok, as it prints out different results 
mod_splines3 <- dt[, .(Spline = list(splinefun_withmorefun(x=x, y=y))),
                   by = c("cat")]

# but not coming through in to the listed function
mod_splines3$Spline[[1]](5)
mod_splines3$Spline[[2]](5)
mod_splines3$Spline[[3]](5)

任何想法，为什么这将是一个问题后，更新将是伟大的！我们担心，使用类似的data.table方法的其他情况现在可能会像这一次一样悄无声息地中断。

谢谢你，琼尼

Answer 1

正如我在https://github.com/Rdatatable/data.table/issues/4298#issuecomment-597737776中回答的那样，在x和y变量上添加copy()将解决这个问题。

原因是splinefun()会尝试存储x和y的值。但是，data.table的内部对象总是通过引用(为了速度).在这种情况下，为了得到预期的答案，您可能必须显式地copy()变量。

总之，不断变化

mod_splines <- dt[, .(Spline = list(splinefun(x=x, y=y, method = "natural"))),
                  by = c("cat")]

至

mod_splines <- dt[, .(Spline = list(splinefun(x=copy(x), y=copy(y), method = "natural"))),
                  by = c("cat")]

或者这个(你可以忽略这一点，但它可能给你一个更好的理解)

mod_splines <- dt[, .(Spline = list(splinefun(x=x+0, y=y+0, method = "natural"))),
                  by = cat]

就够了。