从h2o模型中使用iml克服兼容性问题

Question

我无法像这里( h2o )所详细描述的那样，重现在iml (https://www.r-bloggers.com/2018/08/iml-and-h2o-machine-learning-model-interpretability-and-feature-explanation/)中使用iml (用FeatureImp$new和H2O提取变量重要性时的错误)的唯一示例。有人能指出与h2o一起使用iml的解决方案或其他示例吗？

可复制的例子：

library(rsample)   # data splitting
library(ggplot2)   # allows extension of visualizations
library(dplyr)     # basic data transformation
library(h2o)       # machine learning modeling
library(iml)       # ML interprtation
library(modeldata) #attrition data 


# initialize h2o session
h2o.no_progress()
h2o.init()

# classification data
data("attrition", package = "modeldata")
df <- rsample::attrition %>% 
  mutate_if(is.ordered, factor, ordered = FALSE) %>%
  mutate(Attrition = recode(Attrition, "Yes" = "1", "No" = "0") %>% factor(levels = c("1", "0")))

# convert to h2o object
df.h2o <- as.h2o(df)

# create train, validation, and test splits
set.seed(123)
splits <- h2o.splitFrame(df.h2o, ratios = c(.7, .15), destination_frames = 
    c("train","valid","test"))
names(splits) <- c("train","valid","test")

# variable names for resonse & features
y <- "Attrition"
x <- setdiff(names(df), y) 

# elastic net model 
glm <- h2o.glm(
  x = x, 
  y = y, 
  training_frame = splits$train,
  validation_frame = splits$valid,
  family = "binomial",
  seed = 123
  )

# 1. create a data frame with just the features
features <- as.data.frame(splits$valid) %>% select(-Attrition)

# 2. Create a vector with the actual responses
response <- as.numeric(as.vector(splits$valid$Attrition))

# 3. Create custom predict function that returns the predicted values as a
#    vector (probability of purchasing in our example)
pred <- function(model, newdata)  {
  results <- as.data.frame(h2o.predict(model, as.h2o(newdata)))
  return(results[[3L]])
}

# create predictor object to pass to explainer functions
predictor.glm <- Predictor$new(
  model = glm, 
  data = features, 
  y = response, 
  predict.fun = pred,
  class = "classification"
  )

imp.glm <- FeatureImp$new(predictor.glm, loss = "mse")

获得的错误：

Error in `[.data.frame`(prediction, , self$class, drop = FALSE): undefined columns 
selected

traceback()

1. FeatureImp$new(predictor.glm, loss = "mse")

2. .subset2(public_bind_env, "initialize")(...)

3. private$run.prediction(private$sampler$X)

4. self$predictor$predict(data.frame(dataDesign))

5. prediction[, self$class, drop = FALSE]

6. `[.data.frame`(prediction, , self$class, drop = FALSE)

7. stop("undefined columns selected")

Answer 1

在iml包文档中，它说class参数是“要返回的类列”。当您设置class = "classification"时，它将查找一个名为“分类法”的列，这个列是找不到的。至少在GitHub上，自那篇博客文章以来，iml包似乎经历了相当多的开发，所以我想有些功能可能不再向后兼容了。

在阅读了包文档之后，我认为您可能需要尝试这样的方法：

predictor.glm <- Predictor$new(
  model = glm, 
  data = features, 
  y = "Attrition",
  predict.function = pred,
  type = "prob"
  )

# check ability to predict first
check <- predictor.glm$predict(features)
print(check)

更好的办法可能是利用H2O的广泛功能，围绕机器学习可解释性。

h2o.varimp(glm)将为每个特性赋予用户变量的重要性。

h2o.varimp_plot(glm, 10)将呈现一个图形，显示每个特性的相对重要性。

h2o.explain(glm, as.h2o(features))是可解释接口的包装器，默认情况下将提供混淆矩阵(在本例中)以及变量重要性，以及每个特性的部分依赖图。

对于某些算法(例如，基于树的方法)，h2o.shap_explain_row_plot()和h2o.shap_summary_plot()将提供shap贡献。

在这里，h2o-3文档可能对探索更多信息很有用。