带有predict_type = "prob“的mlr_measures_classif.costs

Question

成本敏感度量值mlr_measures_classif.costs需要'response'预测类型。

msr("classif.costs")
#<MeasureClassifCosts:classif.costs>
#* Packages: -
#* Range: [-Inf, Inf]
#* Minimize: TRUE
#* Properties: requires_task
#* Predict type: response

即使当学习者的predict_type设置为'prob'时，此措施似乎也有效

# get a cost sensitive task
task = tsk("german_credit")

# cost matrix as given on the UCI page of the german credit data set
# https://archive.ics.uci.edu/ml/datasets/statlog+(german+credit+data)
costs = matrix(c(0, 5, 1, 0), nrow = 2)
dimnames(costs) = list(truth = task$class_names, predicted = task$class_names)
print(costs)

# mlr3 needs truth in columns, predictions in rows
costs = t(costs)

# create measure which calculates the absolute costs
m = msr("classif.costs", id = "german_credit_costs", costs = costs, normalize = FALSE)

# fit models and calculate costs
learner = lrn("classif.rpart", predict_type = "prob")
rr = resample(task, learner, rsmp("cv", folds = 3))
rr$aggregate(m)

#german_credit_costs 
#               341

为什么它与predict_type一起工作被设置为'prob'？这是一个bug，还是度量内部将概率转换为类？我猜预测一个类为正或负的阈值在内部设置为0.5？可以更改这个阈值吗？

Answer 1

msr("classif.costs")使用混淆矩阵进行计算：https://github.com/mlr-org/mlr3/blob/master/R/MeasureClassifCosts.R

当predict_type设置为prob时，将生成阈值为0.5的混淆矩阵。要在具有重采样对象后对其进行更改，请执行以下操作：

pred = rr$predictions()
lapply(pred, function(x) x$set_threshold(0.1)) #arbitrary threshold

rr$aggregate(m)

要将其改回，请执行以下操作：

lapply(pred, function(x) x$set_threshold(0.5))
rr$aggregate(m)

R6活动绑定的“美感”。