获取nnet的插入符号的模型参数

Question

我在为nnet提取插入符号的finalModel参数时遇到了问题。如果我使用-在我的脑海中-完全相同的参数插入：：火车和nnet：：get：：get，我得到(有时)很大的差异。我是否忘记了一个参数，或者这是由于神经网络的计算算法造成的？我知道我可以使用caret_net的predict (在下面的例子中)，但我仍然希望只使用nnet来重现结果。

示例：

library(nnet)
library(caret)

len <- 100
set.seed(4321)
X <- data.frame(x1 = rnorm(len, 40, 25), x2 = rnorm(len, 70, 4), x3 = rnorm(len, 1.6, 0.3))
y <- 20000 + X$x1 * 3 - X$x1*X$x2 * 4 - (X$x3**4) * 7 + rnorm(len, 0, 4)
XY <- cbind(X, y)

# pre-processing
preProcPrms <- preProcess(XY, method = c("center", "scale"))
XY_pre <- predict(preProcPrms, XY)

# caret-nnet
controlList <- trainControl(method = "cv", number = 5)
tuneMatrix <- expand.grid(size = c(1, 2), decay = c(0, 0.1))

caret_net <- train(x = XY_pre[ , colnames(XY_pre) != "y"],
                   y = XY_pre[ , colnames(XY_pre) == "y"],
                   method = "nnet",
                   linout = TRUE,
                   TRACE = FALSE,
                   maxit = 100,
                   tuneGrid = tuneMatrix,
                   trControl = controlList)

# nnet-nnet
nnet_net <- nnet(x = XY_pre[ , colnames(XY_pre) != "y"],
                 y = XY_pre[ , colnames(XY_pre) == "y"],
                 linout = caret_net$finalModel$param$linout,
                 TRACE = caret_net$finalModel$param$TRACE,
                 size = caret_net$bestTune$size,
                 decay = caret_net$bestTune$decay,
                 entropy = caret_net$finalModel$entropy,
                 maxit = 100)

# print
print(caret_net$finalModel)
print(nnet_net)

y_caret <- predict(caret_net$finalModel, XY_pre[ , colnames(XY_pre) != "y"])
y_nnet <- predict(nnet_net, XY_pre[ , colnames(XY_pre) != "y"])

plot(y_caret, y_nnet, main = "Hard to spot, but y_caret <> y_nnet - which prm have I forgotten?")
hist(y_caret - y_nnet)

致以友好的问候(&K)

Answer 1

正如评论中所述，差异是由不同的种子引起的。引用@Artem Sokolov:神经网络训练通常从随机状态开始。可以合理地预期插入符号：：train和nnet::nnet从两个不同的状态开始。因此，它们可能会收敛到两个不同的局部最优。

要获得可重现的模型，请从相同的种子开始：

controlList <- trainControl(method = "none", seeds = 1)
tuneMatrix <- expand.grid(size = 2, decay = 0)

set.seed(1)
caret_net <- train(x = XY_pre[ , colnames(XY_pre) != "y"],
                   y = XY_pre[ , colnames(XY_pre) == "y"],
                   method = "nnet",
                   linout = TRUE,
                   TRACE = FALSE,
                   maxit = 100,
                   tuneGrid = tuneMatrix,
                   trControl = controlList)

set.seed(1)
nnet_net <- nnet(x = XY_pre[ , colnames(XY_pre) != "y"],
                 y = XY_pre[ , colnames(XY_pre) == "y"],
                 linout = caret_net$finalModel$param$linout,
                 TRACE = caret_net$finalModel$param$TRACE,
                 size = caret_net$bestTune$size,
                 decay = caret_net$bestTune$decay,
                 entropy = caret_net$finalModel$entropy,
                 maxit = 100)

y_caret <- predict(caret_net, XY_pre[ , colnames(XY_pre) != "y"])
y_nnet <- predict(nnet_net, XY_pre[ , colnames(XY_pre) != "y"])


all.equal(as.vector(y_caret[,1]), y_nnet[,1])
#TRUE

除了设置相同的种子之外，关键是避免在插入符号中进行重新采样，因为它依赖于种子，并且在模型训练之前。