使用skorch & sklearn管道的多输出回归由于dtype而导致运行时错误。

Question

我想用skorch做多输出回归。我创建了一个小玩具例子，如下所示。在这个例子中，神经网络应该预测5个输出。我还想使用一个预处理步骤，该步骤使用sklearn管道(在本例中使用PCA，但它可以是任何其他预处理程序)。在执行此示例时，在torch的Variable._execution_engine.run_backward步骤中得到以下错误：

RuntimeError: Found dtype Double but expected Float

我是不是忘了什么？我怀疑，在某个地方必须要投些什么东西，但是当skorch处理了很多火把的东西时，我不知道是什么，什么地方。

示例：

import torch
import skorch
from sklearn.datasets import make_classification, make_regression
from sklearn.pipeline import Pipeline, make_pipeline
from sklearn.decomposition import PCA

X, y = make_regression(n_samples=1000, n_features=40, n_targets=5)
X = X.astype('float32')

class RegressionModule(torch.nn.Module):
    def __init__(self, input_dim=80):
        super().__init__()
        self.l0 = torch.nn.Linear(input_dim, 10)
        self.l1 = torch.nn.Linear(10, 5)

    def forward(self, X):
        y = self.l0(X)
        y = self.l1(y)
        return y


class InputShapeSetter(skorch.callbacks.Callback):
    def on_train_begin(self, net, X, y):
        net.set_params(module__input_dim=X.shape[-1])


net = skorch.NeuralNetRegressor(
    RegressionModule,
    callbacks=[InputShapeSetter()],
)

pipe = make_pipeline(PCA(n_components=10), net)

pipe.fit(X, y)
print(pipe.predict(X))

编辑1:

从本例中可以看出，从一开始就将X转换为float32并不适用于每个预处理器：

import torch
import skorch
from sklearn.datasets import make_classification, make_regression
from sklearn.pipeline import Pipeline
from sklearn.decomposition import PCA
from category_encoders import OneHotEncoder

X, y = make_regression(n_samples=1000, n_features=40, n_targets=5)
X = pd.DataFrame(X,columns=[f'feature_{i}' for i in range(X.shape[1])])
X['feature_1'] = pd.qcut(X['feature_1'], 3, labels=["good", "medium", "bad"])


y = y.astype('float32')

class RegressionModule(torch.nn.Module):
    def __init__(self, input_dim=80):
        super().__init__()
        self.l0 = torch.nn.Linear(input_dim, 10)
        self.l1 = torch.nn.Linear(10, 5)

    def forward(self, X):
        y = self.l0(X)
        y = self.l1(y)
        return y


class InputShapeSetter(skorch.callbacks.Callback):
    def on_train_begin(self, net, X, y):
        net.set_params(module__input_dim=X.shape[-1])


net = skorch.NeuralNetRegressor(
    RegressionModule,
    callbacks=[InputShapeSetter()],
)

pipe = make_pipeline(OneHotEncoder(cols=['feature_1'], return_df=False), net)

pipe.fit(X, y)
print(pipe.predict(X))

Answer 1

默认情况下，OneHotEncoder返回dtype=float64的numpy数组。因此，当输入数据被输入到模型的X中时，可以简单地转换输入数据forward()：

class RegressionModule(torch.nn.Module):
    def __init__(self, input_dim=80):
        super().__init__()
        self.l0 = torch.nn.Linear(input_dim, 10)
        self.l1 = torch.nn.Linear(10, 5)

    def forward(self, X):
        X = X.to(torch.float32)
        y = self.l0(X)
        y = self.l1(y)
        return y