Pytorch CNN上的Optuna

Question

class ConvolutionalNetwork(nn.Module):
    def __init__(self, in_features, trial):
        super().__init__()
        self.in_features = in_features
        self.trial = trial
        # this computes no of features outputted by 2 conv layers
        c1 = int(((self.in_features - 2)) / 64)  # this is to account for the loss due to conversion to int type
        c2 = int((c1 - 2) / 64)
        self.n_conv = int(c2 * 16)
        # self.n_conv = int((( ( (self.in_features - 2)/4 ) - 2 )/4 ) * 16)
        num_filters1 = trial.suggest_int("num_filters1",16,64,step=16)
        num_filters2 = trial.suggest_int("num_filters2",16,64,step=16)
        #num_filters = 16        
        kernel_size = trial.suggest_int('kernel_size', 2, 7)
        self.conv1 = nn.Conv1d(1, num_filters1, kernel_size, 1)
        self.conv1_bn = nn.BatchNorm1d(num_filters1)
        self.conv2 = nn.Conv1d(num_filters1, num_filters2, kernel_size, 1)
        self.conv2_bn = nn.BatchNorm1d(num_filters2)
        #Add in trial range for dropout to determine optimal dropout value
        self.dp = nn.Dropout(trial.suggest_uniform('dropout_rate',0,1.0))
        self.fc3 = nn.Linear(self.n_conv, 2)

我尝试添加Optuna参数调整试验中的过滤器数量1和2，如下所示，但发生了以下错误。试验0设法通过了，但试验1没有。我对批处理大小进行了调整，但不同的推荐批处理大小出现了相同的错误格式。

num_filters1 = trial.suggest_int("num_filters1",16,64,step=16)
            num_filters2 = trial.suggest_int("num_filters2",16,64,step=16)

[I 2020-12-07 09:22:53,512] Trial 0 finished with value: 0.6597743630409241 and parameters: {'num_filters1': 16, 'num_filters2': 16, 'kernel_size': 7, 'dropout_rate': 0.5225509182876455, 'optimizer': 'SGD', 'lr': 0.00020958259416674875, 'Weight_decay': 3.8111213220887506e-08}. Best is trial 0 with value: 0.6597743630409241.
[W 2020-12-07 09:22:55,502] Trial 1 failed because of the following error: ValueError('Expected input batch_size (8000) to match target batch_size (2000).')

Answer 1

删除了num_filter2的Optuna自动调整，因为在最后一层滤镜之后会有更多的链条效应。

保留了用于Optuna自动调整的num_filter1，模型能够平稳运行。

谢谢!