caffe："Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0) CUDNN_STATUS_BAD_PARAM“

Question

我开始使用caffe进行网络编程，由于我习惯了更舒适和“懒惰”的解决方案，所以我对可能发生的问题感到有点不知所措。

现在我得到的是错误Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0) CUDNN_STATUS_BAD_PARAM

这是众所周知的生产不好的库达或cudnn版本。所以我查过了它们都是最新的。(库达: 8.0.61库丁: 6.0.21)

因为我只有在添加这个ReLU层时才会得到这个错误，所以我认为它是由我混淆了一个参数引起的：

layer{
name: "relu1"
type: "ReLU"
bottom: "pool1"
top: "relu1"
}

为了给您提供所有的信息，下面是我得到的错误消息：

I0319 09:41:09.484148  6909 solver.cpp:44] Initializing solver from parameters:
test_iter: 10
test_interval: 1000
base_lr: 0.001
display: 20
max_iter: 800
lr_policy: "step"
gamma: 0.1
momentum: 0.9
weight_decay: 0.04
stepsize: 200
snapshot: 10000
snapshot_prefix: "models/train"
solver_mode: GPU
net: "train_val.prototxt"
I0319 09:41:09.484392  6909 solver.cpp:87] Creating training net from net file: train_val.prototxt
I0319 09:41:09.485164  6909 net.cpp:294] The NetState phase (0) differed from the phase (1) specified by a rule in layer feed2
I0319 09:41:09.485183  6909 net.cpp:51] Initializing net from parameters:
name: "CaffeNet"
state {
  phase: TRAIN
}
layer {
  name: "feed"
  type: "HDF5Data"
  top: "data"
  top: "label"
  include {
    phase: TRAIN
  }
  hdf5_data_param {
    source: "train_h5_list.txt"
    batch_size: 50
  }
}
layer {
  name: "conv1"
  type: "Convolution"
  bottom: "data"
  top: "conv1"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  convolution_param {
    num_output: 1
    kernel_size: 3
    stride: 1
    weight_filler {
      type: "gaussian"
    }
    bias_filler {
      type: "constant"
    }
  }
}
layer {
  name: "pool1"
  type: "Pooling"
  bottom: "conv1"
  top: "pool1"
  pooling_param {
    pool: MAX
    kernel_size: 2
    stride: 1
  }
}
layer {
  name: "relu1"
  type: "ReLU"
  bottom: "pool1"
  top: "relu1"
}
layer {
  name: "conv2"
  type: "Convolution"
  bottom: "relu1"
  top: "conv2"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  convolution_param {
    num_output: 1
    kernel_size: 3
    stride: 1
    weight_filler {
      type: "gaussian"
    }
    bias_filler {
      type: "constant"
    }
  }
}
layer {
  name: "ip2"
  type: "InnerProduct"
  bottom: "conv2"
  top: "ip2"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  inner_product_param {
    num_output: 1
    weight_filler {
      type: "gaussian"
      std: 0.01
    }
    bias_filler {
      type: "constant"
      value: 0
    }
  }
}
layer {
  name: "sig1"
  type: "Sigmoid"
  bottom: "ip2"
  top: "sig1"
}
layer {
  name: "loss"
  type: "EuclideanLoss"
  bottom: "sig1"
  bottom: "label"
  top: "loss"
}
I0319 09:41:09.485752  6909 layer_factory.hpp:77] Creating layer feed
I0319 09:41:09.485780  6909 net.cpp:84] Creating Layer feed
I0319 09:41:09.485792  6909 net.cpp:380] feed -> data
I0319 09:41:09.485819  6909 net.cpp:380] feed -> label
I0319 09:41:09.485836  6909 hdf5_data_layer.cpp:80] Loading list of HDF5 filenames from: train_h5_list.txt
I0319 09:41:09.485860  6909 hdf5_data_layer.cpp:94] Number of HDF5 files: 1
I0319 09:41:09.486469  6909 hdf5.cpp:32] Datatype class: H5T_FLOAT
I0319 09:41:09.500986  6909 net.cpp:122] Setting up feed
I0319 09:41:09.501011  6909 net.cpp:129] Top shape: 50 227 227 3 (7729350)
I0319 09:41:09.501027  6909 net.cpp:129] Top shape: 50 1 (50)
I0319 09:41:09.501039  6909 net.cpp:137] Memory required for data: 30917600
I0319 09:41:09.501051  6909 layer_factory.hpp:77] Creating layer conv1
I0319 09:41:09.501080  6909 net.cpp:84] Creating Layer conv1
I0319 09:41:09.501087  6909 net.cpp:406] conv1 <- data
I0319 09:41:09.501101  6909 net.cpp:380] conv1 -> conv1
I0319 09:41:09.880740  6909 net.cpp:122] Setting up conv1
I0319 09:41:09.880765  6909 net.cpp:129] Top shape: 50 1 225 1 (11250)
I0319 09:41:09.880781  6909 net.cpp:137] Memory required for data: 30962600
I0319 09:41:09.880808  6909 layer_factory.hpp:77] Creating layer pool1
I0319 09:41:09.880836  6909 net.cpp:84] Creating Layer pool1
I0319 09:41:09.880846  6909 net.cpp:406] pool1 <- conv1
I0319 09:41:09.880861  6909 net.cpp:380] pool1 -> pool1
I0319 09:41:09.880888  6909 net.cpp:122] Setting up pool1
I0319 09:41:09.880899  6909 net.cpp:129] Top shape: 50 1 224 0 (0)
I0319 09:41:09.880913  6909 net.cpp:137] Memory required for data: 30962600
I0319 09:41:09.880921  6909 layer_factory.hpp:77] Creating layer relu1
I0319 09:41:09.880934  6909 net.cpp:84] Creating Layer relu1
I0319 09:41:09.880941  6909 net.cpp:406] relu1 <- pool1
I0319 09:41:09.880952  6909 net.cpp:380] relu1 -> relu1
F0319 09:41:09.881192  6909 cudnn.hpp:80] Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0)  CUDNN_STATUS_BAD_PARAM

编辑:尝试将解决模式设置为CPU，我仍然得到这个错误。

Answer 1

它抛出此错误的原因是因为您没有更多的空间“缩小”。从您的错误消息: 50 1224 0 (0)中，这表示网络的大小在一维中为0。

要修正此错误，您可以调整一些参数，包括(S)tride、(K)ernel大小和(P)相加。要计算下一层(W_new)的尺寸，可以使用以下公式：

W_new = (W_old -K+ 2*P)/S +1

因此，如果我们有一个输入，即227x227x3，我们的第一层有K= 5，S= 2，P= 1，numOutputs = N，那么conv1有一个维数，即：

(227-5+2*1)/2 +1= 112x112xN。

注意:如果你在分子中有一个奇数，在加1后再加起来。

编辑:它出现在ReLU层的原因很可能是因为ReLU层没有什么可通过的，因此它会抛出一个错误。

Answer 2

我发现了其中一个问题。

I0319 09:41:09.880765  6909 net.cpp:129] Top shape: 50 1 225 1 (11250)
I0319 09:41:09.880781  6909 net.cpp:137] Memory required for data: 30962600
I0319 09:41:09.880808  6909 layer_factory.hpp:77] Creating layer pool1
I0319 09:41:09.880836  6909 net.cpp:84] Creating Layer pool1
I0319 09:41:09.880846  6909 net.cpp:406] pool1 <- conv1
I0319 09:41:09.880861  6909 net.cpp:380] pool1 -> pool1
I0319 09:41:09.880888  6909 net.cpp:122] Setting up pool1
I0319 09:41:09.880899  6909 net.cpp:129] Top shape: 50 1 224 0 (0)

正如你所看到的，第一个卷积层将接受一个大小的输入(50 227 227 3)，因为他认为第二维度包含通道，所以有一点问题。

这是唯一的自然，这一卷积层将简单地屠宰尺寸的方式，现在没有进一步的层将得到适当的输入维数。

我成功地解决了这个问题，只需通过这样的方式重新调整输入：

layer {
    name: "reshape"
    type: "Reshape"
    bottom: "data"
    top: "res"
    reshape_param {
      shape {
        dim: 50
        dim: 3
        dim: 227
        dim: 227
      }
    }
  }

其中的第一个维度是批处理大小，因此无论是谁读它，都必须记住在分类阶段的.prototxt文件中将这个dim设置为1(因为它不能用于批处理)

编辑:我将把它标记为一个答案，因为它涵盖了我所拥有的问题的基本解决方案，并且看不到其他解决方案。如果有人想在这件事上有更多的希望，请这样做。