使用Keras进行深度估计

Question

我正在尝试设计一个卷积网络来使用Keras估计图像的深度。

我有形状为3x120x160的RGB输入图像和形状为1x120x160的灰度输出深度图。

我尝试使用类似VGG的架构，其中每一层的深度都会增长，但最后当我想要设计最后一层时，我被卡住了。使用密集层太昂贵了，我试着使用上采样，但效率很低。

我想使用DeConvolution2D，但是我不能让它工作。我最终得到的唯一架构是这样的：

    model = Sequential()
    model.add(Convolution2D(64, 5, 5, activation='relu', input_shape=(3, 120, 160)))
    model.add(Convolution2D(64, 5, 5, activation='relu'))
    model.add(MaxPooling2D())
    model.add(Dropout(0.5))

    model.add(Convolution2D(128, 3, 3, activation='relu'))
    model.add(Convolution2D(128, 3, 3, activation='relu'))
    model.add(MaxPooling2D())
    model.add(Dropout(0.5))

    model.add(Convolution2D(256, 3, 3, activation='relu'))
    model.add(Convolution2D(256, 3, 3, activation='relu'))
    model.add(Dropout(0.5))

    model.add(Convolution2D(512, 3, 3, activation='relu'))
    model.add(Convolution2D(512, 3, 3, activation='relu'))
    model.add(Dropout(0.5))

    model.add(ZeroPadding2D())
    model.add(Deconvolution2D(512, 3, 3, (None, 512, 41, 61), subsample=(2, 2), activation='relu'))
    model.add(Deconvolution2D(512, 3, 3, (None, 512, 123, 183), subsample=(3, 3), activation='relu'))
    model.add(cropping.Cropping2D(cropping=((1, 2), (11, 12))))
    model.add(Convolution2D(1, 1, 1, activation='sigmoid', border_mode='same'))

模型摘要如下所示：

Layer (type)                     Output Shape          Param #     Connected to                     
====================================================================================================
convolution2d_1 (Convolution2D)  (None, 64, 116, 156)  4864        convolution2d_input_1[0][0]      
____________________________________________________________________________________________________
convolution2d_2 (Convolution2D)  (None, 64, 112, 152)  102464      convolution2d_1[0][0]            
____________________________________________________________________________________________________
maxpooling2d_1 (MaxPooling2D)    (None, 64, 56, 76)    0           convolution2d_2[0][0]            
____________________________________________________________________________________________________
dropout_1 (Dropout)              (None, 64, 56, 76)    0           maxpooling2d_1[0][0]             
____________________________________________________________________________________________________
convolution2d_3 (Convolution2D)  (None, 128, 54, 74)   73856       dropout_1[0][0]                  
____________________________________________________________________________________________________
convolution2d_4 (Convolution2D)  (None, 128, 52, 72)   147584      convolution2d_3[0][0]            
____________________________________________________________________________________________________
maxpooling2d_2 (MaxPooling2D)    (None, 128, 26, 36)   0           convolution2d_4[0][0]            
____________________________________________________________________________________________________
dropout_2 (Dropout)              (None, 128, 26, 36)   0           maxpooling2d_2[0][0]             
____________________________________________________________________________________________________
convolution2d_5 (Convolution2D)  (None, 256, 24, 34)   295168      dropout_2[0][0]                  
____________________________________________________________________________________________________
convolution2d_6 (Convolution2D)  (None, 256, 22, 32)   590080      convolution2d_5[0][0]            
____________________________________________________________________________________________________
dropout_3 (Dropout)              (None, 256, 22, 32)   0           convolution2d_6[0][0]            
____________________________________________________________________________________________________
convolution2d_7 (Convolution2D)  (None, 512, 20, 30)   1180160     dropout_3[0][0]                  
____________________________________________________________________________________________________
convolution2d_8 (Convolution2D)  (None, 512, 18, 28)   2359808     convolution2d_7[0][0]            
____________________________________________________________________________________________________
dropout_4 (Dropout)              (None, 512, 18, 28)   0           convolution2d_8[0][0]            
____________________________________________________________________________________________________
zeropadding2d_1 (ZeroPadding2D)  (None, 512, 20, 30)   0           dropout_4[0][0]                  
____________________________________________________________________________________________________
deconvolution2d_1 (Deconvolution2(None, 512, 41, 61)   2359808     zeropadding2d_1[0][0]            
____________________________________________________________________________________________________
deconvolution2d_2 (Deconvolution2(None, 512, 123, 183) 2359808     deconvolution2d_1[0][0]          
____________________________________________________________________________________________________
cropping2d_1 (Cropping2D)        (None, 512, 120, 160) 0           deconvolution2d_2[0][0]          
____________________________________________________________________________________________________
convolution2d_9 (Convolution2D)  (None, 1, 120, 160)   513         cropping2d_1[0][0]               
====================================================================================================
Total params: 9474113

我不能从512减少Deconvolution2D层的大小，因为这样做会导致形状相关的错误，而且似乎我必须添加与前一层中的滤镜数量一样多的Deconvolution2D层。我还必须添加最后一个Convolution2D层才能运行网络。

上面的架构学习起来很慢，而且(我认为)效率很低。我确信我做错了什么，设计不应该是这样的。你能帮我设计一个更好的网络吗？

我也试着像this repository中提到的那样做一个网络，但似乎Keras不像这个千层面的例子那样工作。如果有人能告诉我如何在Keras中设计这样的网络，我会非常感激的。它的架构是这样的：

​

​

谢谢

Answer 1

我建议使用U-Net (参见图1)。在U-Net的前半部分，空间分辨率会随着通道数量的增加而降低(如您所提到的VGG )。在下半场，相反的情况发生了，(通道数量减少，分辨率增加)。不同层之间的“跳过”连接允许网络有效地产生高分辨率输出。

你应该能够找到一个合适的Keras实现(也许是this one)。