美国有线电视新闻网( CNN )在角播节目中的对比

Question

我试图融合两个CNN，但是当我使用级联时，我会收到以下错误：

Traceback (most recent call last):
  File "vggFace_MM.py", line 57, in <module>
    fuse_layer = concatenate([stream_1, stream_2])
  File "/usr/local/lib/python2.7/dist-packages/keras/layers/merge.py", line 508, in concatenate
    return Concatenate(axis=axis, **kwargs)(inputs)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 596, in __call__
    output = self.call(inputs, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keras/layers/merge.py", line 283, in call
    return K.concatenate(inputs, axis=self.axis)
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py", line 1723, in concatenate
    return tf.concat([to_dense(x) for x in tensors], axis)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_ops.py", line 1075, in concat
    dtype=dtypes.int32).get_shape(
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 669, in convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 176, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 165, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 367, in make_tensor_proto
    _AssertCompatible(values, dtype)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 302, in _AssertCompatible
    (dtype.name, repr(mismatch), type(mismatch).__name__))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

这是我的代码：

stream_1 = vgg_model_1.get_layer('pool5').output
stream_1 = Flatten(name='flatten-1')(stream_1)

stream_2 = vgg_model_2.get_layer('pool5').output
stream_2 = Flatten(name='flatten-2')(stream_2)

fuse_layer = keras.layers.concatenate([stream_1, stream_2])

我使用的是VggFace，所以vgg_model_1和vgg_model_2是相同的CNN，但每个都有不同的输入。

Answer 1

我认为你实际上需要使用的不是concatenate，而是merge

请按照此链接答案了解更多详细信息。

跟随此链接获得关于merge的更多详细信息

要详细说明它，您需要为模型创建2个头部，而不是CNN的各个层。所以Model1把你的FaceEmotion数据作为输入，Model2接受其他类型的输入，

因此，要创建这两个头来给出一个组合输出，您需要创建第三个模型，它将合并这两个模型来提供一个单独的组合输出。

它可以按以下方式进行

multi_modal = Sequential()
multi_modal.add(Merge([Model1, Model2], mode='concat'))