级联层的数值误差

Question

嗯，我试图在Pubmed200k随机对照临床试验数据集上复制一个来自纸的模型，你可以在这个纸中找到它。

具有5个唯一标记用于输出的数据集如下所示：

​

嗯，我已经取得了一些进展，我尽力复制了这篇论文，但最后，我得到了一个值错误：

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-55-d583261b0496> in <module>()
     23 
     24 # 4. create an output layer
---> 25 dropout_layer = tf.keras.layers.Dropout(0.5,name='dropout_layer')(concat_model)
     26 dense_layer = tf.keras.layers.Dense(200,name='dense_layer')(dropout_layer)
     27 dropout_layer = tf.keras.layers.Dropout(0.5,name='dropout_layer_2')(dense_layer)

1 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/constant_op.py in convert_to_eager_tensor(value, ctx, dtype)
    100       dtype = dtypes.as_dtype(dtype).as_datatype_enum
    101   ctx.ensure_initialized()
--> 102   return ops.EagerTensor(value, ctx.device_name, dtype)
    103 
    104 

ValueError: Exception encountered when calling layer "dropout_layer" (type Dropout).

Attempt to convert a value (<keras.layers.merge.Concatenate object at 0x7f51a0b5f850>) with an unsupported type (<class 'keras.layers.merge.Concatenate'>) to a Tensor.

Call arguments received:
  • inputs=<keras.layers.merge.Concatenate object at 0x7f51a0b5f850>
  • training=False

对于代码，我尽我最大的努力去评论尽可能多的事情。

# 1. word model:
inputs = tf.keras.layers.Input(
    shape=(1,), dtype=tf.string, name="text_input"
)  # takes a list of input
text_vectorization_layer = text_vectorizer(inputs)  # word level vectorizer
text_embedding_layer = embedding(text_vectorization_layer)  # word level embedding
# flatten_layer = tf.keras.layers.Flatten()(text_embedding_layer)
outputs = tf.keras.layers.Dense(300, activation="relu", name="text_output")(
    text_embedding_layer
)  # output of 300 from fig 1. https://arxiv.org/pdf/1612.05251.pdf
token_model = tf.keras.Model(inputs, outputs)  # token model

# 2. char model:
inputs = tf.keras.layers.Input(shape=(1,), dtype=tf.string, name="char_input")
char_vectorization_layer = char_vectorizer(inputs)  # character level vectorization
char_embedding_layer = char_embedding(char_vectorization_layer)  # char level embedding
outputs = tf.keras.layers.Bidirectional(
    tf.keras.layers.LSTM(25), name="bi_directional_lstm"
)(
    char_embedding_layer
)  # bi directional lstm as output
char_model = tf.keras.Model(inputs, outputs)

# 3. concatinating both the model
concat_model = tf.keras.layers.Concatenate([token_model, char_model])

# 4. create output layer
dropout_layer = tf.keras.layers.Dropout(0.5, name="dropout_layer")(concat_model)
dense_layer = tf.keras.layers.Dense(200, name="dense_layer")(dropout_layer)
dropout_layer = tf.keras.layers.Dropout(0.5, name="dropout_layer_2")(dense_layer)
output_layer = tf.keras.layer.Dense(5, activation="softmax", name="final_output_layer")(
    dropout_layer
)

我没有得到连接层的输出结果，因为它导致了辍学层的错误。

谢谢@Djinn，修正了代码：

# 1. word model:
inputs = tf.keras.layers.Input(
    shape=(1,), dtype=tf.string, name="text_input"
)  # takes a list of input
text_vectorization_layer = text_vectorizer(inputs)  # word level vectorizer
text_embedding_layer = embedding(text_vectorization_layer)  # word level embedding
flatten_layer = tf.keras.layers.Flatten()(text_embedding_layer)
outputs = tf.keras.layers.Dense(300, activation="relu", name="text_output")(
    flatten_layer
)  # output of 300 from fig 1. https://arxiv.org/pdf/1612.05251.pdf
token_model = tf.keras.Model(inputs, outputs)  # token model

# 2. char model:
inputs = tf.keras.layers.Input(shape=(1,), dtype=tf.string, name="char_input")
char_vectorization_layer = char_vectorizer(inputs)  # character level vectorization
char_embedding_layer = char_embedding(char_vectorization_layer)  # char level embedding
outputs = tf.keras.layers.Bidirectional(
    tf.keras.layers.LSTM(25), name="bi_directional_lstm"
)(
    char_embedding_layer
)  # bi directional lstm as output
char_model = tf.keras.Model(inputs, outputs)

# 3. concatinating both the model
concat_model = tf.keras.layers.Concatenate()([token_model.output, char_model.output])

# 4. create output layer
dropout_layer = tf.keras.layers.Dropout(0.5, name="dropout_layer")(concat_model)
dense_layer = tf.keras.layers.Dense(200, name="dense_layer")(dropout_layer)
dropout_layer = tf.keras.layers.Dropout(0.5, name="dropout_layer_2")(dense_layer)
output_layer = tf.keras.layers.Dense(
    5, activation="softmax", name="final_output_layer"
)(dropout_layer)

# 5.final model
model_2 = tf.keras.Model(
    inputs=[token_model.input, char_model.input],
    outputs=output_layer,
    name="token_char_model",
)

Answer 1

你正在连接两个模型。您可能希望连接它们的输出(或者最后一层，如果您想这样做的话)。如果需要，可以连接任何兼容的层)。您也错误地调用了Concatenate。大写为C的Concatenate是Concatenate(params)([layers])。具有小写c的concatenate是concatenate([layers], params)

更改：

concat_model = tf.keras.layers.Concatenate([token_model, char_model])

至

concat_model = tf.keras.layers.Concatenate()([token_model.output, char_model.output])

# or if you changed the name of the two outputs to, for example, [output0, output1]
concat_model = tf.keras.layers.Concatenate()([output0, output1])