如何使用Keras对视频数据进行培训？“转移学习”

Question

我想训练我的视频数据的手势识别模型，提出使用低LSTM和TimeDistributed层。这会是解决我问题的理想方法吗？

# Convolution
pool_size = 4

# LSTM
lstm_output_size = 1

print('Build model...')

model = Sequential()

model.add(TimeDistributed(Dense(62), input_shape=(img_width, img_height,3)))
model.add(Conv2D(32, (3, 3)))
model.add(Dropout(0.25))
model.add(Conv2D(32, (3, 3)))
model.add(MaxPooling2D(pool_size=pool_size))

# model.add(Dense(1))
model.add(TimeDistributed(Flatten()))
model.add(CuDNNLSTM(256, return_sequences=True))
model.add(CuDNNLSTM(256, return_sequences=True))
model.add(CuDNNLSTM(256, return_sequences=True))
model.add(CuDNNLSTM(lstm_output_size))
model.add(Dense(units = 1, activation = 'sigmoid'))
print('Train...')
model.summary()
# run epochs of sampling data then training

Answer 1

对于时序数据，LSTM网络通常是正确的选择。如果你想分析视频，那么结合2d卷积对我来说是合理的。但是，您必须将TimeDistributed应用于所有不期望序列数据的层上。在您的示例中，这意味着所有的外行都期望LSTM。

# Convolution
pool_size = 4

# LSTM
lstm_output_size = 1

print('Build model...')

model = Sequential()

model.add(TimeDistributed(Dense(62), input_shape=(img_width, img_height,3)))
model.add(TimeDistributed(Conv2D(32, (3, 3))))
model.add(Dropout(0.25))
model.add(TimeDistributed(Conv2D(32, (3, 3))))
model.add(TimeDistributed(MaxPooling2D(pool_size=pool_size)))

# model.add(Dense(1))
model.add(TimeDistributed(Flatten()))
model.add(CuDNNLSTM(256, return_sequences=True))
model.add(CuDNNLSTM(256, return_sequences=True))
model.add(CuDNNLSTM(256, return_sequences=True))
model.add(CuDNNLSTM(lstm_output_size))
model.add(Dense(units = 1, activation = 'sigmoid'))
print('Train...')
model.summary()
# run epochs of sampling data then training

最后一个密集层可以保持这种方式，因为最终的lstm没有输出序列。