如何使用英特尔推理引擎(OpenVINO)优化具有批处理规范层的keras模型？

Question

无法使用英特尔推理引擎(OpenVINO工具包R.5)优化keras模型

我冻结了我的模型，就像tutorial建议的那样。对keras模型进行了训练和测试。我需要对其进行优化以进行推断。但是，在自定义模型上运行模型优化器(mo.py脚本)时出现错误。

[ ERROR ] shapes (128,9) and (0,) not aligned: 9 (dim 1) != 0 (dim 0)

我的模型的最后几层(9是类的输出数)是：

conv2d_4 (Conv2D) (None, 4, 4, 128) 204928 batch_normalization_3[0][0]
__________________________________________________________________________________________________
activation_4 (Activation) (None, 4, 4, 128) 0 conv2d_4[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 4, 4, 128) 512 activation_4[0][0]
__________________________________________________________________________________________________
average_pooling2d_2 (AveragePoo (None, 1, 1, 128) 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
dropout_2 (Dropout) (None, 1, 1, 128) 0 average_pooling2d_2[0][0]
__________________________________________________________________________________________________
flatten (Flatten) (None, 128) 0 dropout_2[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 128) 16512 flatten[0][0]
__________________________________________________________________________________________________
activation_5 (Activation) (None, 128) 0 dense[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 128) 512 activation_5[0][0]
__________________________________________________________________________________________________
dropout_3 (Dropout) (None, 128) 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 9) 1161 dropout_3[0][0]
__________________________________________________________________________________________________
color_prediction (Activation) (None, 9) 0 dense_1[0][0]
__________________________________________________________________________________________________

由于存在BatchNormalization层，模型优化器失败。当我删除它们时，它会成功运行。然而，我使用冻结图形

tf.keras.backend.set_learning_phase(0) 

所以像BatchNormalization和Dropout这样的节点必须在冻结的图中删除，我不明白为什么不删除它们。

非常感谢!

Answer 1

我设法在具有批处理归一化层的Keras模型上运行了OpenVINO模型优化器。该模型的收敛速度似乎也略快一些。虽然测试分类率比没有BN的模型低5-7% (并且测试和训练数据集的分类率之间的差距更大)。我不确定是否从我的解决方案中的模型中正确地删除了BatchNormalization (但是openVINO模型文件不包含一个，所以它被删除了)。

删除BN和Dropout层：

#Clear any previous session.
tf.keras.backend.clear_session()
#This line must be executed before loading Keras model.
tf.keras.backend.set_learning_phase(0) 
model = tf.keras.models.load_model(weights_path)

for layer in model.layers:
    layer.training = False
    if isinstance(layer, tf.keras.layers.BatchNormalization):
        layer._per_input_updates = {}
    elif isinstance(layer, tf.keras.layers.Dropout):
        layer._per_input_updates = {}

然后冻结会话：

def freeze_session(session, keep_var_names=None, output_names=None, clear_devices=True):
"""
Freezes the state of a session into a pruned computation graph.

Creates a new computation graph where variable nodes are replaced by
constants taking their current value in the session. The new graph will be
pruned so subgraphs that are not necessary to compute the requested
outputs are removed.
@param session The TensorFlow session to be frozen.
@param keep_var_names A list of variable names that should not be frozen,
                    or None to freeze all the variables in the graph.
@param output_names Names of the relevant graph outputs.
@param clear_devices Remove the device directives from the graph for better portability.
@return The frozen graph definition.
"""
from tensorflow.python.framework.graph_util import convert_variables_to_constants
graph = session.graph
with graph.as_default():
    freeze_var_names = list(set(v.op.name for v in tf.global_variables()).difference(keep_var_names or []))
    output_names = output_names or []
    output_names += [v.op.name for v in tf.global_variables()]
    # Graph -> GraphDef ProtoBuf
    input_graph_def = graph.as_graph_def()
    if clear_devices:
        for node in input_graph_def.node:
            node.device = ""
    frozen_graph = convert_variables_to_constants(session, input_graph_def,
                                                output_names, freeze_var_names)
    return frozen_graph