VGG 16模型训练与tensorflow

Question

我正在尝试使用来自keras的VGG16来训练图像检测的模型。

基于这些文章(https://www.pyimagesearch.com/2019/06/03/fine-tuning-with-keras-and-deep-learning/和https://learnopencv.com/keras-tutorial-fine-tuning-using-pre-trained-models/)，我在VGG 16模型中添加了一些密集层。20例epoche的训练准确率在35% ~ 41%之间，与这些文章的结果不一致(90%以上)。

因此，我想知道，下面的代码做错了什么吗？

基本设定

url='/content/drive/My Drive/fer2013.csv'
batch_size = 64
img_width,img_height = 48,48

# 0=Angry, 1=Disgust, 2=Fear, 3=Happy, 4=Sad, 5=Surprise, 6=Neutral
num_classes = 7 
model_path = '/content/drive/My Drive/Af/cnn.h5'

df=pd.read_csv(url)  

def _load_fer():
    # Load training and eval data
    df = pd.read_csv(url, sep=',')
    train_df = df[df['Usage'] == 'Training']
    eval_df = df[df['Usage'] == 'PublicTest']
    return train_df, eval_df

def _preprocess_fer(df,label_col='emotion',feature_col='pixels'):
    labels, features = df.loc[:, label_col].values.astype(np.int32), [
        np.fromstring(image, np.float32, sep=' ')
        for image in df.loc[:, feature_col].values]
    
    labels = [to_categorical(l, num_classes=num_classes) for l in labels]

    features = np.stack((features,) * 3, axis=-1)
    features /= 255
    features = features.reshape(features.shape[0], img_width, img_height,3)
    return features, labels

# Load fer data
train_df, eval_df = _load_fer()

# preprocess fer data
x_train, y_train = _preprocess_fer(train_df)
x_valid, y_valid = _preprocess_fer(eval_df)

gen = ImageDataGenerator(
        rotation_range=40,
        width_shift_range=0.2,
        height_shift_range=0.2,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True,
        fill_mode='nearest')

train_generator = gen.flow(x_train, y_train, batch_size=batch_size)
predict_size_train = int(np.math.ceil(len(x_train) / batch_size)) 

input_tensor = Input(shape=(img_width, img_height, 3)) 

现在是模特儿训练部分

baseModel = VGG16(
    include_top=False, weights='imagenet',
    input_tensor=input_tensor
    )

# Construct the head of the model that will be placed on top of the base model (fine tuning)

headModel = baseModel.output
headModel = Flatten()(headModel)
headModel = Dense(1024, activation="relu")(headModel)
#headModel = Dropout(0.5)(headModel)
headModel = BatchNormalization()(headModel)
headModel = Dense(num_classes, activation="softmax")(headModel)

model = Model(inputs=baseModel.input, outputs=headModel)

for layer in baseModel.layers:
  layer.trainable = False

模型摘要

model.compile(loss='categorical_crossentropy', 
                       optimizer=tf.keras.optimizers.Adam(lr=0.001), 
                       metrics=['accuracy'])

 history = model.fit(train_generator, 
                    steps_per_epoch=predict_size_train * 1, 
                    epochs=20,
                    validation_data=valid_generator,
                    validation_steps=predict_size_valid)

结果：训练后结果会非常感谢您的建议。诚挚的问候。

Answer 1

因为你冻结了所有的图层，只有一个致密的图层可能不能给你想要的精度。另外，如果您不着急，您可能不会设置validation_steps和steps_per_epochs参数。在这教程中，模型也有波动，这是不想要的。

我建议：

    for layer in baseModel.layers:
    layer.trainable = False

base_out = baseModel.get_layer('block3_pool').output // layer name may be different, 
                                                       check with model baseModel.summary

这样你就可以得到特效层的输出。得到输出后，您可以添加一些卷积。在旋转之后，尝试堆叠更密集的层，如：

x = tf.keras.layers.Flatten()(x)

x = Dense(512, activation= 'relu')(x)
x = Dropout(0.3)(x)
x = Dense(256, activation= 'relu')(x)
x = Dropout(0.2)(x)

output_model = Dense(num_classes, activation = 'softmax')(x)

如果您不想添加卷积并完全使用baseModel，那么这也很好，但是您可以这样做：

for layer in baseModel.layers[:12]: // 12 is random you can try different ones. Not 
                                       all layers are frozen this time.
    layer.trainable = False

for i, layer in enumerate(baseModel.layers):
       print(i, layer.name, layer.trainable)
       // check frozen layers   

之后，您可以尝试设置：

headModel = baseModel.output
headModel = Flatten()(headModel)
headModel = Dense(1024, activation="relu")(headModel)
headModel = Dropout(0.5)(headModel)
headModel = Dense(512, activation="relu")(headModel)
headModel = Dense(num_classes, activation="softmax")(headModel)

如果你看到你的模式是学习，但你的损失有波动，那么你可以降低学习率。或者您可以使用ReduceLROnPlateau回调：

rd_lr = ReduceLROnPlateau(monitor='val_loss', factor = np.sqrt(0.1), patience= 4, verbose = 1, min_lr = 5e-8)

参数完全取决于你的模型。有关更多详细信息，请参见文档

Answer 2

y_train内容的形式是什么？如果它们是整数值，则需要将它们转换为

y_train=tf.keras.utils.to_categorical(train, num_classes)

因为您在model.compile中使用的是损失=‘范畴交叉熵’。此外，VGG16要求像素在-1和+1之间缩放，因此包括

gen = ImageDataGenerator(tf.keras.applications.vgg16.preprocess_input, etc

当你在训练的时候

for layer in baseModel.layers:
  layer.trainable = False

因此，你只是训练密集的一层，这是可以的，但可能不会给你高精度。您可能想离开VGG作为可训练，但这当然需要更长的时间。或者，在你和VGG一起训练之后，再把它改为可训练的，再运行几个时代来微调模型。