模型拟合时的训练精度没有反映在混淆矩阵中

Question

我正在训练一个图像分类模型，以对某些包含山、瀑布和人类的图像进行分类。目前，我正在使用Vgg-net (传输学习)来训练这些数据。在训练过程中，我对训练数据的准确率几乎达到93%，对验证数据的准确率达到90%。然而，当我想用训练数据中的混淆矩阵来检查训练过程中的错误分类时，其准确性似乎要低得多。只有大约30%的数据被正确分类。

我试过在其他类似的图像分类问题上检查混淆矩阵，但它似乎显示了我们在训练时所看到的准确性。

创建用于培训和验证的ImageDataGenerator对象的代码

from keras.applications.inception_v3 import preprocess_input, decode_predictions
#Train DataSet Generator with Augmentation
print("\nTraining Data Set")
train_generator = ImageDataGenerator(preprocessing_function=preprocess_input)
train_flow = train_generator.flow(
    train_images, train_labels,
    batch_size = BATCH_SIZE
)

#Validation DataSet Generator with Augmentation
print("\nValidation Data Set")
val_generator = ImageDataGenerator(preprocessing_function=preprocess_input)
val_flow = val_generator.flow(
    validation_images,validation_labels,
    batch_size = BATCH_SIZE
)

构建模型并编译它的代码

# Initialize InceptionV3 with transfer learning
base_model = applications.vgg16.VGG16(weights='imagenet', 
                                include_top=False, 
                                input_shape=(WIDTH, HEIGHT,3))

# add a global spatial average pooling layer
x = base_model.output

x = GlobalAveragePooling2D()(x)
# and a dense layer
x = Dense(1024, activation='relu')(x)
predictions = Dense(len(categories), activation='softmax')(x)

# first: train only the top layers (which were randomly initialized)
# i.e. freeze all convolutional InceptionV3 layers
for layer in base_model.layers:
    layer.trainable = False


# this is the model we will train
model = Model(inputs=base_model.input, outputs=predictions)

# compile the model (should be done *after* setting layers to non-trainable)
model.compile(optimizer=optimizers.RMSprop(lr=1e-4), metrics=['accuracy'], loss='categorical_crossentropy')
model.summary()

用于将模型与培训数据集相匹配并使用验证数据集进行验证的代码。

import math
top_layers_file_path=r"/content/drive/My Drive/intel-image-classification/intel-image-classification/top_layers.iv3.hdf5"

checkpoint = ModelCheckpoint(top_layers_file_path, monitor='loss', verbose=1, save_best_only=True, mode='min')
tb = TensorBoard(log_dir=r'/content/drive/My Drive/intel-image-classification/intel-image-classification/logs', batch_size=BATCH_SIZE, write_graph=True, update_freq='batch')
early = EarlyStopping(monitor="loss", mode="min", patience=5)
csv_logger = CSVLogger(r'/content/drive/My Drive/intel-image-classification/intel-image-classification/logs/iv3-log.csv', append=True)

history = model.fit_generator(train_flow, 
                              epochs=30, 
                              verbose=1,
                              validation_data=val_flow, 
                              validation_steps=math.ceil(validation_images.shape[0]/BATCH_SIZE),                              
                              steps_per_epoch=math.ceil(train_images.shape[0]/BATCH_SIZE),
                              callbacks=[checkpoint, early, tb, csv_logger])

培训步骤显示了以下准确性：

Epoch 1/30

91/91 [==============================] - 44s 488ms/step - loss: 0.6757 - acc: 0.7709 - val_loss: 0.4982 - val_acc: 0.8513

Epoch 2/30

91/91 [==============================] - 32s 349ms/step - loss: 0.4454 - acc: 0.8395 - val_loss: 0.3980 - val_acc: 0.8557

.
.

Epoch 20/30

91/91 [==============================] - 32s 349ms/step - loss: 0.2026 - acc: 0.9238 - val_loss: 0.2684 - val_acc: 0.8940

.
.

Epoch 30/30

91/91 [==============================] - 32s 349ms/step - loss: 0.1739 - acc: 0.9364 - val_loss: 0.2616 - val_acc: 0.8984

对培训数据集本身进行预测：

import math
import numpy as np
predictions = model.predict_generator(
    train_flow,
    verbose=1,
    steps=math.ceil(train_images.shape[0]/BATCH_SIZE))
predicted_classes = [x[0] for x in enc.inverse_transform(predictions)]
true_classes = [x[0] for x in enc.inverse_transform(train_labels)]

enc - OneHotEncoder

然而，混淆矩阵如下所示：

import sklearn

sklearn.metrics.confusion_matrix(
    true_classes, 
    predicted_classes,
    labels=['mountain','people','waterfall'])

混乱矩阵(无法上传更好看的图片)

  ([[315, 314, 283],

   [334, 309, 273],

   [337, 280, 263]])

完整的代码已上载到v7.ipynb上。

Answer 1

我相信这是因为train_generator.flow has shuffle=True by 默认设置。predicted_classes中的这一结果与train_labels不匹配。

也许将shuffle=False设置在train_generator.flow上应该对此有所帮助，或者使用类似的方法可能更容易理解。

predicted_classes = []
true_classes = []
for i in range(len(train_flow)):  # you can just use `len(train_flow)` instead of `math.ceil(train_images.shape[0]/BATCH_SIZE))`
    x, y = train_flow[i]  # you can use `train_flow[i]` like this
    z = model.predict(x)
    for j in range(x.shape[0]):
        predicted_classes.append(z[j])
        true_classes.append(y[j])

我还没试过这个，不过应该管用的。