为什么历史存储auc和val_auc是递增的整数(auc_2，auc_4，...)？

Question

我是keras的初学者，今天我遇到了这样的问题，我不知道该如何处理。auc和val_auc的值以第一个偶数整数存储在history中，如auc、auc_2、auc_4、auc_6……诸若此类。

这阻止了我在Kfold交叉验证过程中管理和研究这些值，因为我不能访问history.history['auc']值，因为并不总是有这样的关键'auc'。代码如下：

from tensorflow.keras.models import Sequential # pylint: disable= import-error
from tensorflow.keras.layers import Dense # pylint: disable= import-error
from tensorflow.keras import Input # pylint: disable= import-error
from sklearn.model_selection import StratifiedKFold
from keras.utils.vis_utils import plot_model
from keras.metrics import AUC, Accuracy # pylint: disable= import-error

BATCH_SIZE  = 32
EPOCHS      = 10
K           = 5
N_SAMPLE    = 1168
METRICS     = ['AUC', 'accuracy']

SAVE_PATH   = '../data/exp/final/submodels/'


def create_mlp(model_name, keyword, n_sample= N_SAMPLE, batch_size= BATCH_SIZE, epochs= EPOCHS):

    df = readCSV(n_sample)
    skf = StratifiedKFold(n_splits = K, random_state = 7, shuffle = True)

    for train_index, valid_index in skf.split(np.zeros(n_sample), df[['target']]):

        x_train, y_train, x_valid, y_valid = get_train_valid_dataset(keyword, df, train_index, valid_index)
        model = get_model(keyword)

        history = model.fit(
            x = x_train,
            y = y_train,
            validation_data = (x_valid, y_valid),
            epochs = epochs
        )

def get_train_valid_dataset(keyword, df, train_index, valid_index):
    aux = df[[c for c in columns[keyword]]]
    return aux.iloc[train_index].values, df['target'].iloc[train_index].values, aux.iloc[valid_index].values, df['target'].iloc[valid_index].values

def create_callbacks(model_name, save_path, fold_var):
    checkpoint = ModelCheckpoint(
        save_path + model_name + '_' +str(fold_var),
        monitor=CALLBACK_MONITOR, 
        verbose=1,
        save_best_only= True,
        save_weights_only= True,
        mode='max'
    )

    return [checkpoint]

在main.py中，我调用了create_mlp('model0', 'euler', n_sample=100)，日志是(只有相关行)：

Epoch 9/10
32/80 [===========>..................] - ETA: 0s - loss: 0.6931 - auc: 0.5000 - acc: 0.5625
Epoch 00009: val_auc did not improve from 0.50000
80/80 [==============================] - 0s 1ms/sample - loss: 0.6931 - auc: 0.5000 - acc: 0.5000 - val_loss: 0.6931 - val_auc: 0.5000 - val_acc: 0.5000
Epoch 10/10
32/80 [===========>..................] - ETA: 0s - loss: 0.6932 - auc: 0.5000 - acc: 0.4375
Epoch 00010: val_auc did not improve from 0.50000
80/80 [==============================] - 0s 1ms/sample - loss: 0.6931 - auc: 0.5000 - acc: 0.5000 - val_loss: 0.6931 - val_auc: 0.5000 - val_acc: 0.5000
Train on 80 samples, validate on 20 samples
Epoch 1/10
32/80 [===========>..................] - ETA: 0s - loss: 0.7644 - auc_2: 0.3075 - acc: 0.5000WARNING:tensorflow:Can save best model only with val_auc available, skipping.
80/80 [==============================] - 1s 10ms/sample - loss: 0.7246 - auc_2: 0.4563 - acc: 0.5250 - val_loss: 0.6072 - val_auc_2: 0.8250 - val_acc: 0.6500
Epoch 2/10
32/80 [===========>..................] - ETA: 0s - loss: 0.7046 - auc_2: 0.4766 - acc: 0.5000WARNING:tensorflow:Can save best model only with val_auc available, skipping.
80/80 [==============================] - 0s 1ms/sample - loss: 0.6511 - auc_2: 0.6322 - acc: 0.5625 - val_loss: 0.5899 - val_auc_2: 0.8000 - val_acc: 0.6000

任何帮助都将不胜感激。我正在使用：

keras==2.3.1
tensorflow==1.14.0

Answer 1

使用tf.keras.backend.clear_session()

https://www.tensorflow.org/api_docs/python/tf/keras/backend/clear_session

Answer 2

在这行代码中：

 for train_index, valid_index in skf.split(np.zeros(n_sample), df[['target']]):

实际发生的情况是，您正在运行多个训练实例，原则5是sklearn默认的。

尽管您可以在以下位置获得不同的训练和验证集：

 x_train, y_train, x_valid, y_valid = get_train_valid_dataset(keyword, df, train_index, valid_index)

当你运行model.fit()时，

   history = model.fit(
            x = x_train,
            y = y_train,
            validation_data = (x_valid, y_valid),
            epochs = epochs,
            callbacks=create_callbacks(keyword + '_' + model_name, SAVE_PATH, folder)
        )

您可以看到，create_callbacks的参数是静态的，并且不会从一个训练实例更改到另一个。Keyword、model_name、SAVE_PATH和folder是在训练的5个实例中保持不变的参数。

因此，在TensorBoard中，所有结果都写入相同的路径。

您不希望这样做，您希望每次迭代都将其结果写入不同的路径。

您必须更改logdir参数，给它一个唯一的标识符。在这种情况下，每个训练迭代将在不同的位置写入其结果，因此混乱将消失。

Answer 3

我通过改用tensorflow==2.1.0解决了这个问题。希望它能帮助其他任何人。