Keras版本的组合交叉熵和校准损失

Question

我最近读了一篇题为“用于医学影像分类的神经网络的改进的可训练校准方法”的论文。该研究通过测量预测置信度和准确性(DCA)之间的差异，并将其作为辅助项添加到交叉熵损失中，将校准纳入深度学习模型的训练过程。https://github.com/GB-TonyLiang/DCA上提供了GitHub代码。据说DCA项适用于当交叉熵损失减少时应用惩罚，但准确性处于平台期。Pytorch中的代码如下：

import torch
from torch.nn import functional as F

def cross_entropy_with_dca_loss(logits, labels, weights=None, alpha=1., beta=10.):        
    ce = F.cross_entropy(logits, labels, weight=weights)

    softmaxes = F.softmax(logits, dim=1)
    confidences, predictions = torch.max(softmaxes, 1)
    accuracies = predictions.eq(labels)
    mean_conf = confidences.float().mean()
    acc = accuracies.float().sum()/len(accuracies)
    dca = torch.abs(mean_conf-acc)
    loss = alpha*ce+beta*dca
    
    return loss

我需要帮助在Keras中将其转换为自定义函数，并将其用于使用真实标签(y_true)和预测概率(y_pred)而不是逻辑的多类分类的分类交叉熵损失。

Answer 1

下面的代码可能是Keras中上述PyTorch代码的等价物。

除了weights参数之外。下面的代码片段可能会对您有所帮助。

请检查输出。如果出了什么问题。分享你的评论。

import tensorflow as tf
from keras.losses import CategoricalCrossentropy
from keras.activations import softmax

def cross_entropy_with_dca_loss(logits, labels, weights=None, alpha=1., beta=10.):
    cce = CategoricalCrossentropy() 
    ce = cce(logits, labels) # not sure about weights parameter.
    softmaxes = softmax(logits, axis=1)
    confidences = tf.reduce_max(softmaxes, axis=1)
    mean_conf = tf.reduce_mean(confidences)
    acc = tf.reduce_mean(tf.cast(tf.equal(logits, labels), dtype=tf.float32))
    dca = tf.abs(mean_conf - acc)
    loss = alpha * ce + beta * dca
    return loss

Answer 2

此代码片段可以获取真实的标签和预测的概率。y_pred是探测张量。不需要使用softmax功能。

import tensorflow as tf
from keras.metrics import CategoricalAccuracy
from keras.losses import CategoricalCrossentropy

# Assuming y_pred is prob tensor, y_true is one-hot encoded
def cross_entropy_with_dca_loss(y_true, y_pred, alpha=1., beta=10.):        
    ce = CategoricalCrossentropy(from_logits=False)(y_true,y_pred)
    predictions = tf.math.argmax(y_pred, axis=1)
    confidences = tf.reduce_max(y_pred, axis=1)
    mean_conf = tf.reduce_mean(confidences)
    acc_m = CategoricalAccuracy()
    acc_m.update_state(y_true, y_pred)
    acc = acc_m.result().numpy()
    dca = tf.abs(mean_conf-acc)
    loss = alpha*ce+beta*dca
    return loss

# test on a sample data
y_true = tf.constant([[0, 1, 0], [0, 0, 1]])
y_pred = tf.constant([[0.05, 0.95, 0], [0.1, 0.8, 0.1]])
L = cross_entropy_with_dca_loss(y_true, y_pred)
print("loss", L.numpy())