当数据中没有正类时，AUC不是definied[sic]

Question

在尝试训练模型时，我面临以下问题：

Inferring Columns ...
Creating Data loader ...
Loading data ...
Exploring multiple ML algorithms and settings to find you the best model for ML task: binary-classification
For further learning check: https://aka.ms/mlnet-cli
|     Trainer                              Accuracy      AUC    AUPRC  F1-score  Duration #Iteration             |
[Source=AutoML, Kind=Trace] Channel started
[Source=AutoML, Kind=Trace] Evaluating pipeline xf=ColumnConcatenating{ col=Features:V1,V2,V3,V4,V5,V6} xf=Normalizing{ col=Features:Features} tr=AveragedPerceptronBinary{}  cache=+
[Source=AutoML, Kind=Error] Pipeline crashed: xf=ColumnConcatenating{ col=Features:V1,V2,V3,V4,V5,V6} xf=Normalizing{ col=Features:Features} tr=AveragedPerceptronBinary{}  cache=+ . Exception: System.ArgumentOutOfRangeException: AUC is not definied when there is no positive class in the data
Parameter name: PosSample
   at Microsoft.ML.Data.EvaluatorBase`1.AucAggregatorBase`1.ComputeWeightedAuc(Double& unweighted)
   at Microsoft.ML.Data.BinaryClassifierEvaluator.Aggregator.Finish()
   at Microsoft.ML.Data.BinaryClassifierEvaluator.<>c__DisplayClass32_0.<GetAggregatorConsolidationFuncs>b__0(UInt32 stratColKey, ReadOnlyMemory`1 stratColVal, Aggregator agg)
   at Microsoft.ML.Data.EvaluatorBase`1.ProcessData(IDataView data, RoleMappedSchema schema, Func`2 activeColsIndices, TAgg aggregator, AggregatorDictionaryBase[] dictionaries)
   at Microsoft.ML.Data.EvaluatorBase`1.Microsoft.ML.Data.IEvaluator.Evaluate(RoleMappedData data)
   at Microsoft.ML.Data.BinaryClassifierEvaluator.Evaluate(IDataView data, String label, String score, String predictedLabel)
   at Microsoft.ML.AutoML.BinaryMetricsAgent.EvaluateMetrics(IDataView data, String labelColumn)
   at Microsoft.ML.AutoML.RunnerUtil.TrainAndScorePipeline[TMetrics](MLContext context, SuggestedPipeline pipeline, IDataView trainData, IDataView validData, String labelColumn, IMetricsAgent`1 metricsAgent, ITransformer preprocessorTransform, FileInfo modelFileInfo, DataViewSchema modelInputSchema, AutoMLLogger logger)
[Source=AutoML, Kind=Trace] 1   NaN 00:00:00.3095729    xf=ColumnConcatenating{ col=Features:V1,V2,V3,V4,V5,V6} xf=Normalizing{ col=Features:Features} tr=AveragedPerceptronBinary{}  cache=+
|1    AveragedPerceptronBinary                  NaN      NaN      NaN       NaN       0.3          0             |
System.ArgumentOutOfRangeException: AUC is not definied when there is no positive class in the data
Parameter name: PosSample
   at Microsoft.ML.Data.EvaluatorBase`1.AucAggregatorBase`1.ComputeWeightedAuc(Double& unweighted)
   at Microsoft.ML.Data.BinaryClassifierEvaluator.Aggregator.Finish()
   at Microsoft.ML.Data.BinaryClassifierEvaluator.<>c__DisplayClass32_0.<GetAggregatorConsolidationFuncs>b__0(UInt32 stratColKey, ReadOnlyMemory`1 stratColVal, Aggregator agg)
   at Microsoft.ML.Data.EvaluatorBase`1.ProcessData(IDataView data, RoleMappedSchema schema, Func`2 activeColsIndices, TAgg aggregator, AggregatorDictionaryBase[] dictionaries)
   at Microsoft.ML.Data.EvaluatorBase`1.Microsoft.ML.Data.IEvaluator.Evaluate(RoleMappedData data)
   at Microsoft.ML.Data.BinaryClassifierEvaluator.Evaluate(IDataView data, String label, String score, String predictedLabel)
   at Microsoft.ML.AutoML.BinaryMetricsAgent.EvaluateMetrics(IDataView data, String labelColumn)
   at Microsoft.ML.AutoML.RunnerUtil.TrainAndScorePipeline[TMetrics](MLContext context, SuggestedPipeline pipeline, IDataView trainData, IDataView validData, String labelColumn, IMetricsAgent`1 metricsAgent, ITransformer preprocessorTransform, FileInfo modelFileInfo, DataViewSchema modelInputSchema, AutoMLLogger logger)
[Source=AutoML, Kind=Trace] Evaluating pipeline xf=ColumnConcatenating{ col=Features:V1,V2,V3,V4,V5,V6} xf=Normalizing{ col=Features:Features} tr=SdcaLogisticRegressionBinary{}  cache=+
[Source=AutoML, Kind=Error] Pipeline crashed: xf=ColumnConcatenating{ col=Features:V1,V2,V3,V4,V5,V6} xf=Normalizing{ col=Features:Features} tr=SdcaLogisticRegressionBinary{}  cache=+ . Exception: System.ArgumentOutOfRangeException: AUC is not definied when there is no positive class in the data
Parameter name: PosSample
   at Microsoft.ML.Data.EvaluatorBase`1.AucAggregatorBase`1.ComputeWeightedAuc(Double& unweighted)
   at Microsoft.ML.Data.BinaryClassifierEvaluator.Aggregator.Finish()
   at Microsoft.ML.Data.BinaryClassifierEvaluator.<>c__DisplayClass32_0.<GetAggregatorConsolidationFuncs>b__0(UInt32 stratColKey, ReadOnlyMemory`1 stratColVal, Aggregator agg)
   at Microsoft.ML.Data.EvaluatorBase`1.ProcessData(IDataView data, RoleMappedSchema schema, Func`2 activeColsIndices, TAgg aggregator, AggregatorDictionaryBase[] dictionaries)
   at Microsoft.ML.Data.EvaluatorBase`1.Microsoft.ML.Data.IEvaluator.Evaluate(RoleMappedData data)
   at Microsoft.ML.Data.BinaryClassifierEvaluator.Evaluate(IDataView data, String label, String score, String predictedLabel)
   at Microsoft.ML.AutoML.BinaryMetricsAgent.EvaluateMetrics(IDataView data, String labelColumn)
   at Microsoft.ML.AutoML.RunnerUtil.TrainAndScorePipeline[TMetrics](MLContext context, SuggestedPipeline pipeline, IDataView trainData, IDataView validData, String labelColumn, IMetricsAgent`1 metricsAgent, ITransformer preprocessorTransform, FileInfo modelFileInfo, DataViewSchema modelInputSchema, AutoMLLogger logger)
[Source=AutoML, Kind=Trace] 2   NaN 00:00:00.5833636    xf=ColumnConcatenating{ col=Features:V1,V2,V3,V4,V5,V6} xf=Normalizing{ col=Features:Features} tr=SdcaLogisticRegressionBinary{}  cache=+
|2    SdcaLogisticRegressionBinary              NaN      NaN      NaN       NaN       0.6          0             |
System.ArgumentOutOfRangeException: AUC is not definied when there is no positive class in the data
Parameter name: PosSample
   at Microsoft.ML.Data.EvaluatorBase`1.AucAggregatorBase`1.ComputeWeightedAuc(Double& unweighted)
   at Microsoft.ML.Data.BinaryClassifierEvaluator.Aggregator.Finish()
   at Microsoft.ML.Data.BinaryClassifierEvaluator.<>c__DisplayClass32_0.<GetAggregatorConsolidationFuncs>b__0(UInt32 stratColKey, ReadOnlyMemory`1 stratColVal, Aggregator agg)
   at Microsoft.ML.Data.EvaluatorBase`1.ProcessData(IDataView data, RoleMappedSchema schema, Func`2 activeColsIndices, TAgg aggregator, AggregatorDictionaryBase[] dictionaries)
   at Microsoft.ML.Data.EvaluatorBase`1.Microsoft.ML.Data.IEvaluator.Evaluate(RoleMappedData data)
   at Microsoft.ML.Data.BinaryClassifierEvaluator.Evaluate(IDataView data, String label, String score, String predictedLabel)
   at Microsoft.ML.AutoML.BinaryMetricsAgent.EvaluateMetrics(IDataView data, String labelColumn)
   at Microsoft.ML.AutoML.RunnerUtil.TrainAndScorePipeline[TMetrics](MLContext context, SuggestedPipeline pipeline, IDataView trainData, IDataView validData, String labelColumn, IMetricsAgent`1 metricsAgent, ITransformer preprocessorTransform, FileInfo modelFileInfo, DataViewSchema modelInputSchema, AutoMLLogger logger)
[Source=AutoML, Kind=Trace] Evaluating pipeline xf=ColumnConcatenating{ col=Features:V1,V2,V3,V4,V5,V6} tr=LightGbmBinary{}  cache=-
[Source=AutoML, Kind=Error] Pipeline crashed: xf=ColumnConcatenating{ col=Features:V1,V2,V3,V4,V5,V6} tr=LightGbmBinary{}  cache=- . Exception: System.ArgumentOutOfRangeException: AUC is not definied when there is no positive class in the data
Parameter name: PosSample
   at Microsoft.ML.Data.EvaluatorBase`1.AucAggregatorBase`1.ComputeWeightedAuc(Double& unweighted)
   at Microsoft.ML.Data.BinaryClassifierEvaluator.Aggregator.Finish()
   at Microsoft.ML.Data.BinaryClassifierEvaluator.<>c__DisplayClass32_0.<GetAggregatorConsolidationFuncs>b__0(UInt32 stratColKey, ReadOnlyMemory`1 stratColVal, Aggregator agg)
   at Microsoft.ML.Data.EvaluatorBase`1.ProcessData(IDataView data, RoleMappedSchema schema, Func`2 activeColsIndices, TAgg aggregator, AggregatorDictionaryBase[] dictionaries)
   at Microsoft.ML.Data.EvaluatorBase`1.Microsoft.ML.Data.IEvaluator.Evaluate(RoleMappedData data)
   at Microsoft.ML.Data.BinaryClassifierEvaluator.Evaluate(IDataView data, String label, String score, String predictedLabel)
   at Microsoft.ML.AutoML.BinaryMetricsAgent.EvaluateMetrics(IDataView data, String labelColumn)
   at Microsoft.ML.AutoML.RunnerUtil.TrainAndScorePipeline[TMetrics](MLContext context, SuggestedPipeline pipeline, IDataView trainData, IDataView validData, String labelColumn, IMetricsAgent`1 metricsAgent, ITransformer preprocessorTransform, FileInfo modelFileInfo, DataViewSchema modelInputSchema, AutoMLLogger logger)
[Source=AutoML, Kind=Trace] 3   NaN 00:00:00.0955045    xf=ColumnConcatenating{ col=Features:V1,V2,V3,V4,V5,V6} tr=LightGbmBinary{}  cache=-
|3    LightGbmBinary                            NaN      NaN      NaN       NaN       0.1          0             |
System.ArgumentOutOfRangeException: AUC is not definied when there is no positive class in the data
Parameter name: PosSample
   at Microsoft.ML.Data.EvaluatorBase`1.AucAggregatorBase`1.ComputeWeightedAuc(Double& unweighted)
   at Microsoft.ML.Data.BinaryClassifierEvaluator.Aggregator.Finish()
   at Microsoft.ML.Data.BinaryClassifierEvaluator.<>c__DisplayClass32_0.<GetAggregatorConsolidationFuncs>b__0(UInt32 stratColKey, ReadOnlyMemory`1 stratColVal, Aggregator agg)
   at Microsoft.ML.Data.EvaluatorBase`1.ProcessData(IDataView data, RoleMappedSchema schema, Func`2 activeColsIndices, TAgg aggregator, AggregatorDictionaryBase[] dictionaries)
   at Microsoft.ML.Data.EvaluatorBase`1.Microsoft.ML.Data.IEvaluator.Evaluate(RoleMappedData data)
   at Microsoft.ML.Data.BinaryClassifierEvaluator.Evaluate(IDataView data, String label, String score, String predictedLabel)
   at Microsoft.ML.AutoML.BinaryMetricsAgent.EvaluateMetrics(IDataView data, String labelColumn)
   at Microsoft.ML.AutoML.RunnerUtil.TrainAndScorePipeline[TMetrics](MLContext context, SuggestedPipeline pipeline, IDataView trainData, IDataView validData, String labelColumn, IMetricsAgent`1 metricsAgent, ITransformer preprocessorTransform, FileInfo modelFileInfo, DataViewSchema modelInputSchema, AutoMLLogger logger)
Exception occured while exploring pipelines:
Training failed with the exception: System.ArgumentOutOfRangeException: AUC is not definied when there is no positive class in the data
Parameter name: PosSample
   at Microsoft.ML.Data.EvaluatorBase`1.AucAggregatorBase`1.ComputeWeightedAuc(Double& unweighted)
   at Microsoft.ML.Data.BinaryClassifierEvaluator.Aggregator.Finish()
   at Microsoft.ML.Data.BinaryClassifierEvaluator.<>c__DisplayClass32_0.<GetAggregatorConsolidationFuncs>b__0(UInt32 stratColKey, ReadOnlyMemory`1 stratColVal, Aggregator agg)
   at Microsoft.ML.Data.EvaluatorBase`1.ProcessData(IDataView data, RoleMappedSchema schema, Func`2 activeColsIndices, TAgg aggregator, AggregatorDictionaryBase[] dictionaries)
   at Microsoft.ML.Data.EvaluatorBase`1.Microsoft.ML.Data.IEvaluator.Evaluate(RoleMappedData data)
   at Microsoft.ML.Data.BinaryClassifierEvaluator.Evaluate(IDataView data, String label, String score, String predictedLabel)
   at Microsoft.ML.AutoML.BinaryMetricsAgent.EvaluateMetrics(IDataView data, String labelColumn)
   at Microsoft.ML.AutoML.RunnerUtil.TrainAndScorePipeline[TMetrics](MLContext context, SuggestedPipeline pipeline, IDataView trainData, IDataView validData, String labelColumn, IMetricsAgent`1 metricsAgent, ITransformer preprocessorTransform, FileInfo modelFileInfo, DataViewSchema modelInputSchema, AutoMLLogger logger)
System.InvalidOperationException: Training failed with the exception: System.ArgumentOutOfRangeException: AUC is not definied when there is no positive class in the data
Parameter name: PosSample
   at Microsoft.ML.Data.EvaluatorBase`1.AucAggregatorBase`1.ComputeWeightedAuc(Double& unweighted)
   at Microsoft.ML.Data.BinaryClassifierEvaluator.Aggregator.Finish()
   at Microsoft.ML.Data.BinaryClassifierEvaluator.<>c__DisplayClass32_0.<GetAggregatorConsolidationFuncs>b__0(UInt32 stratColKey, ReadOnlyMemory`1 stratColVal, Aggregator agg)
   at Microsoft.ML.Data.EvaluatorBase`1.ProcessData(IDataView data, RoleMappedSchema schema, Func`2 activeColsIndices, TAgg aggregator, AggregatorDictionaryBase[] dictionaries)
   at Microsoft.ML.Data.EvaluatorBase`1.Microsoft.ML.Data.IEvaluator.Evaluate(RoleMappedData data)
   at Microsoft.ML.Data.BinaryClassifierEvaluator.Evaluate(IDataView data, String label, String score, String predictedLabel)
   at Microsoft.ML.AutoML.BinaryMetricsAgent.EvaluateMetrics(IDataView data, String labelColumn)
   at Microsoft.ML.AutoML.RunnerUtil.TrainAndScorePipeline[TMetrics](MLContext context, SuggestedPipeline pipeline, IDataView trainData, IDataView validData, String labelColumn, IMetricsAgent`1 metricsAgent, ITransformer preprocessorTransform, FileInfo modelFileInfo, DataViewSchema modelInputSchema, AutoMLLogger logger)
   at Microsoft.ML.CLI.CodeGenerator.CodeGenerationHelper.GenerateCode()
   at Microsoft.ML.CLI.Program.<>c__DisplayClass1_0.<Main>b__0(NewCommandSettings options)
Please see the log file for more info.
Exiting ...

以下是我正在尝试训练的数据样本的一小部分。即使我只使用这个小的子集，训练也会失败，并出现上面的错误。注意:列"Res“是我要求ML系统预测的内容。

V1,V2,V3,V4,V5,V6,Res
1.04,0,0,93,0.93,30,1
1.33,3,0.6,81,0.81,37,1
1.2,3,0.6,90,0.9,30,1
1.13,0,0,74,0.74,19,1
1.06,0,0,78,0.78,18,1
1.25,3,0.6,86,0.86,21,1
1.25,4,0.8,89,0.89,18,0
1.25,5,1,96,0.96,23,0

可见，"Res“一栏确实有正面和负面的标签。此外，我还尝试将此列中的值更改为"True/False“、"Yes/No”和"1/0“，如下所示。有没有人能提出解决这个问题的建议？

Answer 1

这个问题的解决方案是我没有加载足够的数据。验证和测试拆分是自动生成的，但有一些拆分没有正确类的示例。

我不知道这是否仍然是最新版本的问题，因为这已经有一段时间了。

加载大量数据！

Answer 2

你是如何加载数据的，这可能看起来你的标签列是空的，所以可能数据加载不正确。如果使用内置分隔符，请记住它缺省为TSV，因此您必须在其配置中指定mlContext.Data.LoadFromTextFile，如下所示：

mlContext.Data.LoadFromTextFile<YourClass>(@"c:\path\to.file", separatorChar:',',hasHeader:true)

Answer 3

确保您的测试和训练数据必须包含真和假两种数据，一旦我在测试和训练文件/数据集中包括真和假，我的问题就解决了。