使用YellowBrick的分类报告

Question

我最近在虹膜数据集上实现了概率神经网络。我试图使用YellowBrick分类器打印分类报告，但当我运行此代码时，我得到一个错误。如下所示。

from neupy import algorithms
model = algorithms.PNN(std=0.1, verbose=True, batch_size = 500)
model.train(X_train, Y_train)
predictions = model.predict(X_test)


from yellowbrick.classifier import ClassificationReport
visualizer = ClassificationReport(model, support=True)

visualizer.fit(X_train, Y_train)  # Fit the visualizer and the model
visualizer.score(X_test, Y_test)  # Evaluate the model on the test data
visualizer.show()  

此代码返回此错误。

YellowbrickTypeError: This estimator is not a classifier; try a regression or clustering score visualizer instead!

当我为其他分类模型尝试相同的分类报告代码时，它起作用了。我没有头绪。为什么会发生这种情况？有人能帮我解决这个问题吗？

Answer 1

Yellowbrick旨在与scikit-learn一起使用，并使用sklearn的类型检查系统来检测模型是否适合特定类别的机器学习问题。如果neupy PNN模型实现了scikit学习估计器API (例如fit()和predict()) -则可以直接使用该模型，并通过使用force_model=True参数绕过类型检查，如下所示：

visualizer = ClassificationReport(model, support=True, force_model=True)

然而，在快速浏览neupy documentation之后，这似乎不一定有效，因为新方法被命名为train而不是fit，并且因为PNN模型不实现score()方法，也不支持_后缀学习参数。

解决方案是创建一个围绕PNN模型的轻量级包装器，该模型将其公开为sklearn估计器。在Yellowbrick数据集上进行测试，这似乎是有效的：

from sklearn import metrics
from neupy import algorithms
from sklearn.base import BaseEstimator
from yellowbrick.datasets import load_occupancy
from yellowbrick.classifier import ClassificationReport
from sklearn.model_selection import train_test_split


class PNNWrapper(algorithms.PNN, BaseEstimator):
    """
    The PNN wrapper implements BaseEstimator and allows the classification
    report to score the model and understand the learned classes.
    """

    @property
    def classes_(self):
        return self.classes

    def score(self, X_test, y_test):
        y_hat = self.predict(X_test)
        return metrics.accuracy_score(y_test, y_hat)


# Load the binary classification dataset 
X, y = load_occupancy()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Create and train the PNN model using the sklearn wrapper
model = PNNWrapper(std=0.1, verbose=True, batch_size=500)
model.train(X_train, y_train)

# Create the classification report
viz = ClassificationReport(
    model, 
    support=True, 
    classes=["not occupied", "occupied"], 
    is_fitted=True, 
    force_model=True, 
    title="PNN"
)

# Score the report and show it
viz.score(X_test, y_test)
viz.show()

虽然neupy目前还不被Yellowbrick支持，但如果你感兴趣的话，它可能值得submitting an issue建议将neupy添加到contrib中，类似于statsmodels在Yellowbrick中的实现方式。