如何评估rasa nlu模型

Question

使用kaggle笔记本中的run_evaluation模块评估rasa nlu模型时出错，如下所示：

TypeError: classification_report() got an unexpected keyword argument 'output_dict'



from rasa_nlu.evaluate import run_evaluation

run_evaluation("nlu.md", model_directory)
expected output:


INFO:rasa_nlu.evaluate:There were some nlu intent classification errors. Use `--verbose` to show them in the log.
INFO:rasa_nlu.evaluate:Model prediction errors saved to errors.json.
INFO:rasa_nlu.evaluate:Confusion matrix, without normalization: 
[[10  1  0  0  0  0  0]
 [ 0 13  0  0  0  0  0]
 [ 0  0 10  0  0  0  0]
 [ 0  0  0  5  0  0  0]
 [ 0  0  0  0  6  0  0]
 [ 0  0  0  0  0 15  0]
 [ 0  0  0  0  0  0 25]]

Answer 1

这似乎和你用过的scikit学习版(https://github.com/RasaHQ/rasa/issues/1742)有关。请尝试安装scikit-learn 0.20.2，或者更好地通过执行pip install rasa升级到Rasa 1.x。

Answer 2

evaluate函数现在称为test。试试这个：

从rasa_nlu.test导入run_evaluation

参考资料来源：https://forum.rasa.com/t/run-evaluation-function-can-not-be-imported/9258