SHAP DeepExplainer:包含"nan“值的shap_values

Question

我的shap值有问题，下面是我的模型：

Model: "model_4"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_5 (InputLayer)            [(None, 158)]        0                                            
__________________________________________________________________________________________________
model_1 (Model)                 (None, 158)          57310       input_5[0][0]                    
__________________________________________________________________________________________________
subtract_4 (Subtract)           (None, 158)          0           input_5[0][0]                    
                                                                 model_1[5][0]                    
__________________________________________________________________________________________________
multiply_4 (Multiply)           (None, 158)          0           subtract_4[0][0]                 
                                                                 subtract_4[0][0]                 
__________________________________________________________________________________________________
lambda_4 (Lambda)               (None,)              0           multiply_4[0][0]                 
__________________________________________________________________________________________________
reshape_3 (Reshape)             (None, 1)            0           lambda_4[0][0]                   
==================================================================================================
Total params: 57,310
Trainable params: 57,310
Non-trainable params: 0
__________________________________________________________________________________________________

我呼吁：

scores = new_model.predict(X_test_scaled)
scores = scores.reshape(scores.shape[0],1)
toexplain = np.append(X_test_scaled, scores, axis = 1)
toexplain = pd.DataFrame(toexplain)
toexplain.sort_values(by = [158], ascending=False, inplace=True)
toexplain = toexplain.iloc[0:16]
toexplain.drop(columns = [158], axis = 1, inplace = True)

explainer=shap.DeepExplainer(new_model, df_sampled_X_train_scaled)
shap_values = explainer.shap_values(toexplain, check_additivity=False)

但是，我的shap值看起来如下(对于第一个实例)：

shap_values[0]

array([        nan,         nan,         nan,  0.08352888,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,  0.03286453,
               nan,         nan,  0.2984612 ,         nan,         nan,
               nan,  0.01110088, -0.85235232,         nan,         nan,
               nan,         nan,         nan,         nan, -0.27935541,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan, -0.18422949,  0.01466912,         nan,
               nan,         nan, -0.1688329 ,  0.07462809,  0.03071906,
               nan, -0.00554245,         nan,         nan,         nan,
               nan,  0.04587848,         nan,         nan,         nan,
               nan,  0.05448143,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,  0.00933742,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,  0.00919492,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,         nan,         nan,         nan,
               nan,         nan,         nan])

我很确定我不应该在我的shap_values中包含nan值，但是我似乎找不到最初的问题。此外，shap.force_plot给出的预测值与我模型的预测值不同，这就是为什么我首先检查了shap_values。

有人知道我怎么解决这个问题吗？

Answer 1

好吧，通过阅读shap的源代码，我意识到它没有考虑到这些数据是熊猫的数据，即使文档中说的不是这样。

它使用numpy.arrays工作