StackingRegressor sklearn的超参数调优

Question

在我的问题中，我想使用一个简单的RandomizedSearchCV调谐器调优sklearn.ensemble.StackingRegressor。因为我们需要在实例化StackingRegressor()时定义估计器，所以我无法在我的param_distribution随机搜索中正确地定义估计器的参数空间。

我尝试了以下方法，但遇到了错误：

from sklearn.datasets import load_diabetes
from sklearn.linear_model import RidgeCV
from sklearn.svm import LinearSVR
from sklearn.model_selection import RandomizedSearchCV
from sklearn.ensemble import RandomForestRegressor, 
GradientBoostingRegressor
from sklearn.ensemble import StackingRegressor
X, y = load_diabetes(return_X_y=True)

rfr = RandomForestRegressor()
gbr = GradientBoostingRegressor()

estimators = [rfr, gbr]
sreg = StackingRegressor(estimators=estimators)
params = {'rfr__max_depth': [3, 5, 10, 100],
          'gbr__max_depth': [3, 5, 10, 100]}

grid = RandomizedSearchCV(estimator=sreg, 
                          param_distributions=params,
                          cv=3)
grid.fit(X,y)

我遇到了错误AttributeError: 'RandomForestRegressor' object has no attribute 'estimators_'。

有没有办法在StackingRegressor中调优不同估计器的参数？

Answer 1

如果您将估计器定义为估计器名称和估计器实例的元组列表，如下所示，您的代码应该可以工作。

import pandas as pd
from sklearn.datasets import load_diabetes
from sklearn.model_selection import RandomizedSearchCV
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor
from sklearn.ensemble import StackingRegressor

X, y = load_diabetes(return_X_y=True)

rfr = RandomForestRegressor()
gbr = GradientBoostingRegressor()

estimators = [('rfr', rfr), ('gbr', gbr)]

sreg = StackingRegressor(estimators=estimators)

params = {
    'rfr__max_depth': [3, 5],
    'gbr__max_depth': [3, 5]
}

grid = RandomizedSearchCV(
    estimator=sreg,
    param_distributions=params,
    n_iter=2,
    cv=3,
    verbose=1,
    random_state=100
)

grid.fit(X, y)

res = pd.DataFrame(grid.cv_results_)
print(res)
#    mean_fit_time  std_fit_time  ...  std_test_score  rank_test_score
# 0       1.121728      0.024188  ...        0.024546                2
# 1       1.096936      0.034377  ...        0.013047                1