如何实现多层感知器的超参数调谐？

Question

我是Python的初学者。我试图调整我的MLP模型如下。我有以下问题：

1)这是调整我的MLP的正确方法吗？

2)在运行代码之后，在给出最佳参数之前，它一直给我以粉红色的长警告，这个警告是什么？(我提供了下面模型的输出)。

3)“隐藏层的大小”：(100，)，(50,100，)，(50,75,100 )，我不确定这一行代码中隐藏层的数目和中子数，我需要改变这些数字吗？我有9个输入和19个输出，如果我需要改变这些数字，那么最好的隐藏层和中子数是多少来代替这些数字？

提前谢谢你！

from sklearn.neural_network import MLPRegressor
from sklearn.model_selection import GridSearchCV
X, y = scaled_df[[ "Part's Z-Height (mm)","Part's Weight (N)","Part's Volume (cm^3)","Part's Surface Area (cm^2)","Part's Orientation (Support's height) (mm)","Part's Orientation (Support's volume) (cm^3)","Layer Height (mm)","Printing/Scanning Speed (mm/s)","Infill Density (%)"]], scaled_df [["Climate change (kg CO2 eq.)","Climate change, incl biogenic carbon (kg CO2 eq.)","Fine Particulate Matter Formation (kg PM2.5 eq.)","Fossil depletion (kg oil eq.)","Freshwater Consumption (m^3)","Freshwater ecotoxicity (kg 1,4-DB eq.)","Freshwater Eutrophication (kg P eq.)","Human toxicity, cancer (kg 1,4-DB eq.)","Human toxicity, non-cancer (kg 1,4-DB eq.)","Ionizing Radiation (Bq. C-60 eq. to air)","Land use (Annual crop eq. yr)","Marine ecotoxicity (kg 1,4-DB eq.)","Marine Eutrophication (kg N eq.)","Metal depletion (kg Cu eq.)","Photochemical Ozone Formation, Ecosystem (kg NOx eq.)","Photochemical Ozone Formation, Human Health (kg NOx eq.)","Stratospheric Ozone Depletion (kg CFC-11 eq.)","Terrestrial Acidification (kg SO2 eq.)","Terrestrial ecotoxicity (kg 1,4-DB eq.)"]]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

mlp = MLPRegressor()
param_grid = {'hidden_layer_sizes': [(100,), (50,100,), (50,75,100,)],
              'activation': ['tanh','relu','lbfgs'],
              'solver': ['sgd', 'adam'],
              'learning_rate': ['constant','adaptive','invscaling'],
              'alpha': [0.0001, 0.05],
              'max_iter': [10000000000],
              'early_stopping': [False],
              'warm_start': [False]}
GS = GridSearchCV(mlp, param_grid=param_grid,n_jobs= -1,cv=5, scoring='r2')
                  
                  
GS.fit(X_train, y_train)

print(GS.best_params_)

模型输出：

/Users/opt/anaconda3/lib/python3.9/site-packages/sklearn/model_selection/_validation.py:615: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details: 
Traceback (most recent call last):
  File "/Users/opt/anaconda3/lib/python3.9/site-packages/sklearn/model_selection/_validation.py", line 598, in _fit_and_score
    estimator.fit(X_train, y_train, **fit_params)
  File "/Users/opt/anaconda3/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py", line 673, in fit
    return self._fit(X, y, incremental=False)
  File "/Users/opt/anaconda3/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py", line 357, in _fit
    self._validate_hyperparameters()
  File "/Users/opt/anaconda3/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py", line 448, in _validate_hyperparameters
    raise ValueError("The activation '%s' is not supported. Supported "
ValueError: The activation 'lbfgs' is not supported. Supported activations are ['identity', 'logistic', 'relu', 'softmax', 'tanh'].

  
/Users/opt/anaconda3/lib/python3.9/site-packages/sklearn/model_selection/_validation.py:615: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan. Details: 
Traceback (most recent call last):
  File "/Users/opt/anaconda3/lib/python3.9/site-packages/sklearn/model_selection/_validation.py", line 598, in _fit_and_score
    estimator.fit(X_train, y_train, **fit_params)
  File "/Users/opt/anaconda3/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py", line 673, in fit
    return self._fit(X, y, incremental=False)
  File "/Users/opt/anaconda3/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py", line 357, in _fit
    self._validate_hyperparameters()
  File "/Users/opt/anaconda3/lib/python3.9/site-packages/sklearn/neural_network/_multilayer_perceptron.py", line 448, in _validate_hyperparameters
    raise ValueError("The activation '%s' is not supported. Supported "
ValueError: The activation 'lbfgs' is not supported. Supported activations are ['identity', 'logistic', 'relu', 'softmax', 'tanh'].

  

  warnings.warn("Estimator fit failed. The score on this train-test"
/Users/opt/anaconda3/lib/python3.9/site-packages/sklearn/model_selection/_search.py:922: UserWarning: One or more of the test scores are non-finite: [-1.77668202e+01 -5.36865257e+00 -2.36108673e+01 -4.87857020e+00
 -5.37871220e+01 -7.33847388e+00 -1.18104844e+01 -4.31770716e+00
 -1.07982957e+01 -5.90410809e+00 -6.90785910e+01 -3.52662014e+00
 -1.28640696e+01 -2.08943515e+00 -8.90821473e+00 -3.54330679e+00
 -6.41294642e+01 -3.43469060e+00 -2.03283528e+01 -7.13374531e+00
 -1.94873065e+01 -6.80254241e+00 -8.26138602e+01 -3.60145086e+00
 -1.44247785e+01  1.82313539e-01 -1.27466817e+01  1.42093292e-02
 -8.58307682e+01  2.04012758e-01 -4.14186486e+00  6.44037466e-01
 -9.19520894e+00  5.57317462e-01 -7.21849359e+01  7.55050142e-01
 -2.59194519e+01 -3.13963793e+00 -3.21942292e+01 -3.62738244e+00
 -4.22512000e+01 -2.97608298e+00 -1.82051179e+01 -3.43752161e+00
 -1.99742744e+01 -2.04712456e+00 -6.01852277e+01 -4.21323595e+00
 -2.38904146e+01 -1.14621391e+00 -1.68763533e+01 -4.01997962e-02
 -5.93074730e+01 -3.83233190e-01 -2.86875402e+01 -2.29876280e+00
 -3.05378372e+01 -1.79326275e+00 -4.12499389e+01 -3.32545463e+00
 -2.34686687e+01  1.92512129e-01 -1.40481244e+01 -2.00564895e-01
 -2.84224859e+01  3.25566168e-01 -1.77123178e+01  7.28578976e-01
 -2.35861764e+01  5.12092072e-01 -3.44553330e+01  6.87623634e-01
             nan             nan             nan             nan
             nan             nan             nan             nan
             nan             nan             nan             nan
             nan             nan             nan             nan
             nan             nan             nan             nan
             nan             nan             nan             nan
             nan             nan             nan             nan
             nan             nan             nan             nan
             nan             nan             nan             nan]


'{'activation': 'tanh', 'alpha': 0.05, 'early_stopping': False, 'hidden_layer_sizes': (50, 75, 100), 'learning_rate': 'invscaling', 'max_iter': 10000000000, 'solver': 'adam', 'warm_start': False}

Answer 1

你的方法是好的，然而，它很难知道正确的层数/神经元前手。它确实是依赖于问题的。

网格搜索是一个选项，特别是要找到参数(10、100、1000)的数量级。然后，人们通常使用RandomizedSearchCV来围绕前面步骤中找到的最佳值来细化搜索。

请注意，还有一些更高级的工具，如KerasTuner，它允许应用更复杂的策略执行超参数搜索，例如在超参数空间上使用贝叶斯优化，但不确定这些工具是否与sklearn兼容。

尽管如此，您的警告/错误消息是因为您试图验证的某些值在您正在使用的sklearn版本中不可用，或者它们根本不存在。还要考虑的是，您正在验证的超参数的所有组合都必须是一致的(例如，如果docs说对于param a=0，b必须是非负的，则必须确保param网格的每个点都满足该条件)。

因此，总之，重复检查文档以确保您的超级there网格正常，并仔细阅读您的堆栈跟踪，因为其中描述了一些错误(比如不可用的激活lbsgf)。