用skopt优化超参数hidden_layer_size MLPClassifier

Question

如何利用MLPClassifier和skopt优化神经网络中的层数和隐层大小

通常，我会指定我的空间，比如：

Space([Integer(name = 'alpha_2', low = 1, high = 2),
       Real(10**-5, 10**0, "log-uniform", name='alpha_2')])

(比方说超参数alpha_1和alpha_2)。

使用sklearn中的神经网络实现，我需要调优hidden_layer_sizes，这是一个元组：

hidden_layer_sizes : tuple，length = n_layers - 2，default=(100 ) ith元素表示ith隐藏层中神经元的数目。

我如何用Space来表示这个

Answer 1

如果您正在使用gp_minimize，您可以将隐藏层的数量和每层的神经元作为Space中的参数。在目标函数的定义中，您可以手动创建超参数hidden_layer_sizes。

这是来自scikit优化主页的一个示例，现在使用一个MLPRegressor：

import numpy as np
from sklearn.datasets import load_boston
from sklearn.neural_network import MLPRegressor
from sklearn.model_selection import cross_val_score
from skopt.space import Real, Integer, Categorical 
from skopt.utils import use_named_args
from skopt import gp_minimize

boston = load_boston()
X, y = boston.data, boston.target
n_features = X.shape[1]

reg = MLPRegressor(random_state=0)

space=[
    Categorical(['tanh','relu'],name='activation'),
    Integer(1,4,name='n_hidden_layer'),
    Integer(200,2000,name='n_neurons_per_layer')]

@use_named_args(space)

def objective(**params):
    n_neurons=params['n_neurons_per_layer']
    n_layers=params['n_hidden_layer']

    # create the hidden layers as a tuple with length n_layers and n_neurons per layer
    params['hidden_layer_sizes']=(n_neurons,)*n_layers

    # the parameters are deleted to avoid an error from the MLPRegressor
    params.pop('n_neurons_per_layer')
    params.pop('n_hidden_layer')

    reg.set_params(**params)

    return -np.mean(cross_val_score(reg, X, y, cv=5, n_jobs=-1,
                                    scoring="neg_mean_absolute_error"))

res_gp = gp_minimize(objective, space, n_calls=50, random_state=0)