在scikit KerasRegressor中使用优化的示例

Question

我正在使用非常棒的scikit-optimize工具箱来进行超参数优化。我的目标是比较keras和scikit-learn模型。

根据示例https://scikit-optimize.github.io/stable/auto_examples/sklearn-gridsearchcv-replacement.html#sphx-glr-auto-examples-sklearn-gridsearchcv-replacement-py，仅使用了scikit学习模型。尝试下面的代码不允许在BayesSearchCV中集成keras模式。

# Function to create model, required for KerasRegressor
def create_model(optimizer='rmsprop', init='glorot_uniform'):
    # create model
    model = Sequential()
    model.add(Dense(12, input_dim=8, kernel_initializer=init, activation='relu'))
    model.add(Dense(8, kernel_initializer=init, activation='relu'))
    model.add(Dense(1, kernel_initializer=init, activation='linear'))
    # Compile model
    model.compile(loss='mse', optimizer=optimizer, metrics=['r2'])
    return model

model = KerasRegressor(build_fn=create_model, verbose=0)
NN_search = {
    'model': [model()],
    'model__optimizers': optimizers,
    'model__epochs' : epochs, 
    'model__batch_size' : batches, 
    'model__init' : init
}

有没有人成功地将merge分类器/正则器合并到BayesSearch CV中？

这将是非常有用的！

Answer 1

好吧，我找到了一个选项来定义基于全局参数建立的模型。因此，在scikit opt最小化函数中，目标函数被调用，这里设置了全局参数，并在create_model_NN函数中使用，该函数构建在keras scikit learn KerasRegressor包装器上。

def create_model_NN():
    #start the model making process and create our first layer
    model = Sequential()
    model.add(Dense(num_input_nodes, input_shape=(40,), activation=activation
                   ))
    #create a loop making a new dense layer for the amount passed to this model.
    #naming the layers helps avoid tensorflow error deep in the stack trace.
    for i in range(num_dense_layers):
        name = 'layer_dense_{0}'.format(i+1)
        model.add(Dense(num_dense_nodes,
                 activation=activation,
                        name=name
                 ))
    #add our classification layer.
    model.add(Dense(1,activation='linear'))
    
    #setup our optimizer and compile
    adam = Adam(lr=learning_rate)
    model.compile(optimizer=adam, loss='mean_squared_error',
                 metrics=['mse'])
    return model

def objective_NN(**params):
    print(params)

    global learning_rate
    learning_rate=params["learning_rate"]
    global num_dense_layers
    num_dense_layers=params["num_dense_layers"]
    global num_input_nodes
    num_input_nodes=params["num_input_nodes"]
    global num_dense_nodes
    num_dense_nodes=params["num_dense_nodes"]
    global activation
    activation=params["activation"]
    
    model = KerasRegressor(build_fn=create_model, epochs=100, batch_size=1000, verbose=0)
    X_train, X_test, y_train, y_test = train_test_split(X_time, y_time, test_size=0.33, random_state=42)
    
    model.fit(X_train, y_train)
    
    y_pr = model.predict(X_test)
    
    res = metrics.r2_score(y_test, y_pr)
return -res

并将其称为：

res_gp = gp_minimize(objective_NN, space_NN, n_calls=10, random_state=0)