在做GridSearchCV时,通过grid.score(.)获得的分数之间有什么区别?和grid.best_score_
请假定模型、特性、目标和param_grid已经就位。下面是我很想知道的代码的一部分。
grid = GridSearchCV(X_train, y_train)
grid.fit(X_train, y_train)
scores = grid.score(estimator=my_model, param_grid=params, cv=3,
return_train_score=True, scoring='neg_mean_squared_error')
best_score_1 = scores
best_score_2 = grid.best_score_best_score_1和best_score_2各有两种不同的输出,我想知道两者之间的区别,以及以下哪一种应该被认为是从给定的param_grid中得到的最好的分数。
以下是全部功能。
def apply_grid (df, model, features, target, params, test=False):
'''
Performs GridSearchCV after re-splitting the dataset, provides
comparison between train's MSE and test's MSE to check for
Generalization and optionally deploys the best-found parameters
on the Test Set as well.
Args:
df: DataFrame
model: a model to use
features: features to consider
target: labels
params: Param_Grid for Optimization
test: False by Default, if True, predicts on Test
Returns:
MSE scores on models and slice from the cv_results_
to compare the models generalization performance
'''
my_model = model()
# Split the dataset into train and test
X_train, X_test, y_train, y_test = train_test_split(df[features],
df[target], random_state=0)
# Resplit the train dataset for GridSearchCV into train2 and valid to keep the test set separate
X_train2, X_valid, y_train2, y_valid = train_test_split(train[features],
train[target] , random_state=0)
# Use Grid Search to find the best parameters from the param_grid
grid = GridSearchCV(estimator=my_model, param_grid=params, cv=3,
return_train_score=True, scoring='neg_mean_squared_error')
grid.fit(X_train2, y_train2)
# Evaluate on Valid set
scores = grid.score(X_valid, y_valid)
scores = scores # CONFUSION
print('Best MSE through GridSearchCV: ', grid.best_score_) # CONFUSION
print('Best MSE through GridSearchCV: ', scores)
print('I AM CONFUSED ABOUT THESE TWO OUTPUTS ABOVE. WHY ARE THEY DIFFERENT')
print('Best Parameters: ',grid.best_params_)
print('-'*120)
print('mean_test_score is rather mean_valid_score')
report = pd.DataFrame(grid.cv_results_)
# If test is True, deploy the best_params_ on the test set
if test == True:
my_model = model(**grid.best_params_)
my_model.fit(X_train, y_train)
predictions = my_model.predict(X_test)
mse = mean_squared_error(y_test, predictions)
print('TEST MSE with the best params: ', mse)
print('-'*120)
return report[['mean_train_score', 'mean_test_score']] 发布于 2021-09-03 09:40:20
更新
正如在滑雪文献中解释的那样,GridSearchCV接受传递的参数的所有参数列表,并尝试所有可能的组合以找到最佳参数。
为了评估哪些参数是最佳参数,计算了每个参数组合的k倍交叉验证。通过k重交叉验证,将训练集分为训练集和验证集(这是一个测试集)。例如,如果选择cv=5,则将数据集划分为5个不重叠的折叠,每个折叠用作验证集,而所有其他折叠都用作培训集。因此,在本例中,GridSearchCV计算5倍中每一次的平均验证分数(可以是准确性或其他),并对每个参数组合进行计算。然后,在GridsearchCV结束时,每个参数组合都有一个平均验证分数,并返回平均验证分数最高的参数。因此,与最佳参数相关联的平均验证分数存储在grid.best_score_变量中。
另一方面,grid.score(X_valid, y_valid)方法给出了对给定数据的评分,如果估计器已被修正,则(refit=True).This表示它不是5倍的平均精度,而是采用参数最好的模型,并使用训练集进行训练。然后,计算了X_valid上的预测值,并与y_valid进行了比较,以得到分数。
https://stackoverflow.com/questions/69040683
复制相似问题