如何使用sklearn Column Transformer？

Question

我先使用LabelEncoder，然后使用OneHotEncoder，尝试将分类值(在我的示例中是country列)转换为编码值，并且能够转换分类值。但是我得到了警告，像'categorical_features‘关键字被弃用了，“使用OneHotEncoder ColumnTransformer代替”。那么我如何使用ColumnTransformer来达到同样的效果呢？

下面是我的输入数据集和我尝试过的代码

Input Data set

Country Age Salary
France  44  72000
Spain   27  48000
Germany 30  54000
Spain   38  61000
Germany 40  67000
France  35  58000
Spain   26  52000
France  48  79000
Germany 50  83000
France  37  67000


import pandas as pd
import numpy as np
from sklearn.preprocessing import LabelEncoder, OneHotEncoder

#X is my dataset variable name

label_encoder = LabelEncoder()
x.iloc[:,0] = label_encoder.fit_transform(x.iloc[:,0]) #LabelEncoder is used to encode the country value
hot_encoder = OneHotEncoder(categorical_features = [0])
x = hot_encoder.fit_transform(x).toarray()

我得到的输出是，如何使用列转换器获得相同的输出

0(fran) 1(ger) 2(spain) 3(age)  4(salary)
1         0       0      44        72000
0         0       1      27        48000
0         1       0      30        54000
0         0       1      38        61000
0         1       0      40        67000
1         0       0      35        58000
0         0       1      36        52000
1         0       0      48        79000
0         1       0      50        83000
1         0       0      37        67000

我尝试了下面的代码

from sklearn.compose import ColumnTransformer, make_column_transformer

preprocess = make_column_transformer(

    ( [0], OneHotEncoder())
)
x = preprocess.fit_transform(x).toarray()

我可以用上面的代码对country列进行编码，但在转换后x varible中缺少年龄和薪水列

Answer 1

将连续数据编码为薪水有点奇怪。除非你已经将你的薪水绑定到特定的范围/类别，否则它没有任何意义。如果我是你，我会这么做：

import pandas as pd
import numpy as np

from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder



numeric_features = ['Salary']
numeric_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='median')),
    ('scaler', StandardScaler())])

categorical_features = ['Age','Country']
categorical_transformer = Pipeline(steps=[
    ('imputer', SimpleImputer(strategy='constant', fill_value='missing')),
    ('onehot', OneHotEncoder(handle_unknown='ignore'))])

preprocessor = ColumnTransformer(
    transformers=[
        ('num', numeric_transformer, numeric_features),
        ('cat', categorical_transformer, categorical_features)])

在这里，你可以用一个分类器来表示它。

clf = Pipeline(steps=[('preprocessor', preprocessor),
                  ('classifier', LogisticRegression(solver='lbfgs'))])  

按如下方式使用它：

clf.fit(X_train,y_train)

这将应用预处理器，然后将转换后的数据传递给预测器。

更新：

如果我们想要动态地选择数据类型，我们可以修改我们的预处理器以按数据类型使用列选择器：

from sklearn.compose import make_column_selector as selector

preprocessor = ColumnTransformer(
    transformers=[
        ('num', numeric_transformer, selector(dtype_include="numeric")),
        ('cat', categorical_transformer, selector(dtype_include="category"))])

使用GridSearch

param_grid = {
    'preprocessor__num__imputer__strategy': ['mean', 'median'],
    'classifier__C': [0.1, 1.0, 10, 100],
    'classifier__solver': ['lbfgs', 'sag'],
}

grid_search = GridSearchCV(clf, param_grid, cv=10)
grid_search.fit(X_train,y_train)

Answer 2

我认为这张海报并没有试图改变年龄和薪水。在文档(https://scikit-learn.org/stable/modules/generated/sklearn.compose.make_column_transformer.html)中，您只对转换器中指定的列执行ColumnTransformer (和make_column_transformer)操作(即，在您的示例中)。您应该设置remainder="passthrough“来获取其余的列。换句话说：

preprocessor = make_column_transformer( (OneHotEncoder(),[0]),remainder="passthrough")
x = preprocessor.fit_transform(x)

Answer 3

最简单的方法是在CVS数据帧上使用pandas虚拟对象

dataset = pd.read_csv("yourfile.csv")
dataset = pd.get_dummies(dataset,columns=['Country'])

完成后，数据集将如下所示

​