Bert模型列车不想停下来

Question

我正在使用这个代码来训练Bert的土耳其语言模型分类与2个标签。但是当我运行以下代码时：

import numpy as np
import pandas as pd

df = pd.read_excel (r'preparedDataNoId.xlsx')
df = df.sample(frac = 1)

from sklearn.model_selection import train_test_split

train_df, test_df = train_test_split(df, test_size=0.10)

print('train shape: ',train_df.shape)
print('test shape: ',test_df.shape)

train_df["text"]=train_df["text"].apply(lambda r: str(r))
train_df['label']=train_df['label'].astype(int)
from simpletransformers.classification import ClassificationModel

model = ClassificationModel('bert', 'dbmdz/bert-base-turkish-uncased', use_cuda=False,num_labels=2,
                            args={'reprocess_input_data': True, 'overwrite_output_dir': True, 'num_train_epochs': 3, "train_batch_size": 64 , "fp16":False, "output_dir": "bert_model"})

model.train_model(train_df) 

这需要很长时间，它不会停止，屏幕会一直显示：

This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:

    if __name__ == '__main__':
        freeze_support()
        ...

Answer 1

如错误所示，您应该用if __name__ == '__main__':包装您的代码

所以你的代码应该是：

import numpy as np
import pandas as pd

if __name__ == '__main__':
    df = pd.read_excel(r'preparedDataNoId.xlsx')
    df = df.sample(frac=1)

    from sklearn.model_selection import train_test_split

    train_df, test_df = train_test_split(df, test_size=0.10)

    print('train shape: ', train_df.shape)
    print('test shape: ', test_df.shape)

    train_df["text"] = train_df["text"].apply(lambda r: str(r))
    train_df['label'] = train_df['label'].astype(int)
    from simpletransformers.classification import ClassificationModel

    model = ClassificationModel('bert', 'dbmdz/bert-base-turkish-uncased', use_cuda=False, num_labels=2,
                                args={'reprocess_input_data': True, 'overwrite_output_dir': True, 'num_train_epochs': 3,
                                      "train_batch_size": 64, "fp16": False, "output_dir": "bert_model"})

    model.train_model(train_df)

为什么会发生这种情况？

在Windows上，子进程将在启动时导入(即执行)主模块。您需要在主模块中插入if __name__ == '__main__':防护，以避免递归创建子流程。

引用自：https://stackoverflow.com/a/18205006/6025629