如何将BertforSequenceClassification模型的权重加载到BertforTokenClassification模型中？

Question

最初，我使用文本分类数据集对基于BERT的模型进行了微调，为此我使用了BertforSequenceClassification类。

from transformers import BertForSequenceClassification, AdamW, BertConfig

# Load BertForSequenceClassification, the pretrained BERT model with a single 
# linear classification layer on top. 
model = BertForSequenceClassification.from_pretrained(
    "bert-base-uncased", # Use the 12-layer BERT model, with an uncased vocab.
    num_labels = 2, # The number of output labels--2 for binary classification.
                    # You can increase this for multi-class tasks.   
    output_attentions = False, # Whether the model returns attentions weights.
    output_hidden_states = False, # Whether the model returns all hidden-states.
)

现在，我想使用这个经过微调的BERT模型权重来识别命名实体，为此我必须使用BertforTokenClassification类。我不知道如何将微调的BERT模型权重加载到使用BertforTokenClassification创建的新模型中。

预先致谢..

Answer 1

您可以从第一个模型中的bert获得权重，然后加载到第二个模型中的bert中：

new_model = BertForTokenClassification(config=config)
new_model.bert.load_state_dict(model.bert.state_dict())

Answer 2

这对我很有效

new_model = BertForTokenClassification.from_pretrained('/config path')
new_model.bert.load_state_dict(model.bert.state_dict())