在加载模型上运行SageMaker批处理转换失败

Question

我正在尝试使用HuggineFace类和精调模型以及自定义推理文件来运行批处理转换作业。这个工作在加载模型时失败了，但是我可以在本地加载它。我需要定制推理文件，因为我需要保持输入文件的原样，所以我必须从输入json文件中更改输入键。

以下是例外情况：

PredictionException(str(e), 400)
2022-05-08 16:49:45,499 [INFO ] W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - mms.service.PredictionException: Can't load config for '/.sagemaker/mms/models/model'. Make sure that:
2022-05-08 16:49:45,499 [INFO ] W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - 
2022-05-08 16:49:45,499 [INFO ] W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - - '/.sagemaker/mms/models/model' is a correct model identifier listed on 'https://huggingface.co/models'
2022-05-08 16:49:45,499 [INFO ] W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - 
2022-05-08 16:49:45,500 [INFO ] W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - - or '/.sagemaker/mms/models/model' is the correct path to a directory containing a config.json file
2022-05-08 16:49:45,500 [INFO ] W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - 
2022-05-08 16:49:45,500 [INFO ] W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle -  : 400

我正在运行脚本模式：

from sagemaker.huggingface.model import HuggingFaceModel

hub = {
   # 'HF_MODEL_ID':'cardiffnlp/twitter-roberta-base-sentiment',
    'HF_TASK':'text-classification',
    'INPUT_TEXTS': 'Description'
}

huggingface_model = HuggingFaceModel(model_data='../model/model.tar.gz',
                                     role=role,
                                     source_dir="../model/pytorch_model/code",
                                     transformers_version="4.6", 
                                     pytorch_version="1.7", 
                                     py_version="py36",
                                     entry_point="inference.py",
                                     env=hub
                                     )

batch_job = huggingface_model.transformer(
    instance_count=1,
    instance_type='ml.p3.2xlarge',
    output_path=output_s3_path, # we are using the same s3 path to save the output with the input
    strategy='SingleRecord',
    accept='application/json',
    assemble_with='Line'

)

batch_job.transform(
    data=s3_file_uri,
    content_type='application/json',
    split_type='Line',
    #input_filter='$[1:]',
    join_source='Input'

)

自定义inference.py

import json
import os
from transformers import pipeline
import torch
DEVICE = 'cuda' if torch.cuda.is_available() else 'cpu'


def model_fn(model_dir):
    model = pipeline(task=os.environ.get('HF_TASK', 'text-classification'), model=model_dir, tokenizer=model_dir)
    return model

def transform_fn(model, input_data, content_type, accept):
    input_data = json.loads(input_data)
    input_text = os.environ.get('INPUT_TEXTS', 'inputs')
    inputs = input_data.pop(input_text, None)
    parameters = input_data.pop("parameters", None)

    # pass inputs with all kwargs in data
    if parameters is not None:
        prediction = model(inputs, **parameters)
    else:
        prediction = model(inputs)
    return json.dumps(
        prediction,
        ensure_ascii=False,
        allow_nan=False,
        indent=None,
        separators=(",", ":"),
    )

Answer 1

我认为问题在于"model_data“参数。它应该指向一个S3对象(model.tar.gz)。

然后转换作业将从S3下载模型文件并加载它。

Answer 2

解决办法是将“任务”改为“情感分析”。

hub = {
   # 'HF_MODEL_ID':'cardiffnlp/twitter-roberta-base-sentiment',
    'HF_TASK':'sentiment-analysis',
    'INPUT_TEXTS': 'Description'
}