如何利用无服务器框架从Lambda中利用Huggingface深层学习容器进行推理

Question

这是ML New蜜蜂的一个问题:-)

我正在使用Serverless框架构建AWS StepFunction，其中一个步骤是使用HuggingFace深度学习容器(DLC)部署一个Sagemaker端点。

问题是，我无法让Lambda使用SageMaker (构建估计器)。

我所拥有的解决方案之一是使用SageMaker工作室手动启动端点，但我确实希望代码中包含所有内容。

下面是我要做的事情，让萨吉梅克工作

def installPack(package):
    import subprocess
    import sys
    subprocess.check_call([sys.executable, "-m", "pip", "install", package])

installPack('sagemaker')
from sagemaker.huggingface import HuggingFaceModel
import sagemaker 

role = sagemaker.get_execution_role()

# Hub Model configuration. https://huggingface.co/models
hub = {
        'HF_MODEL_ID':'distilbert-base-uncased-distilled-squad', # model_id from hf.co/models
        'HF_TASK':'question-answering' # NLP task you want to use for predictions
        }

# create Hugging Face Model Class
huggingface_model = sagemaker.HuggingFaceModel(
            env=hub,
            role=role, # iam role with permissions to create an Endpoint
            transformers_version="4.6", # transformers version used
            pytorch_version="1.7", # pytorch version used
            py_version="py36", # python version of the DLC

..........

我所犯的错误是

WARNING: The directory '/home/sbx_user1051/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.

(...然后有许多日志行，如收集pyparsing>=2.0.2下载、解析-2.4.7-py2.py3-no- are (67 KB)。

Downloading pox-0.3.0-py2.py3-none-any.whl (30 kB)
Collecting multiprocess>=0.70.12
Downloading multiprocess-0.70.12.2-py38-none-any.whl (128 kB)
Using legacy 'setup.py install' for sagemaker, since package 'wheel' is not installed.
Using legacy 'setup.py install' for protobuf3-to-dict, since package 'wheel' is not installed.
Installing collected packages: dill, zipp, pytz, pyparsing, protobuf, ppft, pox, numpy, multiprocess, smdebug-rulesconfig, protobuf3-to-dict, pathos, pandas, packaging, importlib-metadata, google-pasta, attrs, sagemaker
ERROR: Could not install packages due to an OSError: [Errno 30] Read-only file system: '/home/sbx_user1051'

Answer 1

找到答案：

1. API是使用studio部署的，如所描述的这里。
2. Lambda的推理代码如下：

    import os, io, boto3, json

    ENDPOINT_NAME = "huggingface-pytorch-inference-xxxxxxxxxxxxxxxxx"
    runtime= boto3.client('runtime.sagemaker')

    inputs = {'inputs': {
                        'question': 'What is used for inference?',
                        'context': 'My Name is Philipp and I live in Nuremberg. This model is used with sagemaker for inference.'}
    }
    payload = json.dumps(inputs, indent=2).encode('utf-8')

    print(f"payload: {type(payload)}, {payload}")
        
    response = runtime.invoke_endpoint(EndpointName=ENDPOINT_NAME,
                                        ContentType='text/csv',
                                        Body=payload)