AWS MLOps -使用SageMaker管道将新版本的模型部署到现有端点

Question

我在使用SageMaker管道进行MLOps时遇到了问题，我遵循了这个例子，他们似乎只有一次部署的例子，我的项目需要每周重新训练模型，如果再对模型进行重新培训和部署，我也会检查AWS文档，我找不到任何例子来更新运行端点的模型版本，我的解决方法是再次删除和重新创建端点，但是它会导致停机时间。

有无建议的解决方案，以更新新的模式而不停机？

下面是我的代码：

调度程序代码：

    sklearn_preprocessor = SKLearn(
                entry_point=script_path,
                role=role,
                framework_version="0.23-1",
                base_job_name="test-model",
                instance_type=env.TRAIN_INSTANCE_TYPE,
                sagemaker_session=sagemaker_session,
            )
    
            train_step = TrainingStep(
                name="TrainingStep",
                display_name="Traning Step",
                estimator=sklearn_preprocessor,
                inputs={"train": train_input},
            )
    
            model = Model(
                image_uri=sklearn_preprocessor.image_uri,
                model_data=train_step.properties.ModelArtifacts.S3ModelArtifacts,  # pylint: disable=no-member
                sagemaker_session=sagemaker_session,
                role=role,
                name="test-model",
            )
    
            step_register_pipeline_model = RegisterModel(
                name="RegisterModelStep",
                display_name="Register Model Step",
                model=model,
                content_types=["text/csv"],
                response_types=["text/csv"],
                inference_instances=[env.TRAIN_INSTANCE_TYPE],
                transform_instances=[env.INFERENCE_INSTANCE_TYPE],
                model_package_group_name="test-model-group",
                approval_status="Approved",
            )
    
            inputs = CreateModelInput(
                instance_type=env.INFERENCE_INSTANCE_TYPE,
            )
    
            step_create_model = CreateModelStep(
                name="CreateModelStep", display_name="Create Model Step", model=model, inputs=inputs
            )
    
            lambda_fn = Lambda(
                function_arn="arn:aws:lambda:ap-southeast-1:xxx:function:model-deployment"
            )
    
            step_deploy_lambda = LambdaStep(
                name="DeploymentStep",
                display_name="Deployment Step",
                lambda_func=lambda_fn,
                inputs={
                    "model_name": "test-model",
                    "endpoint_config_name": "test-model",
                    "endpoint_name": "test-endpoint",
                    "model_package_arn": step_register_pipeline_model.steps[
                        0
                    ].properties.ModelPackageArn,
                    "role": "arn:aws:iam::xxx:role/service-role/xxxx-role"
                },
            )
    
            pipeline = Pipeline(
                name="sagemaker-pipeline",
                steps=[train_step, step_register_pipeline_model, step_deploy_lambda],
            )
            pipeline.upsert(
                role_arn="arn:aws:iam::xxx:role/service-role/xxxx-role"
            )
            pipeline.start()

用于部署的lambda函数：

import json
import boto3

def lambda_handler(event, context):
    model_name = event["model_name"]
    model_package_arn = event["model_package_arn"]
    endpoint_config_name = event["endpoint_config_name"]
    endpoint_name = event["endpoint_name"]
    role = event["role"]
    
    sm_client = boto3.client("sagemaker")
    container = {"ModelPackageName": model_package_arn}
    create_model_respose = sm_client.create_model(ModelName=model_name, ExecutionRoleArn=role, Containers=[container] )

    create_endpoint_config_response = sm_client.create_endpoint_config(
        EndpointConfigName=endpoint_config_name,
        ProductionVariants=[
            {
                "InstanceType": "ml.m5.xlarge",
                "InitialInstanceCount": 1,
                "ModelName": model_name,
                "VariantName": "AllTraffic",
            }
        ]
    )

    create_endpoint_response = sm_client.create_endpoint(EndpointName=endpoint_name, EndpointConfigName=endpoint_config_name)


    return {
        'statusCode': 200,
        'body': json.dumps('Done!')
    }

Answer 1

您可以将Lambda代码更新为"update_endpoint“，而不是创建它。您可以在代码中添加检查，以查看端点是否已经存在，如果已经存在，则调用update端点而不是create。