kubeflow dsl-编译- kfp_server_api.exceptions.ApiException：(400)

Question

请帮助理解是什么导致错误和如何解决。

实验()指的是实验() #6120 github问题，作者通过该问题查找解决问题。

我收到了开发人员的反馈(请参阅封闭式发行)。这是当前多用户模式的一个警告(参见文档)。现在，#5138支持这种用法。

然而，我不知道到底是什么原因，以及如何解决它。它看起来使用SDK客户端连接到Kubeflow管道给出了配置，但不确定我需要做什么。

复制步骤

1. 在远程实例上部署Minikube并安装kubectl连接。
2. 通过遵循用一个命令安装部署Kubeflow 1.5.0。
3. 验证连接并确认从本地膝上型计算机运行的吊舱。

$ kubectl get pods -n kubeflow
NAME                                                         READY   STATUS    RESTARTS   AGE
admission-webhook-deployment-7df7558c67-drdzw                1/1     Running   5          2d18h
cache-deployer-deployment-6f4bcc969-8kpm6                    2/2     Running   15         2d18h
cache-server-575d97c95-k7rv4                                 2/2     Running   10         2d18h
centraldashboard-5dd4f57bbd-gcxn5                            2/2     Running   10         2d18h
jupyter-web-app-deployment-5886974887-8c2cf                  1/1     Running   5          2d18h
katib-controller-58ddb4b856-mzq2l                            1/1     Running   46         2d18h
katib-db-manager-6df878f5b8-c9dhr                            1/1     Running   8          2d18h
katib-mysql-6dcb447c6f-lz5b8                                 1/1     Running   5          2d18h
katib-ui-f787b9d88-8h25n                                     1/1     Running   6          2d18h
kfserving-controller-manager-0                               2/2     Running   50         2d18h
kfserving-models-web-app-7884f597cf-m9n59                    2/2     Running   10         2d18h
kserve-models-web-app-5c64c8d8bb-bpdsb                       2/2     Running   10         2d18h
kubeflow-pipelines-profile-controller-84bcbdb899-669hr       1/1     Running   5          2d18h
metacontroller-0                                             1/1     Running   6          2d18h
metadata-envoy-deployment-7b847ff6c5-d2fjv                   1/1     Running   5          2d18h
metadata-grpc-deployment-6f6f7776c5-2vqp6                    2/2     Running   21         2d18h
metadata-writer-78fc7d5bb8-q8hfq                             2/2     Running   11         2d18h
minio-5b65df66c9-fttpm                                       2/2     Running   10         2d18h
ml-pipeline-75b5c59d7f-k7mm7                                 2/2     Running   59         2d18h
ml-pipeline-persistenceagent-87b6888c4-swv8k                 2/2     Running   10         2d18h
ml-pipeline-scheduledworkflow-665847bb9-4b5vr                2/2     Running   10         2d18h
ml-pipeline-ui-68cc764f66-892rz                              2/2     Running   14         2d18h
ml-pipeline-viewer-crd-68777557fb-6lq88                      2/2     Running   16         2d18h
ml-pipeline-visualizationserver-58ccb76855-qz2rc             2/2     Running   12         2d18h
mysql-f7b9b7dd4-2dpqv                                        2/2     Running   10         2d18h
notebook-controller-deployment-6c5f5d6cfc-mxmzw              2/2     Running   17         2d18h
profiles-deployment-5cdc5dc577-szhjk                         3/3     Running   61         2d18h
tensorboard-controller-controller-manager-5cbddb7fb5-xgq2v   3/3     Running   21         2d18h
tensorboards-web-app-deployment-7c5db448d7-t8xqp             1/1     Running   5          2d18h
training-operator-7b8cc9865d-qr8hm                           1/1     Running   7          2d18h
volumes-web-app-deployment-87484c848-qvsnc                   1/1     Running   5          2d18h
workflow-controller-6bf87db995-snfdn                         2/2     Running   20         2d18h

1. 在本地膝上型电脑中安装kubeflow SDK。

$ pip list | grep kfp
kfp                              1.8.12
kfp-pipeline-spec                0.1.15
kfp-server-api                   1.8.1

1. 按照使用SDK客户端连接到Kubeflow管道连接到kubeflow管道

kubectl端口-正向svc/ml-管道-ui 3000:80 -命名空间kubeflow\

1. 验证管道ui显示在文档中。

您可以通过在浏览器中访问http://localhost:3000来验证端口转发是否正常工作。如果端口转发工作正常，则会出现Kubeflow管道UI。

​

​

1. 运行代码。

import kfp
client = kfp.Client(host='http://localhost:3000', namespace='kubeflow')
print(client.list_experiments(namespace='kubeflow'))

搞错了。

Traceback (most recent call last):
  File "connect_kubeflow_pipeline.py", line 8, in <module>
    print(client.list_experiments(namespace='kubeflow'))
  File "/Users/1245095/venv/ml/lib/python3.7/site-packages/kfp/_client.py", line 540, in list_experiments
    filter=filter)
  File "/Users/1245095/venv/ml/lib/python3.7/site-packages/kfp_server_api/api/experiment_service_api.py", line 567, in list_experiment
    return self.list_experiment_with_http_info(**kwargs)  # noqa: E501
  File "/Users/1245095/venv/ml/lib/python3.7/site-packages/kfp_server_api/api/experiment_service_api.py", line 682, in list_experiment_with_http_info
    collection_formats=collection_formats)
  File "/Users/1245095/venv/ml/lib/python3.7/site-packages/kfp_server_api/api_client.py", line 369, in call_api
    _preload_content, _request_timeout, _host)
  File "/Users/1245095/venv/ml/lib/python3.7/site-packages/kfp_server_api/api_client.py", line 188, in __call_api
    raise e
  File "/Users/1245095/venv/ml/lib/python3.7/site-packages/kfp_server_api/api_client.py", line 185, in __call_api
    _request_timeout=_request_timeout)
  File "/Users/1245095/venv/ml/lib/python3.7/site-packages/kfp_server_api/api_client.py", line 393, in request
    headers=headers)
  File "/Users/1245095/venv/ml/lib/python3.7/site-packages/kfp_server_api/rest.py", line 234, in GET
    query_params=query_params)
  File "/Users/1245095/venv/ml/lib/python3.7/site-packages/kfp_server_api/rest.py", line 224, in request
    raise ApiException(http_resp=r)
kfp_server_api.exceptions.ApiException: (500)
Reason: Internal Server Error
HTTP response headers: HTTPHeaderDict({'X-Powered-By': 'Express', 'content-type': 'application/json', 'date': 'Tue, 24 May 2022 04:58:42 GMT', 'x-envoy-upstream-service-time': '2', 'server': 'envoy', 'connection': 'close', 'transfer-encoding': 'chunked'})
/*
* 提示：该行代码过长，系统自动注释不进行高亮。一键复制会移除系统注释 
* HTTP response body: {"error":"Internal error: Unauthenticated: Request header error: there is no user identity header.: Request header error: there is no user identity header.\nFailed to authorize with API resource references\ngithub.com/kubeflow/pipelines/backend/src/common/util.Wrap\n\t/go/src/github.com/kubeflow/pipelines/backend/src/common/util/error.go:279\ngithub.com/kubeflow/pipelines/backend/src/apiserver/server.(*ExperimentServer).canAccessExperiment\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/server/experiment_server.go:249\ngithub.com/kubeflow/pipelines/backend/src/apiserver/server.(*ExperimentServer).ListExperiment\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/server/experiment_server.go:148\ngithub.com/kubeflow/pipelines/backend/api/go_client._ExperimentService_ListExperiment_Handler.func1\n\t/go/src/github.com/kubeflow/pipelines/backend/api/go_client/experiment.pb.go:1089\nmain.apiServerInterceptor\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/interceptor.go:30\ngithub.com/kubeflow/pipelines/backend/api/go_client._ExperimentService_ListExperiment_Handler\n\t/go/src/github.com/kubeflow/pipelines/backend/api/go_client/experiment.pb.go:1091\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357\nFailed to authorize with API resource references\ngithub.com/kubeflow/pipelines/backend/src/common/util.Wrap\n\t/go/src/github.com/kubeflow/pipelines/backend/src/common/util/error.go:279\ngithub.com/kubeflow/pipelines/backend/src/apiserver/server.(*ExperimentServer).ListExperiment\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/server/experiment_server.go:150\ngithub.com/kubeflow/pipelines/backend/api/go_client._ExperimentService_ListExperiment_Handler.func1\n\t/go/src/github.com/kubeflow/pipelines/backend/api/go_client/experiment.pb.go:1089\nmain.apiServerInterceptor\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/interceptor.go:30\ngithub.com/kubeflow/pipelines/backend/api/go_client._ExperimentService_ListExperiment_Handler\n\t/go/src/github.com/kubeflow/pipelines/backend/api/go_client/experiment.pb.go:1091\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357","code":13,"message":"Internal error: Unauthenticated: Request header error: there is no user identity header.: Request header error: there is no user identity header.\nFailed to authorize with API resource references\ngithub.com/kubeflow/pipelines/backend/src/common/util.Wrap\n\t/go/src/github.com/kubeflow/pipelines/backend/src/common/util/error.go:279\ngithub.com/kubeflow/pipelines/backend/src/apiserver/server.(*ExperimentServer).canAccessExperiment\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/server/experiment_server.go:249\ngithub.com/kubeflow/pipelines/backend/src/apiserver/server.(*ExperimentServer).ListExperiment\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/server/experiment_server.go:148\ngithub.com/kubeflow/pipelines/backend/api/go_client._ExperimentService_ListExperiment_Handler.func1\n\t/go/src/github.com/kubeflow/pipelines/backend/api/go_client/experiment.pb.go:1089\nmain.apiServerInterceptor\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/interceptor.go:30\ngithub.com/kubeflow/pipelines/backend/api/go_client._ExperimentService_ListExperiment_Handler\n\t/go/src/github.com/kubeflow/pipelines/backend/api/go_client/experiment.pb.go:1091\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357\nFailed to authorize with API resource references\ngithub.com/kubeflow/pipelines/backend/src/common/util.Wrap\n\t/go/src/github.com/kubeflow/pipelines/backend/src/common/util/error.go:279\ngithub.com/kubeflow/pipelines/backend/src/apiserver/server.(*ExperimentServer).ListExperiment\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/server/experiment_server.go:150\ngithub.com/kubeflow/pipelines/backend/api/go_client._ExperimentService_ListExperiment_Handler.func1\n\t/go/src/github.com/kubeflow/pipelines/backend/api/go_client/experiment.pb.go:1089\nmain.apiServerInterceptor\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/interceptor.go:30\ngithub.com/kubeflow/pipelines/backend/api/go_client._ExperimentService_ListExperiment_Handler\n\t/go/src/github.com/kubeflow/pipelines/backend/api/go_client/experiment.pb.go:1091\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357","details":[{"@type":"type.googleapis.com/api.Error","error_message":"Internal error: Unauthenticated: Request header error: there is no user identity header.: Request header error: there is no user identity header.\nFailed to authorize with API resource references\ngithub.com/kubeflow/pipelines/backend/src/common/util.Wrap\n\t/go/src/github.com/kubeflow/pipelines/backend/src/common/util/error.go:279\ngithub.com/kubeflow/pipelines/backend/src/apiserver/server.(*ExperimentServer).canAccessExperiment\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/server/experiment_server.go:249\ngithub.com/kubeflow/pipelines/backend/src/apiserver/server.(*ExperimentServer).ListExperiment\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/server/experiment_server.go:148\ngithub.com/kubeflow/pipelines/backend/api/go_client._ExperimentService_ListExperiment_Handler.func1\n\t/go/src/github.com/kubeflow/pipelines/backend/api/go_client/experiment.pb.go:1089\nmain.apiServerInterceptor\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/interceptor.go:30\ngithub.com/kubeflow/pipelines/backend/api/go_client._ExperimentService_ListExperiment_Handler\n\t/go/src/github.com/kubeflow/pipelines/backend/api/go_client/experiment.pb.go:1091\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357\nFailed to authorize with API resource references\ngithub.com/kubeflow/pipelines/backend/src/common/util.Wrap\n\t/go/src/github.com/kubeflow/pipelines/backend/src/common/util/error.go:279\ngithub.com/kubeflow/pipelines/backend/src/apiserver/server.(*ExperimentServer).ListExperiment\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/server/experiment_server.go:150\ngithub.com/kubeflow/pipelines/backend/api/go_client._ExperimentService_ListExperiment_Handler.func1\n\t/go/src/github.com/kubeflow/pipelines/backend/api/go_client/experiment.pb.go:1089\nmain.apiServerInterceptor\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/interceptor.go:30\ngithub.com/kubeflow/pipelines/backend/api/go_client._ExperimentService_ListExperiment_Handler\n\t/go/src/github.com/kubeflow/pipelines/backend/api/go_client/experiment.pb.go:1091\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357","error_details":"Internal error: Unauthenticated: Request header error: there is no user identity header.: Request header error: there is no user identity header.\nFailed to authorize with API resource references\ngithub.com/kubeflow/pipelines/backend/src/common/util.Wrap\n\t/go/src/github.com/kubeflow/pipelines/backend/src/common/util/error.go:279\ngithub.com/kubeflow/pipelines/backend/src/apiserver/server.(*ExperimentServer).canAccessExperiment\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/server/experiment_server.go:249\ngithub.com/kubeflow/pipelines/backend/src/apiserver/server.(*ExperimentServer).ListExperiment\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/server/experiment_server.go:148\ngithub.com/kubeflow/pipelines/backend/api/go_client._ExperimentService_ListExperiment_Handler.func1\n\t/go/src/github.com/kubeflow/pipelines/backend/api/go_client/experiment.pb.go:1089\nmain.apiServerInterceptor\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/interceptor.go:30\ngithub.com/kubeflow/pipelines/backend/api/go_client._ExperimentService_ListExperiment_Handler\n\t/go/src/github.com/kubeflow/pipelines/backend/api/go_client/experiment.pb.go:1091\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357\nFailed to authorize with API resource references\ngithub.com/kubeflow/pipelines/backend/src/common/util.Wrap\n\t/go/src/github.com/kubeflow/pipelines/backend/src/common/util/error.go:279\ngithub.com/kubeflow/pipelines/backend/src/apiserver/server.(*ExperimentServer).ListExperiment\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/server/experiment_server.go:150\ngithub.com/kubeflow/pipelines/backend/api/go_client._ExperimentService_ListExperiment_Handler.func1\n\t/go/src/github.com/kubeflow/pipelines/backend/api/go_client/experiment.pb.go:1089\nmain.apiServerInterceptor\n\t/go/src/github.com/kubeflow/pipelines/backend/src/apiserver/interceptor.go:30\ngithub.com/kubeflow/pipelines/backend/api/go_client._ExperimentService_ListExperiment_Handler\n\t/go/src/github.com/kubeflow/pipelines/backend/api/go_client/experiment.pb.go:1091\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357"}]}
*/

Answer 1

找到了如何使用Dex进行编程身份验证？#140。通过按照那里提供的解决方案修改代码，它起了作用。

import requests
import kfp
import kfp.dsl as dsl
from kfp.components import create_component_from_func


# Does not work
#import kfp
#client = kfp.Client(host='http://localhost:3000', namespace='kubeflow')
#print(client.list_experiments(namespace='kubeflow'))


# --------------------------------------------------------------------------------
# https://github.com/kubeflow/kfctl/issues/140#issuecomment-719894529
# How to do programmatic authentication with Dex? #140
# --------------------------------------------------------------------------------
HOST = "http://localhost:8080/"
USERNAME = "user@example.com"
PASSWORD = "12341234"
NAMESPACE = "kubeflow-user-example-com"

session = requests.Session()
response = session.get(HOST)

headers = {
    "Content-Type": "application/x-www-form-urlencoded",
}

data = {"login": USERNAME, "password": PASSWORD}
session.post(response.url, headers=headers, data=data)
session_cookie = session.cookies.get_dict()["authservice_session"]

client = kfp.Client(
    host=f"{HOST}/pipeline",
    cookies=f"authservice_session={session_cookie}",
    namespace=NAMESPACE,
)

print(client.list_pipelines())


def add(a: float, b: float) -> float:
    '''Calculates sum of two arguments'''
    return a + b


add_op = create_component_from_func(
    add, output_component_file='add_component.yaml')


@dsl.pipeline(
    name='Addition pipeline',
    description='An example pipeline that performs addition calculations.'
)
def add_pipeline(
        a='1',
        b='7',
):
    # Passes a pipeline parameter and a constant value to the `add_op` factory
    # function.
    first_add_task = add_op(a, 4)
    # Passes an output reference from `first_add_task` and a pipeline parameter
    # to the `add_op` factory function. For operations with a single return
    # value, the output reference can be accessed as `task.output` or
    # `task.outputs['output_name']`.
    second_add_task = add_op(first_add_task.output, b)

# Specify argument values for your pipeline run.
arguments = {'a': '7', 'b': '8'}

# Create a pipeline run, using the client you initialized in a prior step.
#client.create_run_from_pipeline_func(add_pipeline, arguments=arguments)

kfp.compiler.Compiler().compile(
    pipeline_func=add_pipeline,
    package_path='pipeline.yaml')