用寄存器模型导入天蓝色自动环境

Question

我试图在我的代码中使用一个天蓝色自动生成的模型，用于补充射线测试(所以我暂时不想部署它。我使用以下代码加载模型：

import joblib
from azureml.core.model import Model
from azureml.core import Workspace
import os
import pickle

ws = Workspace.from_config()
path=Model.get_model_path('my_automl_model', version=1, _workspace=ws)
model_path = os.path.join(path, 'model.pkl')
model = joblib.load(model_path)

运行此代码时，将获得以下错误：

/anaconda/envs/azureml_py38/lib/python3.8/site-packages/spacy/util.py:873: UserWarning: [W094] Model 'en_core_web_sm' (2.1.0) specifies an under-constrained spaCy version requirement: >=2.1.0. This can lead to compatibility problems with older versions, or as new spaCy versions are released, because the model may say it's compatible when it's not. Consider changing the "spacy_version" in your meta.json to a version range, with a lower and upper pin. For example: >=3.4.0,<3.5.0
  warnings.warn(warn_msg)
Output exceeds the size limit. Open the full output data in a text editor
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
/home/azureuser/....ipynb Cellule 4 in <cell line: 11>()
      9 model_path = os.path.join(path, 'model.pkl')
     10 print(os.stat(model_path))
---> 11 model = joblib.load(model_path)

File /anaconda/envs/azureml_py38/lib/python3.8/site-packages/joblib/numpy_pickle.py:605, in load(filename, mmap_mode)
    599             if isinstance(fobj, _basestring):
    600                 # if the returned file object is a string, this means we
    601                 # try to load a pickle file generated with an version of
    602                 # Joblib so we load it with joblib compatibility function.
    603                 return load_compatibility(fobj)
--> 605             obj = _unpickle(fobj, filename, mmap_mode)
    607 return obj

File /anaconda/envs/azureml_py38/lib/python3.8/site-packages/joblib/numpy_pickle.py:529, in _unpickle(fobj, filename, mmap_mode)
    527 obj = None
    528 try:
--> 529     obj = unpickler.load()
    530     if unpickler.compat_mode:
    531         warnings.warn("The file '%s' has been generated with a "
    532                       "joblib version less than 0.10. "
    533                       "Please regenerate this pickle file."
    534                       % filename,
...
    679     return config.from_disk(
    680         config_path, overrides=overrides, interpolate=interpolate
    681     )

OSError: [E053] Could not read config file from /anaconda/envs/azureml_py38/lib/python3.8/site-packages/en_core_web_sm/en_core_web_sm-2.1.0/config.cfg

在automl model文件夹中，我有以下文件：

​

​

我认为这个错误是因为我在Pyton3.8.5上运行，当模型在python3.7.9上生成时(如conda.yaml文件所示)：

channels:
- anaconda
- conda-forge
- pytorch
dependencies:
- python=3.7.9
- pip=20.2.4
- pip:
  - mlflow
  - https://aka.ms/automl-resources/packages/en_core_web_sm-2.1.0.tar.gz
  - regex==2022.8.17
  - fsspec==2022.8.2
  - certifi==2022.6.15
  - pytz==2022.2.1
  - setuptools==63.4.1
  - azureml-dataprep-native==38.0.0
  - cryptography==37.0.4
  - pyzmq==23.2.1
  - attrs==22.1.0
  - pyopenssl==22.0.0
  - contextlib2==21.6.0
  - packaging==21.3
  - azure-mgmt-resource==21.1.0
  - pip==20.2.4
  - gunicorn==20.1.0
  - azure-mgmt-storage==20.0.0
  - azure-storage-blob==12.13.0
  - azure-storage-queue==12.4.0
  - azure-mgmt-keyvault==10.1.0
  - humanfriendly==10.0
  - azure-mgmt-containerregistry==10.0.0
  - pillow==9.2.0
  - ipython==7.34.0
  - jupyter-client==7.3.5
  - click==7.1.2
  - thinc==7.0.8
  - ipykernel==6.6.0
  - tornado==6.2
  - pyarrow==6.0.0
  - pyyaml==6.0
  - importlib-resources==5.9.0
  - psutil==5.9.0
  - traitlets==5.3.0
  - cachetools==5.2.0
  - decorator==5.1.1
  - docker==5.0.3
  - smmap==5.0.0
  - py-cpuinfo==5.0.0
  - tqdm==4.64.1
  - fonttools==4.37.1
  - jsonschema==4.15.0
  - importlib-metadata==4.12.0
  - jupyter-core==4.11.1
  - rsa==4.9
  - pexpect==4.8.0
  - typing-extensions==4.3.0
  - azureml-dataprep==4.2.2
  - ephem==4.1.3
  - gitdb==4.0.9
  - bcrypt==4.0.0
  - protobuf==3.20.1
  - gensim==3.8.3
  - zipp==3.8.1
  - configparser==3.7.4
  - h5py==3.7.0
  - matplotlib==3.5.3
  - secretstorage==3.3.3
  - idna==3.3
  - lightgbm==3.2.1
  - oauthlib==3.2.0
  - gitpython==3.1.27
  - dotnetcore2==3.1.23
  - mpi4py==3.1.3
  - prompt-toolkit==3.0.31
  - flask-cors==3.0.10
  - pyparsing==3.0.9
  - boto==2.49.0
  - distributed==2.30.1
  - dask==2.30.0
  - requests==2.28.1
  - pycparser==2.21
  - pystan==2.19.1.1
  - semver==2.13.0
  - pygments==2.13.0
  - jinja2==2.11.2
  - google-auth==2.11.0
  - paramiko==2.11.0
  - google-api-core==2.10.0
  - python-dateutil==2.8.2
  - azureml-dataprep-rslex==2.8.1
  - portalocker==2.5.1
  - liac-arff==2.5.0
  - networkx==2.5
  - bokeh==2.4.3
  - sortedcontainers==2.4.0
  - pyjwt==2.4.0
  - convertdate==2.3.2
  - mkl-service==2.3.0
  - werkzeug==2.2.2
  - cloudpickle==2.2.0
  - jsonpickle==2.2.0
  - zict==2.2.0
  - spacy==2.1.8
  - flask==2.1.3
  - itsdangerous==2.1.2
  - charset-normalizer==2.1.1
  - en-core-web-sm==2.1.0
  - flatbuffers==2.0.7
  - cymem==2.0.6
  - preshed==2.0.1
  - markupsafe==2.0.1
  - azure-mgmt-authorization==2.0.0
  - argcomplete==2.0.0
  - googleapis-common-protos==1.56.4
  - azureml-automl-runtime==1.45.0
  - azureml-core==1.45.0
  - azureml-train-automl-client==1.45.0
  - azureml-dataset-runtime==1.45.0
  - azureml-training-tabular==1.45.0
  - azureml-train-restclients-hyperdrive==1.45.0
  - azureml-automl-core==1.45.0
  - azureml-pipeline-core==1.45.0
  - azureml-train-core==1.45.0
  - azureml-mlflow==1.45.0
  - azureml-responsibleai==1.45.0
  - azureml-telemetry==1.45.0
  - mlflow-skinny==1.28.0
  - urllib3==1.26.12
  - azure-core==1.25.1
  - numpy==1.21.6
  - botocore==1.18.18
  - msal==1.18.0
  - six==1.16.0
  - boto3==1.15.18
  - cffi==1.15.1
  - wrapt==1.12.1
  - onnx==1.12.0
  - onnxruntime==1.11.1
  - sympy==1.10.1
  - azure-identity==1.10.0
  - smart-open==1.9.0
  - pkginfo==1.8.3
  - nimbusml==1.8.0
  - pmdarima==1.7.1
  - pysocks==1.7.1
  - sklearn-pandas==1.7.0
  - distro==1.7.0
  - tblib==1.7.0
  - debugpy==1.6.3
  - onnxconverter-common==1.6.0
  - keras2onnx==1.6.0
  - nest-asyncio==1.5.5
  - scipy==1.5.3
  - pynacl==1.5.0
  - skl2onnx==1.4.9
  - kiwisolver==1.4.4
  - pydot==1.4.2
  - onnxmltools==1.4.1
  - websocket-client==1.4.1
  - torch==1.4.0
  - pkgutil-resolve-name==1.3.10
  - xgboost==1.3.3
  - azure-mgmt-core==1.3.2
  - requests-oauthlib==1.3.1
  - partd==1.3.0
  - adal==1.2.7
  - mpmath==1.2.1
  - setuptools-git==1.2
  - azure-common==1.1.28
  - opencensus-ext-azure==1.1.7
  - pandas==1.1.5
  - termcolor==1.1.0
  - joblib==1.1.0
  - murmurhash==1.0.8
  - srsly==1.0.5
  - msgpack==1.0.4
  - heapdict==1.0.1
  - jmespath==1.0.1
  - azureml-model-management-sdk==1.0.1b6.post1
  - backports.weakref==1.0.post1
  - pytorch-transformers==1.0.0
  - backports.tempfile==1.0
  - locket==1.0.0
  - msal-extensions==1.0.0
  - azure-graphrbac==0.61.1
  - numba==0.55.2
  - shap==0.39.0
  - llvmlite==0.38.1
  - wheel==0.37.1
  - cython==0.29.32
  - interpret-community==0.26.0
  - scikit-learn==0.22.1
  - responsibleai==0.21.0
  - horovod==0.19.4
  - jedi==0.18.1
  - pyrsistent==0.18.1
  - databricks-cli==0.17.3
  - econml==0.13.1
  - sparse==0.13.0
  - toolz==0.12.0
  - applicationinsights==0.11.10
  - statsmodels==0.11.1
  - cycler==0.11.0
  - opencensus==0.11.0
  - holidays==0.10.3
  - pathspec==0.10.1
  - wasabi==0.10.1
  - plac==0.9.6
  - cmdstanpy==0.9.5
  - knack==0.9.0
  - tabulate==0.8.10
  - parso==0.8.3
  - jeepney==0.8.0
  - dice-ml==0.8
  - azureml-inference-server-http==0.7.5
  - pickleshare==0.7.5
  - fbprophet==0.7.1
  - dowhy==0.7.1
  - msrest==0.7.1
  - fairlearn==0.7.0
  - ptyprocess==0.7.0
  - msrestazure==0.6.4
  - isodate==0.6.1
  - dataclasses==0.6
  - pymeeus==0.5.11
  - patsy==0.5.2
  - ndg-httpsclient==0.5.1
  - pyasn1==0.4.8
  - sqlparse==0.4.2
  - entrypoints==0.4
  - fire==0.4.0
  - erroranalysis==0.3.9
  - s3transfer==0.3.7
  - dill==0.3.5.1
  - pyasn1-modules==0.2.8
  - interpret-core==0.2.7
  - wcwidth==0.2.5
  - blis==0.2.4
  - korean-lunar-calendar==0.2.1
  - backcall==0.2.0
  - raiutils==0.2.0
  - ml-wrappers==0.2.0
  - json-logging-py==0.2
  - sentencepiece==0.1.97
  - matplotlib-inline==0.1.6
  - opencensus-context==0.1.3
  - lunarcalendar==0.0.9
  - slicer==0.0.7
  - azureml-train-automl-runtime==1.45.0
  - inference-schema
  - azureml-interpret==1.45.0
  - azureml-defaults==1.45.0
- numpy~=1.21.6
- scikit-learn==0.22.1
- pandas~=1.1.5
- holidays==0.10.3
- setuptools-git
- pytorch=1.4.0
- cudatoolkit=10.0.130
- psutil>5.0.0,<6.0.0
- numpy==1.21.6
- pandas==1.1.5
- py-xgboost<=1.3.3
- fbprophet==0.7.1
- psutil==5.9.0
name: project_environment

现在我的问题是:用它加载模型环境的最佳方法是什么，而不是使用另一个环境？

谢谢。

Answer 1

在模型生成中自动使用的python版本是这里的主要问题。检查conda.yml文件中模型的当前工作版本，如下所述。

​

​

目前起作用的模式是3.7.9。但在笔记本电脑里，版本会有所不同。

​

​

放弃笔记本的版本或在conda.yml文件中升级python版本并注册模型。

import joblib
from azureml.core.model import Model
from azureml.core import Workspace
import os
import pickle

ws = Workspace.from_config()
path=Model.get_model_path('my_automl_model', version=1, _workspace=ws)
model_path = os.path.join(path, 'model.pkl')
model = joblib.load(model_path)

以上代码将随着python运行版本的更改而运行。由于模型已经创建，因此需要在笔记本中取消默认版本的python。