通过conda更新熊猫在萨格梅克上的速度非常慢

Question

我的工作环境中托管的默认python环境已经过时了，因此必须更新它们的conda环境。然而，这是非常慢(15-30分钟)，我想找到一个更快的方法来获得一个工作环境。

以下是我的最新情况：

!conda update pandas fsspec --yes

这提供了以下输出，关键问题是启动环境不一致(如何？)如failed with repodata from current_repodata.json, will retry with next repodata source. Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): done所示

产出：

Collecting package metadata (current_repodata.json): done
Solving environment: / 
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:

  - defaults/linux-64::pandas==1.0.1=py36h0573a6f_0
  - defaults/noarch::jupyterlab==1.2.6=pyhf63ae98_0
  - defaults/linux-64::scikit-learn==0.22.1=py36hd81dba3_0
  - defaults/linux-64::python-language-server==0.31.7=py36_0
  - defaults/linux-64::bkcharts==0.2=py36_0
  - defaults/linux-64::nb_conda==2.2.1=py36_0
  - defaults/noarch::numpydoc==0.9.2=py_0
  - defaults/linux-64::pytest-arraydiff==0.3=py36h39e3cac_0
  - defaults/linux-64::bottleneck==1.3.2=py36heb32a55_0
  - defaults/linux-64::pywavelets==1.1.1=py36h7b6447c_0
  - defaults/noarch::pytest-astropy==0.8.0=py_0
  - defaults/linux-64::numexpr==2.7.1=py36h423224d_0
  - defaults/noarch::anaconda-project==0.8.4=py_0
  - defaults/noarch::boto3==1.9.162=py_0
  - defaults/linux-64::s3transfer==0.2.1=py36_0
  - defaults/linux-64::nbconvert==5.6.1=py36_0
  - defaults/linux-64::h5py==2.10.0=py36h7918eee_0
  - defaults/linux-64::bokeh==1.4.0=py36_0
  - defaults/noarch::jupyterlab_server==1.0.6=py_0
  - defaults/linux-64::numpy-base==1.18.1=py36hde5b4d6_1
  - defaults/noarch::botocore==1.12.189=py_0
  - defaults/linux-64::jupyter==1.0.0=py36_7
  - defaults/linux-64::astropy==4.0=py36h7b6447c_0
  - defaults/linux-64::patsy==0.5.1=py36_0
  - defaults/linux-64::scikit-image==0.16.2=py36h0573a6f_0
  - defaults/linux-64::matplotlib-base==3.1.3=py36hef1b27d_0
  - defaults/linux-64::imageio==2.6.1=py36_0
  - defaults/linux-64::pytables==3.6.1=py36h71ec239_0
  - defaults/linux-64::nb_conda_kernels==2.2.4=py36_0
  - defaults/linux-64::mkl_fft==1.0.15=py36ha843d7b_0
  - defaults/linux-64::statsmodels==0.11.0=py36h7b6447c_0
  - defaults/linux-64::spyder==4.0.1=py36_0
  - defaults/noarch::seaborn==0.10.0=py_0
  - defaults/linux-64::requests==2.22.0=py36_1
  - defaults/linux-64::numba==0.48.0=py36h0573a6f_0
  - defaults/linux-64::scipy==1.4.1=py36h0b6359f_0
  - defaults/noarch::pytest-doctestplus==0.5.0=py_0
  - defaults/linux-64::mkl_random==1.1.0=py36hd6b4f25_0
  - defaults/noarch::dask==2.11.0=py_0
  - defaults/noarch::ipywidgets==7.5.1=py_0
  - defaults/linux-64::widgetsnbextension==3.5.1=py36_0
  - defaults/noarch::s3fs==0.4.2=py_0
  - defaults/linux-64::notebook==6.0.3=py36_0
  - defaults/linux-64::matplotlib==3.1.3=py36_0
  - defaults/linux-64::anaconda-client==1.7.2=py36_0
  - defaults/linux-64::numpy==1.18.1=py36h4f9e942_0
failed with repodata from current_repodata.json, will retry with next repodata source.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: | 
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:

  - defaults/noarch::jupyterlab==1.2.6=pyhf63ae98_0
  - defaults/linux-64::python-language-server==0.31.7=py36_0
  - defaults/linux-64::nb_conda==2.2.1=py36_0
  - defaults/noarch::numpydoc==0.9.2=py_0
  - defaults/noarch::anaconda-project==0.8.4=py_0
  - defaults/noarch::boto3==1.9.162=py_0
  - defaults/linux-64::s3transfer==0.2.1=py36_0
  - defaults/linux-64::nbconvert==5.6.1=py36_0
  - defaults/linux-64::bokeh==1.4.0=py36_0
  - defaults/noarch::jupyterlab_server==1.0.6=py_0
  - defaults/noarch::botocore==1.12.189=py_0
  - defaults/linux-64::jupyter==1.0.0=py36_7
  - defaults/linux-64::scikit-image==0.16.2=py36h0573a6f_0
  - defaults/linux-64::imageio==2.6.1=py36_0
  - defaults/linux-64::nb_conda_kernels==2.2.4=py36_0
  - defaults/linux-64::spyder==4.0.1=py36_0
  - defaults/linux-64::requests==2.22.0=py36_1
  - defaults/noarch::dask==2.11.0=py_0
  - defaults/noarch::ipywidgets==7.5.1=py_0
  - defaults/linux-64::widgetsnbextension==3.5.1=py36_0
  - defaults/noarch::s3fs==0.4.2=py_0
  - defaults/linux-64::notebook==6.0.3=py36_0
  - defaults/linux-64::anaconda-client==1.7.2=py36_0
done


==> WARNING: A newer version of conda exists. <==
  current version: 4.8.4
  latest version: 4.9.2

Please update conda by running

    $ conda update -n base conda



## Package Plan ##

  environment location: /home/ec2-user/anaconda3/envs/python3

  added / updated specs:
    - fsspec
    - pandas
    - s3fs


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    astroid-2.4.2              |   py36h9f0ad1d_1         297 KB  conda-forge
    certifi-2020.12.5          |   py36h5fab9bb_1         143 KB  conda-forge
    docutils-0.16              |   py36h5fab9bb_3         738 KB  conda-forge
    pandas-1.1.4               |   py36hd87012b_0        10.5 MB  conda-forge
    pillow-7.1.2               |   py36hb39fc2d_0         604 KB
    pylint-2.6.0               |   py36h9f0ad1d_1         446 KB  conda-forge
    sphinx-3.4.3               |     pyhd8ed1ab_0         1.5 MB  conda-forge
    toml-0.10.2                |     pyhd8ed1ab_0          18 KB  conda-forge
    urllib3-1.25.11            |             py_0          93 KB  conda-forge
    ------------------------------------------------------------
                                           Total:        14.3 MB

The following NEW packages will be INSTALLED:

  astroid            conda-forge/linux-64::astroid-2.4.2-py36h9f0ad1d_1
  bleach             conda-forge/noarch::bleach-3.2.1-pyh9f0ad1d_0
  brotlipy           conda-forge/linux-64::brotlipy-0.7.0-py36he6145b8_1001
  docutils           conda-forge/linux-64::docutils-0.16-py36h5fab9bb_3
  pillow             pkgs/main/linux-64::pillow-7.1.2-py36hb39fc2d_0
  pylint             conda-forge/linux-64::pylint-2.6.0-py36h9f0ad1d_1
  sphinx             conda-forge/noarch::sphinx-3.4.3-pyhd8ed1ab_0
  toml               conda-forge/noarch::toml-0.10.2-pyhd8ed1ab_0
  urllib3            conda-forge/noarch::urllib3-1.25.11-py_0

The following packages will be UPDATED:

  ca-certificates                      2020.11.8-ha878542_0 --> 2020.12.5-ha878542_0
  certifi                          2020.11.8-py36h5fab9bb_0 --> 2020.12.5-py36h5fab9bb_1
  fsspec                       pkgs/main::fsspec-0.6.2-py_0 --> conda-forge::fsspec-0.8.5-pyhd8ed1ab_0
  pandas             pkgs/main::pandas-1.0.1-py36h0573a6f_0 --> conda-forge::pandas-1.1.4-py36hd87012b_0



Downloading and Extracting Packages
pillow-7.1.2         | 604 KB    | ##################################### | 100% 
astroid-2.4.2        | 297 KB    | ##################################### | 100% 
pylint-2.6.0         | 446 KB    | ##################################### | 100% 
sphinx-3.4.3         | 1.5 MB    | ##################################### | 100% 
pandas-1.1.4         | 10.5 MB   | ##################################### | 100% 
docutils-0.16        | 738 KB    | ##################################### | 100% 
urllib3-1.25.11      | 93 KB     | ##################################### | 100% 
certifi-2020.12.5    | 143 KB    | ##################################### | 100% 
toml-0.10.2          | 18 KB     | ##################################### | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

很乐意接受任何建议，如何尽快得到一个蟒蛇笔记本与现代包装。

其他尝试的解决办法：

• 一个快速的pip install -U由于依赖问题而不能工作--笔记本中的本地环境将试图将熊猫指向过时的fsspec，它将崩溃。
• 遵循AWS文档，将我的conda请求添加到启动脚本中是不起作用的，因为启动脚本上有超时(我想是10分钟吧？)因此，一个15+分钟conda update进程只会确保sagemaker实例无法启动。

Answer 1

造成此问题的原因是conda进行依赖项检查。它试图找到与所有软件包兼容的包的版本，同时pip安装所需的包以及可能导致不一致的依赖项。[1]

解决这个问题有两个解决办法，

1. 使用所需的包创建一个自定义环境，并创建一个内核，以便从Sagmake记事本中使用。
2. 使用- pip install pandas==<version> --no-deps选项。您可能需要使用-U选项。

综上所述，我建议要么创建一个自定义环境，要么使用pip安装包及其与选项--no-deps的所有依赖关系。在笔记本运行时，您可能需要尝试这两种方法，然后应用于生命周期配置脚本。