Conda从requirements.txt创建未找到包

Question

我所尝试的

conda create --name ml --file ./requirements.txt

过去，我在另一台计算机上用conda list -e > requirements.txt创建了conda list -e > requirements.txt文件。requirements.txt：https://github.com/penguinsAreFunny/bugFinder-machineLearning/blob/master/requirements.txt

错误

PackagesNotFoundError:以下软件包无法从当前频道获得：

• protobuf==3.19.1=pypi_
• tensorboard-data-server==0.6.1=pypi_
• pygments==2.10.0=pypi_
• scikit-learn==1.0.1=pypi_
• tensorflow-estimator==2.4.0=pypi_
• flake8==4.0.1=pypi_
• nest-asyncio==1.5.1=pypi_0 .

现有渠道：

• https://conda.anaconda.org/conda-forge/win-64
• https://conda.anaconda.org/conda-forge/noarch
• https://repo.anaconda.com/pkgs/main/win-64
• https://repo.anaconda.com/pkgs/main/noarch
• https://repo.anaconda.com/pkgs/r/win-64
• https://repo.anaconda.com/pkgs/r/noarch
• https://repo.anaconda.com/pkgs/msys2/win-64
• https://repo.anaconda.com/pkgs/msys2/noarch
• https://conda.anaconda.org/pickle/win-64
• https://conda.anaconda.org/pickle/noarch
• https://conda.anaconda.org/nltk/win-64
• https://conda.anaconda.org/nltk/noarch

问题

为什么康达找不到频道里的包裹？我认为丢失的包裹应该在康达锻造，对吗？

旧版

conda 4.11.0

Answer 1

问题: PyPI不兼容

按照建议，包可能在Conda中，但是构建字符串"pypi_0“表示它们是在以前的环境中从PyPI安装的。conda list -e命令捕获这个信息，但是conda create命令无法处理它。

解决办法

备选案文1:来自Conda的源

最快的解决方法可能是编辑文件以删除这些包上的构建字符串规范。也就是说，就像：

## remove all PyPI references
sed -e 's/=pypi_0//' requirements.txt > reqs.nopip.txt

## try creating only from Conda packages
conda create -n m1 --file reqs.nopip.txt

然后，Conda将尝试将这些PyPI包规范视为Conda包。但是，这并不总是可靠的，因为在两个存储库中，有些包的名称不同。

备选方案2:出口YAML

或者，序列化到YAML可以同时处理捕获和重新安装Pip安装的包.因此，如果您仍然有旧的环境，请考虑使用：

conda env export > environment.yaml

它可以(在同一个平台上)用

conda env create -n m1 -f environment.yaml

选项3:将requirements.txt转换为YAML

如果环境不再存在，或者requirements.txt是由其他用户提供的，那么另一个选项是将文件转换为YAML格式。下面是一个AWK脚本来完成这个任务：

list_export_to_yaml.awk

#!/usr/bin/env awk -f
#' Author: Mervin Fansler
#' GitHub: @mfansler
#' License: MIT
#' 
#' Basic usage
#' $ conda list --export | awk -f list_export_to_yaml.awk
#' 
#' Omitting builds with 'no_builds'
#' $ conda list --export | awk -v no_builds=1 -f list_export_to_yaml.awk
#' 
#' Specifying channels with 'channels'
#' $ conda list --export | awk -v channels="conda-forge,defaults" -f list_export_to_yaml.awk

BEGIN {
  FS="=";
  if (channels) split(channels, channels_arr, ",");
  else channels_arr[0]="defaults";
}
{
  # skip header
  if ($1 ~ /^#/) next;

  if ($3 ~ /pypi/) {  # pypi packages
    pip=1;
    pypi[i++]="    - "$1"=="$2" ";
  } else {  # conda packages
    if ($1 ~ /pip/) pip=1;
    else {  # should we keep builds?
      if (no_builds) conda[j++]="  - "$1"="$2" ";
      else conda[j++]="  - "$1"="$2"="$3" ";
    }
  }
}
END {
  # emit channel info
  print "channels: ";
  for (k in channels_arr) print "  - "channels_arr[k]" ";

  # emit conda pkg info
  print "dependencies: ";
  for (j in conda) print conda[j];

  # emit PyPI pkg info
  if (pip) print "  - pip ";
  if (length(pypi) > 0) {
      print "  - pip: ";
      for (i in pypi) print pypi[i];
  }
}

对于OP的例子，我们得到：

$ wget -O requirements.txt 'https://github.com/penguinsAreFunny/bugFinder-machineLearning/raw/master/requirements.txt'
$ awk -f list_export_to_yaml.awk requirements.txt > bugfinder-ml.yaml

它的内容如下：

channels: 
  - defaults
dependencies: 
  - brotlipy=0.7.0=py38h294d835_1003
  - ca-certificates=2021.10.8=h5b45459_0
  - cffi=1.15.0=py38hd8c33c5_0
  - chardet=4.0.0=py38haa244fe_2
  - cryptography=35.0.0=py38hb7941b4_2
  - future=0.18.2=py38haa244fe_4
  - h2o=3.34.0.3=py38_0
  - openjdk=11.0.9.1=h57928b3_1
  - openssl=1.1.1l=h8ffe710_0
  - pycparser=2.20=pyh9f0ad1d_2
  - pyopenssl=21.0.0=pyhd8ed1ab_0
  - pysocks=1.7.1=py38haa244fe_4
  - python=3.8.12=h7840368_2_cpython
  - python_abi=3.8=2_cp38
  - requests=2.26.0=pyhd8ed1ab_0
  - setuptools=58.5.3=py38haa244fe_0
  - sqlite=3.36.0=h8ffe710_2
  - tabulate=0.8.9=pyhd8ed1ab_0
  - ucrt=10.0.20348.0=h57928b3_0
  - urllib3=1.26.7=pyhd8ed1ab_0
  - vc=14.2=hb210afc_5
  - vs2013_runtime=12.0.21005=1
  - vs2015_runtime=14.29.30037=h902a5da_5
  - wheel=0.37.0=pyhd8ed1ab_1
  - win_inet_pton=1.1.0=py38haa244fe_3
  - pip
  - pip:
    - absl-py==0.15.0
    - appdirs==1.4.4
    - astroid==2.7.3
    - astunparse==1.6.3
    - autopep8==1.6.0
    - backcall==0.2.0
    - backports-entry-points-selectable==1.1.0
    - black==21.4b0
    - cachetools==4.2.4
    - certifi==2021.10.8
    - cfgv==3.3.1
    - charset-normalizer==2.0.7
    - click==8.0.3
    - cycler==0.11.0
    - deap==1.3.1
    - debugpy==1.5.1
    - decorator==5.1.0
    - dill==0.3.4
    - distlib==0.3.3
    - entrypoints==0.3
    - filelock==3.3.2
    - flake8==4.0.1
    - flatbuffers==1.12
    - gast==0.3.3
    - google-auth==2.3.3
    - google-auth-oauthlib==0.4.6
    - google-pasta==0.2.0
    - grpcio==1.32.0
    - h5py==2.10.0
    - identify==2.3.3
    - idna==3.3
    - importlib-resources==5.4.0
    - ipykernel==6.5.0
    - ipython==7.29.0
    - isort==5.10.0
    - jedi==0.18.0
    - jinja2==3.0.2
    - joblib==1.1.0
    - jupyter-client==7.0.6
    - jupyter-core==4.9.1
    - keras-preprocessing==1.1.2
    - kiwisolver==1.3.2
    - markdown==3.3.4
    - markupsafe==2.0.1
    - matplotlib==3.4.3
    - matplotlib-inline==0.1.3
    - mypy==0.910
    - mypy-extensions==0.4.3
    - nest-asyncio==1.5.1
    - nodeenv==1.6.0
    - numpy==1.19.5
    - oauthlib==3.1.1
    - opt-einsum==3.3.0
    - pandas==1.3.4
    - parso==0.8.2
    - pathspec==0.9.0
    - pickleshare==0.7.5
    - pillow==8.4.0
    - platformdirs==2.4.0
    - pre-commit==2.15.0
    - prompt-toolkit==3.0.22
    - protobuf==3.19.1
    - pyasn1==0.4.8
    - pyasn1-modules==0.2.8
    - pycodestyle==2.8.0
    - pyflakes==2.4.0
    - pygments==2.10.0
    - pylint==2.10.2
    - pyparsing==3.0.4
    - python-dateutil==2.8.2
    - pytz==2021.3
    - pywin32==302
    - pyyaml==6.0
    - pyzmq==22.3.0
    - regex==2021.11.2
    - requests-oauthlib==1.3.0
    - rsa==4.7.2
    - scikit-learn==1.0.1
    - scipy==1.7.1
    - six==1.15.0
    - stopit==1.1.2
    - sweetviz==2.1.3
    - tensorboard==2.7.0
    - tensorboard-data-server==0.6.1
    - tensorboard-plugin-wit==1.8.0
    - tensorflow==2.4.4
    - tensorflow-estimator==2.4.0
    - termcolor==1.1.0
    - threadpoolctl==3.0.0
    - tornado==6.1
    - tpot==0.11.7
    - tqdm==4.62.3
    - traitlets==5.1.1
    - typing-extensions==3.7.4.3
    - update-checker==0.18.0
    - virtualenv==20.10.0
    - wcwidth==0.2.5
    - werkzeug==2.0.2
    - xgboost==1.5.0
    - zipp==3.6.0

注意，由于conda list --export不捕获通道信息，用户必须自行确定这一点。默认情况下，脚本插入一个defaults，但也提供一个参数(channels)，以逗号分隔的格式为YAML指定附加通道。例如。

awk -f list_export_to_yaml.awk -v channels='conda-forge,defaults' requirements.txt

会输出

channels:
  - conda-forge
  - defaults

在YAML中。

还有一个no_builds参数来抑制构建(即，仅限版本)。例如，

awk -f list_export_to_yaml.awk -v no_builds=1 requirements.txt