pip未在conda环境中正确安装numba/llvmlite

Question

我创建了一个新的conda环境

user@machine:~/project$ conda create -n test-env -c numba python=3.5.2 llvmdev=3.8
Fetching package metadata ...........
Solving package specifications: .

Package plan for installation in environment /home/user/anaconda2/envs/test-env:

The following NEW packages will be INSTALLED:

    llvmdev:    3.8.1-7       numba
    openssl:    1.0.2k-0           
    pip:        9.0.1-py35_1       
    python:     3.5.2-0            
    readline:   6.2-2              
    setuptools: 27.2.0-py35_0      
    sqlite:     3.13.0-0           
    system:     5.8-2         numba
    tk:         8.5.18-0           
    wheel:      0.29.0-py35_0      
    xz:         5.2.2-1            
    zlib:       1.2.8-3            

Proceed ([y]/n)? y

#
# To activate this environment, use:
# > source activate test-env
#
# To deactivate this environment, use:
# > source deactivate test-env
#

然后激活它，并尝试专门使用pip (而不是conda)来安装llvmlite和numba，似乎成功了。(注意:我也尝试过--no-cache-dir，但它不会改变任何东西。)

user@machine:~/project$ source activate test-env
(test-env) user@machine:~/project$ pip install llvmlite==0.15 numba==0.30.1
Collecting llvmlite==0.15
Collecting numba==0.30.1
Collecting numpy (from numba==0.30.1)
  Using cached numpy-1.12.0-cp35-cp35m-manylinux1_x86_64.whl
Installing collected packages: llvmlite, numpy, numba
Successfully installed llvmlite-0.15.0 numba-0.30.1 numpy-1.12.0

但是库没有正确安装，

(test-env) user@machine:~/project$ python
Python 3.5.2 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:53:06) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numba
Traceback (most recent call last):
  File "/home/user/anaconda2/envs/test-env/lib/python3.5/site-packages/llvmlite/binding/ffi.py", line 42, in <module>
    lib = ctypes.CDLL(os.path.join(_lib_dir, _lib_name))
  File "/home/user/anaconda2/envs/test-env/lib/python3.5/ctypes/__init__.py", line 347, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /home/user/anaconda2/envs/test-env/lib/python3.5/site-packages/llvmlite/binding/libllvmlite.so: undefined symbol: _ZNKSt14error_category23default_error_conditionEi

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/anaconda2/envs/test-env/lib/python3.5/site-packages/numba/__init__.py", line 9, in <module>
    from . import config, errors, runtests, types
  File "/home/user/anaconda2/envs/test-env/lib/python3.5/site-packages/numba/config.py", line 11, in <module>
    import llvmlite.binding as ll
  File "/home/user/anaconda2/envs/test-env/lib/python3.5/site-packages/llvmlite/binding/__init__.py", line 6, in <module>
    from .dylib import *
  File "/home/user/anaconda2/envs/test-env/lib/python3.5/site-packages/llvmlite/binding/dylib.py", line 4, in <module>
    from . import ffi
  File "/home/user/anaconda2/envs/test-env/lib/python3.5/site-packages/llvmlite/binding/ffi.py", line 47, in <module>
    lib = ctypes.CDLL(_lib_name)
  File "/home/user/anaconda2/envs/test-env/lib/python3.5/ctypes/__init__.py", line 347, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libllvmlite.so: cannot open shared object file: No such file or directory

为什么从numba通道安装llvmdev的conda不能“正常工作”？

在我的用例中，我将进入一个具有pip样式的requirements.txt文件的项目，并且我需要从该文件创建conda环境。一些项目开发人员将使用venv+pip，一些将使用conda，并且它包含的一些包在任何anaconda通道中都找不到，因此pip安装是强制性的。我们不想在requirements.txt之外维护一个单独的envrionment.yaml，所以在conda环境中从requirements.txt安装是我的一部分限制。

看起来一切正常，除了numba/llvmlite的pip安装，它需要一个llvm 3.8+的系统依赖项。不过，作为conda环境的一部分，我想满足这一要求。

如何从conda环境中确保仅存在正确的llvmdev来安装numba和llvmlite？

Answer 1

安装llvmdev :从conda-通道安装llvmdev可以通过使用以下命令将conda-forge添加到您的通道中来实现：

conda config --add channels conda-forge

一旦启用了conda-forge通道，就可以使用以下命令安装llvmdev：

conda install llvmdev

可以使用以下命令列出您的平台上可用的llvmdev的所有版本：

conda search llvmdev --channel conda-forge

帮助:我从他们的GitHub源文件中克隆了相关文件

python setup.py安装

不使用conda获取更多信息，请使用pip

sudo pip install -U llvmlite sudo pip install -U numba

Answer 2

在理想情况下，所需的共享库应该在安装llvmlite之后出现。

(test-env) ~/condaexpts$ conda create -n test-env -c numba python=3.5.2 llvmdev=3.8
(test-env) ~/condaexpts$ source activate test-env
(test-env) ~/condaexpts$ pip install numpy==1.12.0 llvmlite==0.15 numba==0.30.1
(test-env) ~/condaexpts$ find $CONDA_PREFIX | grep libllvmlite
/home/ubuntu/condaexpts/m3/envs/test-env/lib/python3.5/site-packages/llvmlite/binding/libllvmlite.so

如果这个文件在test-env中不存在，那么在安装llvmlite时就出了问题。

另外，从另一方面考虑这种方法。您不需要将需求文件与conda环境文件分开。您可以在conda环境文件本身中同时拥有这两个依赖项：

(test-env) ~/condaexpts$ conda env export               
name: test-env
channels:
- !!python/unicode
  'numba'
- !!python/unicode
  'defaults'
dependencies:
- !!python/unicode
  'openssl=1.0.2k=0'
- !!python/unicode
  'pip=9.0.1=py35_1'
- !!python/unicode
  'python=3.5.2=0'
- !!python/unicode
  'readline=6.2=2'
- !!python/unicode
  'setuptools=27.2.0=py35_0'
- !!python/unicode
  'sqlite=3.13.0=0'
- !!python/unicode
  'tk=8.5.18=0'
- !!python/unicode
  'wheel=0.29.0=py35_0'
- !!python/unicode
  'xz=5.2.2=1'
- !!python/unicode
  'zlib=1.2.8=3'
- !!python/unicode
  'llvmdev=3.8.1=7'
- !!python/unicode
  'system=5.8=2'
- pip:
  - llvmlite==0.15.0
  - numba==0.30.1
  - numpy==1.12.0
prefix: !!python/unicode '/home/ubuntu/condaexpts/m3/envs/test-env'

其次，您可以从'numba‘conda通道本身安装所需的预先构建的二进制文件。通过以下方式：

(root) ~/condaexpts$ ./Miniconda3-latest-Linux-x86_64.sh -b -p m3
(root) ~/condaexpts$ source ./m3/bin/activate 
(root) ~/condaexpts$ conda create -n test-env -c numba python=3.5.2 llvmdev=3.8 numba=0.30.1 llvmlite=0.15.0
(root) ~/condaexpts$ source activate test-env
(test-env) ~/condaexpts$ python
Python 3.5.2 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:53:06) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numba
>>> numba.__version__
'0.30.1'

Answer 3

这就是我在使用Anaconda环境时的工作原理：

pip uninstall llvmlite
pip install -U --ignore-installed numba