在码头添加Intel MKL和MKL

Question

我有与Docker一起部署的ML代码(例如Numpy、Scipy、LightGBM、PyTorch)。我用Python和诗歌，用pip安装软件包。

我应该怎么做才能使用MKL和MKL？我知道最标准的方法是使用Anaconda，但我不能(大型企业，没有商业Anaconda许可证)。

pip install mkl足够了吗？

如何安装MKL，以便PyTorch使用它？

Answer 1

pip安装mkl是否足够？

不，它不会，请参阅numpy安装文档中的部分

pip安装的NumPy在PyPI上的车轮是用OpenBLAS构建的。OpenBLAS库包含在方向盘中。这使得轮子更大，如果用户也安装(例如) SciPy，他们现在将在磁盘上安装两个OpenBLAS副本。

因此，您需要从源构建numpy。

我知道最标准的方法是使用Anaconda，但我不能(大型企业，没有商业Anaconda许可证)。

你有没有考虑过使用迷你锻造和迷你车？IANAL，但我非常肯定，您只是不能在大规模的商业产品中使用ana-/miniconda发行版和anaconda通道，但是可以使用conda锻造仍然可以免费使用。。你应该能够建立所有的要求，你提到的从康达锻造。至少您可能会有一个更容易的时间从源代码编译py手电筒。

Answer 2

我试图将MKL添加到我的码头容器(基于debian)，阅读英特尔文档:我失败了。

然而，有一个码头形象OneAPI对接图像，随numpy (1.21，这是8个月前)和mkl作为默认的BLAS。下面是numpy在我的机器上返回的内容(一台带有i7-i10875H的笔记本电脑)

>>> import numpy as np
>>> np.__config__.show()
blas_mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/opt/intel/oneapi/intelpython/latest/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/opt/intel/oneapi/intelpython/latest/include']
blas_opt_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/opt/intel/oneapi/intelpython/latest/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/opt/intel/oneapi/intelpython/latest/include']
lapack_mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/opt/intel/oneapi/intelpython/latest/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/opt/intel/oneapi/intelpython/latest/include']
lapack_opt_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/opt/intel/oneapi/intelpython/latest/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/opt/intel/oneapi/intelpython/latest/include']
Supported SIMD extensions in this NumPy install:
    baseline = SSE,SSE2,SSE3,SSSE3,SSE41,POPCNT,SSE42
    found = 
    not found = AVX512_ICL

然而，我尝试使用anaconda和基本的坞映像，令我惊讶的是，anaconda虚拟env使用了CBLAS，我的坞映像使用了Openblas BLAS。

我没有执行基准测试，但是由于mkl实现使用了除AVX512_ICL之外的所有指令集体系结构，所以我希望它会更快。

蟒蛇

我还惊讶地在我的anaconda环境中测试了这一点，令我惊讶的是，的blas不是mkl。

$ conda create -n test numpy --yes
[...]
$ conda activate test

>>> import numpy as np
>>> np.__config__.show()
blas_info:
    libraries = ['cblas', 'blas', 'cblas', 'blas']
    library_dirs = ['/home/adrienpacifico/anaconda3/envs/test/lib']
    include_dirs = ['/home/adrienpacifico/anaconda3/envs/test/include']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
blas_opt_info:
    define_macros = [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]
    libraries = ['cblas', 'blas', 'cblas', 'blas']
    library_dirs = ['/home/adrienpacifico/anaconda3/envs/test/lib']
    include_dirs = ['/home/adrienpacifico/anaconda3/envs/test/include']
    language = c
lapack_info:
    libraries = ['lapack', 'blas', 'lapack', 'blas']
    library_dirs = ['/home/adrienpacifico/anaconda3/envs/test/lib']
    language = f77
lapack_opt_info:
    libraries = ['lapack', 'blas', 'lapack', 'blas', 'cblas', 'blas', 'cblas', 'blas']
    library_dirs = ['/home/adrienpacifico/anaconda3/envs/test/lib']
    language = c
    define_macros = [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]
    include_dirs = ['/home/adrienpacifico/anaconda3/envs/test/include']
Supported SIMD extensions in this NumPy install:
    baseline = SSE,SSE2,SSE3
    found = SSSE3,SSE41,POPCNT,SSE42,AVX,F16C,FMA3,AVX2
    not found = AVX512F,AVX512CD,AVX512_KNL,AVX512_KNM,AVX512_SKX,AVX512_CLX,AVX512_CNL,AVX512_ICL

我的base环境使用openblas。

基于python图像的我的停靠者图像-> Openblas

Dockerfile：

FROM python:3.10
ENV SHELL=/bin/bash
RUN apt-get update && \
    apt-get install build-essential
RUN apt-get install -y sudo libaio1 wget unzip htop
RUN pip install numpy

openblas64__info:
    libraries = ['openblas64_', 'openblas64_']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOL_SUFFIX', '64_'), ('HAVE_BLAS_ILP64', None)]
    runtime_library_dirs = ['/usr/local/lib']
blas_ilp64_opt_info:
    libraries = ['openblas64_', 'openblas64_']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOL_SUFFIX', '64_'), ('HAVE_BLAS_ILP64', None)]
    runtime_library_dirs = ['/usr/local/lib']
openblas64__lapack_info:
    libraries = ['openblas64_', 'openblas64_']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOL_SUFFIX', '64_'), ('HAVE_BLAS_ILP64', None), ('HAVE_LAPACKE', None)]
    runtime_library_dirs = ['/usr/local/lib']
lapack_ilp64_opt_info:
    libraries = ['openblas64_', 'openblas64_']
    library_dirs = ['/usr/local/lib']
    language = c
    define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOL_SUFFIX', '64_'), ('HAVE_BLAS_ILP64', None), ('HAVE_LAPACKE', None)]
    runtime_library_dirs = ['/usr/local/lib']
Supported SIMD extensions in this NumPy install:
    baseline = SSE,SSE2,SSE3
    found = SSSE3,SSE41,POPCNT,SSE42,AVX,F16C,FMA3,AVX2
    not found = AVX512F,AVX512CD,AVX512_KNL,AVX512_KNM,AVX512_SKX,AVX512_CLX,AVX512_CNL,AVX512_ICL