在Colab上安装cuDF & cuML与Rapids.ai版本0.11+

Question

我正在尝试用cuDF和cuML安装Rapids库到Colab会话，并执行符合以下示例的代码:从在Googe笔记本上安装RAPIDS库

!wget -nc https://raw.githubusercontent.com/rapidsai/notebooks-contrib/890b04ed8687da6e3a100c81f449ff6f7b559956/utils/rapids-colab.sh
!bash rapids-colab.sh

import sys, os

dist_package_index = sys.path.index("/usr/local/lib/python3.6/dist-packages")
sys.path = sys.path[:dist_package_index] + ["/usr/local/lib/python3.6/site-packages"] + sys.path[dist_package_index:]```
sys.path
if os.path.exists('update_pyarrow.py'): ## This file only exists if you're using RAPIDS version 0.11 or higher
  exec(open("update_pyarrow.py").read(), globals())

在安装过程中，我得到了以下错误：

  - cudf=0.11

Current channels:

  - https://conda.anaconda.org/rapidsai-nightly/label/xgboost/linux-64
  - https://conda.anaconda.org/rapidsai-nightly/label/xgboost/noarch
  - https://conda.anaconda.org/rapidsai-nightly/linux-64
  - https://conda.anaconda.org/rapidsai-nightly/noarch
  - https://conda.anaconda.org/nvidia/linux-64
  - https://conda.anaconda.org/nvidia/noarch
  - https://conda.anaconda.org/conda-forge/linux-64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://repo.anaconda.com/pkgs/main/linux-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/free/linux-64
  - https://repo.anaconda.com/pkgs/free/noarch
  - https://repo.anaconda.com/pkgs/r/linux-64
  - https://repo.anaconda.com/pkgs/r/noarch
  - https://repo.anaconda.com/pkgs/pro/linux-64
  - https://repo.anaconda.com/pkgs/pro/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

我试图分别安装cuDF和cuML

conda install -c rapidsai -c nvidia -c conda-forge \
    -c defaults cudf=0.12 python=3.6 cudatoolkit=10.0

但仍然收到错误：

ModuleNotFoundError Traceback (most recent call last)

<ipython-input-10-a95ca25217db> in <module>()
----> 1 import cudf
      2 import io, requests
      3 
      4 # download CSV file from GitHub
      5 url="https://github.com/plotly/datasets/raw/master/tips.csv"

ModuleNotFoundError: No module named 'cudf'

如何解决这个错误？

Answer 1

更新(12/21/2020)：要在网上跳入GPU驱动的RAPIDS笔记本，您可以使用 BlazingSQL (RAPIDS 0.15+)或继续使用Colabratory (仅有Rapids0.14)

更新(2/19/2020)：回到这个问题上，Colab正在工作@try。玩得开心!

如果你还有其他问题，请告诉我们。如果您需要更新您的个人Colab笔记本，请使用此脚本安装RAPIDS：

# Install RAPIDS
!git clone https://github.com/rapidsai/rapidsai-csp-utils.git
!bash rapidsai-csp-utils/colab/rapids-colab.sh

import sys, os

dist_package_index = sys.path.index('/usr/local/lib/python3.6/dist-packages')
sys.path = sys.path[:dist_package_index] + ['/usr/local/lib/python3.6/site-packages'] + sys.path[dist_package_index:]
sys.path
exec(open('rapidsai-csp-utils/colab/update_modules.py').read(), globals())

先前的答复：

我们正处于中间，或者正在将Colab脚本转换为一个新的回购程序。我们应该尽快更新我们所有的笔记本，并设法帮助其他人迁移。就像在24小时内，如果不是今天的爆炸处理，PST。

Answer 2

运行@TaureanDyerNV代码后，RAPIDS建议对以下代码进行更改。它需要15分钟才能运行。

!git clone https://github.com/rapidsai/rapidsai-csp-utils.git
!bash rapidsai-csp-utils/colab/rapids-colab.sh 0.19

import sys, os, shutil

sys.path.append('/usr/local/lib/python3.7/site-packages/')
os.environ['NUMBAPRO_NVVM'] = '/usr/local/cuda/nvvm/lib64/libnvvm.so'
os.environ['NUMBAPRO_LIBDEVICE'] = '/usr/local/cuda/nvvm/libdevice/'
os.environ['CONDA_PREFIX'] = '/usr/local'
for so in ['cudf', 'rmm', 'nccl', 'cuml', 'cugraph', 'xgboost', 'cuspatial']:
    fn = 'lib'+so+'.so'
    source_fn = '/usr/local/lib/'+fn
    dest_fn = '/usr/lib/'+fn
    if os.path.exists(source_fn):
        print(f'Copying {source_fn} to {dest_fn}')
        shutil.copyfile(source_fn, dest_fn)
if not os.path.exists('/usr/lib64'):
    os.makedirs('/usr/lib64')
for so_file in os.listdir('/usr/local/lib'):
    if 'libstdc' in so_file:
        shutil.copyfile('/usr/local/lib/'+so_file, '/usr/lib64/'+so_file)
        shutil.copyfile('/usr/local/lib/'+so_file, '/usr/lib/x86_64-linux-gnu/'+so_file)