Tensorflow-gpu错误: InvalidArgumentError:无法为操作“MatMul”分配设备

Question

我正在使用tensorflow网站上给出的测试代码在anaconda中用GPU测试tensorflow：

import tensorflow as tf
with tf.device('/device:GPU:0'):
  a = tf.constant([1,2,3,4,5,6],shape=[2,3],name='a')
  b = tf.constant([1,2,3,4,5,6],shape=[3,2],name='b')
  c = tf.matmul(a,b)
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
print(sess.run(c))

我创建了anaconda环境，并使用tensorflow+gpu安装了pip install tensorflow-gpu。IPython记事本用于执行上述代码，并不断获取错误。

InvalidArgumentError: Cannot assign a device for operation 'MatMul': Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available.
     [[Node: MatMul = MatMul[T=DT_INT32, transpose_a=false, transpose_b=false, _device="/device:GPU:0"](a, b)]]

MatMul操作符似乎无法加载到GPU上。我不知道为什么GPU设备没有支持的内核，因为cuda和cudNN安装得很好。否则，tensorflow消息显示gpu被识别：

name: GeForce GTX 1080 Ti
major: 6 minor: 1 memoryClockRate (GHz) 1.683
pciBusID 0000:02:00.0
Total memory: 10.91GiB
Free memory: 10.75GiB
2017-11-17 19:12:50.212054: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0x55a56f0c2420 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that.
2017-11-17 19:12:50.213035: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 1 with properties:
name: GeForce GTX 1080 Ti
major: 6 minor: 1 memoryClockRate (GHz) 1.683
pciBusID 0000:82:00.0
Total memory: 10.91GiB
Free memory: 10.75GiB
2017-11-17 19:12:50.213089: I tensorflow/core/common_runtime/gpu/gpu_device.cc:847] Peer access not supported between device ordinals 0 and 1
2017-11-17 19:12:50.213108: I tensorflow/core/common_runtime/gpu/gpu_device.cc:847] Peer access not supported between device ordinals 1 and 0
2017-11-17 19:12:50.213132: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0 1
2017-11-17 19:12:50.213148: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0:   Y N
2017-11-17 19:12:50.213156: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 1:   N Y
2017-11-17 19:12:50.213169: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0)
2017-11-17 19:12:50.213179: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:1) -> (device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:82:00.0)
Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0
/job:localhost/replica:0/task:0/gpu:1 -> device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:82:00.0
2017-11-17 19:12:50.471348: I tensorflow/core/common_runtime/direct_session.cc:300] Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0
/job:localhost/replica:0/task:0/gpu:1 -> device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:82:00.0

有两个gpus，它们都遇到了相同的问题。cuda和cudnn库安装正确，环境变量在anaconda中设置。cuda示例(deviceQuery)代码能够编译和运行，不会出错，并显示result = pass。否则，Matmul可以加载到CPU上并完成计算。程序中的变量a和b可以加载到GPU设备上。向tensorflow传达了以下信息：

2017-11-17 20:27:25.965655: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0)
2017-11-17 20:27:25.965665: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:1) -> (device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:82:00.0)
Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0
/job:localhost/replica:0/task:0/gpu:1 -> device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:82:00.0
2017-11-17 20:27:26.228395: I tensorflow/core/common_runtime/direct_session.cc:300] Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0
/job:localhost/replica:0/task:0/gpu:1 -> device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:82:00.0

MatMul: (MatMul): /job:localhost/replica:0/task:0/cpu:0
2017-11-17 20:27:26.229489: I tensorflow/core/common_runtime/simple_placer.cc:872] MatMul: (MatMul)/job:localhost/replica:0/task:0/cpu:0
b: (Const): /job:localhost/replica:0/task:0/gpu:0
2017-11-17 20:27:26.229512: I tensorflow/core/common_runtime/simple_placer.cc:872] b: (Const)/job:localhost/replica:0/task:0/gpu:0
a: (Const): /job:localhost/replica:0/task:0/gpu:0
2017-11-17 20:27:26.229526: I tensorflow/core/common_runtime/simple_placer.cc:872] a: (Const)/job:localhost/replica:0/task:0/gpu:0

我多次重新安装了nvidia驱动程序、cuda和anaconda，但从未解决过这个问题。如果有任何建议，那就太好了。

• 操作系统平台和发行版：LinuxUbuntu16.04
• 从TensorFlow安装自：二进制
• TensorFlow版本：1.3
• Python版本：2.7.14
• GCC/编译器版本(如果从源代码编译)：5.4.0
• NVIDIA驱动程序：384.98
• cuda /cuDNN版本：cuda 8.0/CUDN6.0
• GPU模型与存储器：Geforce 1080 GPU

Answer 1

您正在尝试在GPU上使用tf.int32 (DT_INT32)数据类型的多个张量。错误消息是说不支持在GPU上增加DT_INT32张量。

请注意，网站上的代码使用浮动张量(tf.float32) (假设您在讨论gpu中的代码)

改变：

a = tf.constant([1,2,3,4,5,6],shape=[2,3],name='a')

至：

a = tf.constant([1.,2.,3.,4.,5.,6.],shape=[2,3],name='a')

或者：

a = tf.constant([1,2,3,4,5,6],shape=[2,3],name='a',dtype=tf.float32)

同样，对于b来说，错误应该消失，因为当然存在支持float32张量矩阵乘法的内核。

希望这能有所帮助。