Tensorflow:设备序号之间不支持对等访问

Question

如果我有Peer access not supported between device ordinals，还可以在某种多gpu设置中运行培训吗？(据我理解，GPU“不连接”)，例如，在GPU上分别计算每个批处理，然后在CPU上合并，因为我知道这是Caffe后端以数字形式进行的“批积累”工作。

原始产出：

2017-05-10 15:27:54.360688: I tensorflow/core/common_runtime/gpu/gpu_device.cc:779] Peer access not supported between device ordinals 0 and 1
2017-05-10 15:27:54.360949: I tensorflow/core/common_runtime/gpu/gpu_device.cc:779] Peer access not supported between device ordinals 0 and 2
2017-05-10 15:27:54.361504: I tensorflow/core/common_runtime/gpu/gpu_device.cc:779] Peer access not supported between device ordinals 0 and 3
2017-05-10 15:27:54.361738: I tensorflow/core/common_runtime/gpu/gpu_device.cc:779] Peer access not supported between device ordinals 1 and 0
2017-05-10 15:27:54.361892: I tensorflow/core/common_runtime/gpu/gpu_device.cc:779] Peer access not supported between device ordinals 1 and 2
2017-05-10 15:27:54.362065: I tensorflow/core/common_runtime/gpu/gpu_device.cc:779] Peer access not supported between device ordinals 1 and 3
2017-05-10 15:27:54.362263: I tensorflow/core/common_runtime/gpu/gpu_device.cc:779] Peer access not supported between device ordinals 2 and 0
2017-05-10 15:27:54.362485: I tensorflow/core/common_runtime/gpu/gpu_device.cc:779] Peer access not supported between device ordinals 2 and 1
2017-05-10 15:27:54.362693: I tensorflow/core/common_runtime/gpu/gpu_device.cc:779] Peer access not supported between device ordinals 2 and 3
2017-05-10 15:27:54.362885: I tensorflow/core/common_runtime/gpu/gpu_device.cc:779] Peer access not supported between device ordinals 3 and 0
2017-05-10 15:27:54.362927: I tensorflow/core/common_runtime/gpu/gpu_device.cc:779] Peer access not supported between device ordinals 3 and 1
2017-05-10 15:27:54.362967: I tensorflow/core/common_runtime/gpu/gpu_device.cc:779] Peer access not supported between device ordinals 3 and 2
2017-05-10 15:27:54.364638: I tensorflow/core/common_runtime/gpu/gpu_device.cc:908] DMA: 0 1 2 3 
2017-05-10 15:27:54.364668: I tensorflow/core/common_runtime/gpu/gpu_device.cc:918] 0:   Y N N N 
2017-05-10 15:27:54.364687: I tensorflow/core/common_runtime/gpu/gpu_device.cc:918] 1:   N Y N N 
2017-05-10 15:27:54.364702: I tensorflow/core/common_runtime/gpu/gpu_device.cc:918] 2:   N N Y N 
2017-05-10 15:27:54.364717: I tensorflow/core/common_runtime/gpu/gpu_device.cc:918] 3:   N N N Y 

Answer 1

这个消息是良性的(它是一个“信息”消息，而不是一个错误)。Tensorflow中的所有东西都能正常工作，但可能比支持点对点访问的不同硬件更慢。

该消息意味着NVIDIA驱动程序报告说，GPU之间不可能进行对等访问。有关详细信息，请参阅：https://developer.nvidia.com/gpudirect。

您可以使用以下命令

nvidia-smi topo -m

以显示总线拓扑。