为什么tensorflow中的`tf.nn.nce_loss`不能在GPU上运行？

Question

代码tensorflow/examples/tutorials/word2vec/word2vec_basic.py有一个注释# Ops and variables pinned to the CPU because of missing GPU implementation。我还发现GPU不能实现tf.nn.nce_loss操作。那么为什么tf.nn.nce_loss不能在图形处理器上运行呢？

我已经使用log_device_placement查看过MUST BE CPU操作。结果如下：

nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler)/job:localhost/replica:0/task:0/cpu:0
nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits): /job:localhost/replica:0/task:0cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits)/job:localhost/replica:0/task:0/cpu:0
nce_loss/SparseToDense: (SparseToDense): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/SparseToDense: (SparseToDense)/job:localhost/replica:0/task:0/cpu:0
nce_loss/concat: (ConcatV2): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/concat: (ConcatV2)/job:localhost/replica:0/task:0/cpu:0
nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler)/job:localhost/replica:0/task:0/cpu:0
nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits): /job:localhost/replica:0/task:0cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits)/job:localhost/replica:0/task:0/cpu:0
nce_loss/SparseToDense: (SparseToDense): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/SparseToDense: (SparseToDense)/job:localhost/replica:0/task:0/cpu:0
nce_loss/concat: (ConcatV2): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/concat: (ConcatV2)/job:localhost/replica:0/task:0/cpu:0
nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler)/job:localhost/replica:0/task:0/cpu:0
nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits): /job:localhost/replica:0/task:0cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits)/job:localhost/replica:0/task:0/cpu:0
nce_loss/SparseToDense: (SparseToDense): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/SparseToDense: (SparseToDense)/job:localhost/replica:0/task:0/cpu:0
nce_loss/concat: (ConcatV2): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/concat: (ConcatV2)/job:localhost/replica:0/task:0/cpu:0
nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler)/job:localhost/replica:0/task:0/cpu:0
nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits): /job:localhost/replica:0/task:0cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits)/job:localhost/replica:0/task:0/cpu:0
nce_loss/SparseToDense: (SparseToDense): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/SparseToDense: (SparseToDense)/job:localhost/replica:0/task:0/cpu:0
nce_loss/concat: (ConcatV2): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/concat: (ConcatV2)/job:localhost/replica:0/task:0/cpu:0
nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler)/job:localhost/replica:0/task:0/cpu:0
nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits): /job:localhost/replica:0/task:0cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits)/job:localhost/replica:0/task:0/cpu:0
nce_loss/SparseToDense: (SparseToDense): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/SparseToDense: (SparseToDense)/job:localhost/replica:0/task:0/cpu:0
nce_loss/concat: (ConcatV2): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/concat: (ConcatV2)/job:localhost/replica:0/task:0/cpu:0

但是，我仍然不知道是哪种特殊操作导致了这个问题。有没有GPU无法实现的特殊运维？或者这是由于tensorflow本身没有实现？

Answer 1

您也可以使用gpu加速器执行此示例。

请参阅tensorflow支持的设备https://www.tensorflow.org/guide/using_gpu#supported_devices

您必须添加以下代码行

with tf.device('/gpu:0'):

并删除#with tf.device('/cpu:0'):

从根本上用图形处理器取代cpu：https://github.com/tensorflow/tensorflow/blob/bd965956535b1026a0769d04c3f6419d020f5f9c/tensorflow/examples/tutorials/word2vec/word2vec_basic.py#L189

希望这能有所帮助。