如何使用tf.check_numerics

Question

受this SO post的启发，我正在尝试使用tf.check_numerics (TensorFlow 1.2)来防止渐变中的NaN。我的代码是：

optimizer = tf.train.RMSPropOptimizer(learning_rate, decay=0.99)
grads_and_vars = optimizer.compute_gradients(graph.loss_total)
grads, variables = zip(*grads_and_vars)
clipped_gradients, _ = (tf.clip_by_global_norm(grads, 1.))

grad_check = tf.check_numerics(clipped_gradients, 'check_numerics caught bad gradients')
     # ^ this line causes an error
with tf.control_dependencies([grad_check]):
    graph.train_op = optimizer.apply_gradients(zip(clipped_gradients, variables))

但我得到了错误消息：

ValueError: Tried to convert 'tensor' to a tensor and failed. Error: Shapes must be equal rank, but are 2 and 1
    From merging shape 2 with other shapes. for 'training/CheckNumerics/packed' (op: 'Pack') with input shapes: [4,16], [16], [16,2], [2].

我做错了什么，我该如何修复它？

Answer 1

clipped_gradients是一个列表。试一试

grad_check = tf.check_numerics(clipped_gradients[0], 'check_numerics caught bad gradients')