如何在seq2seq_model的注意译码器中获取注意力值来绘制bleu分数

Question

我正在研究一种语言翻译模式。

  1. I want to visualize data as mentioned in [http://www.wildml.com/2016/01/attention-and-memory-in-deep-learning-and-nlp/](http://www.wildml.com/2016/01/attention-and-memory-in-deep-learning-and-nlp/) using bleu score.

2.  
for a in xrange(num_heads):
    with variable_scope.variable_scope("Attention_%d" % a):
      y = linear(query, attention_vec_size, True)
      y = array_ops.reshape(y, [-1, 1, 1, attention_vec_size])
      # Attention mask is a softmax of v^T * tanh(...).
      s = math_ops.reduce_sum(
          v[a] * math_ops.tanh(hidden_features[a] + y), [2, 3])
      a = nn_ops.softmax(s)
      # Now calculate the attention-weighted vector d.
      d = math_ops.reduce_sum(
          array_ops.reshape(a, [-1, attn_length, 1, 1]) * hidden,
          [1, 2])
      ds.append(array_ops.reshape(d, [-1, attn_size]))
  return ds

如何修改代码以恢复可视化的"a“值？

Answer 1

您首先需要在python列表中保存对这些张量的引用。然后将python列表传递给session.run函数。结果将是一个包含这些张量的numpy值的列表。