Keras全连接密集输出M x N？

Question

我正在查看Keras中的示例/ example _orc.py，当我运行它时，我看到如下内容

_______________
max2 (MaxPooling2D)              (None, 32, 16, 16)    0           conv2[0][0]                      
____________________________________________________________________________________________________
reshape (Reshape)                (None, 32, 256)       0           max2[0][0]                       
____________________________________________________________________________________________________
dense1 (Dense)                   (None, 32, 32)        8224        reshape[0][0]                    
_____________________________________________________________________________________

密集层输出张量32x32。我正在尝试在使用tf.matmul的pur TensorFlow中复制这段代码，但是如何使用matmul输出32x32？

添加：

我并不是要完全复制Keras的例子，

w = 128; h = 64
# junk image, only one
dataset = np.zeros((1,w,h,1))

import tensorflow as tf

pool_size = 1
num_filters = 16

def weight_variable(shape):
  initial = tf.truncated_normal(shape, stddev=0.1)
  return tf.Variable(initial)

def bias_variable(shape):
  initial = tf.constant(0.1, shape=shape)
  return tf.Variable(initial)

def conv2d(x, W):
  return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')

def max_pool_2x2(x):
  return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],
                        strides=[1, 2, 2, 1], padding='SAME')

inputs = tf.placeholder(tf.float32, [None, w, h, 1])

W_conv1 = weight_variable([3, 3, 1, num_filters])
b_conv1 = bias_variable([num_filters])
h_conv1 = tf.nn.relu(conv2d(inputs, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)

W_conv2 = weight_variable([3, 3, num_filters, num_filters])
b_conv2 = bias_variable([num_filters])
h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)

h_pool2_flat = tf.reshape(h_pool2, [-1, 32, 256])

W_fc1 = weight_variable([256, 32])
b_fc1 = bias_variable([32])
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)

print inputs.shape
with tf.Session() as sess:
     sess.run(tf.global_variables_initializer())
     output = sess.run(h_pool2_flat, feed_dict={inputs: dataset})
     print 'output',output.shape

我得到了

ValueError: Shape must be rank 2 but is rank 3 for 'MatMul_5' (op: 'MatMul') with input shapes: [?,32,256], [256,32].

一个较小的例子

import numpy as np
import tensorflow as tf

dataset = np.zeros((3,2,4))

inputs = tf.placeholder(tf.float32, [None, 2, 4])
print inputs
W = tf.zeros((4,5))
print W
W2 = tf.matmul(inputs, W)

with tf.Session() as sess:
     sess.run(tf.global_variables_initializer())
     output = sess.run(W2, feed_dict={inputs: dataset})
     print 'output',output.shape

这也会产生类似的错误

ValueError: Shape must be rank 2 but is rank 3 for 'MatMul_12' (op: 'MatMul') with input shapes: [?,2,4], [4,5].

有什么想法吗？

谢谢,

Answer 1

32在那里，因为它在上一层中。它保持不变。

如here所述，考虑到最后两个维度，tf.matmul相乘。(请参阅采用两个以上维度的示例)

我看到你有一个Dense(32)，输入大小= 256。

这意味着权重矩阵是(256,32)。在keras中，乘法as seen here是inputs x kernel。

因此，如果您的input张量形状为(?, any, 256)，weights矩阵形状为(256,32)，那么您需要做的就是：

output = tf.matmul(input,weights)

这将输出一个形状(?, any, 32) - any is there原封不动，因为它以前就在那里。

您可能还想对偏差求和，这将遵循相同的原则。需要一个形状为(32,)的偏移向量。