张量流DataSets中的稀疏张量

Question

我有一个使用TFRecord的tensorflow程序，我想用tf.contrib.data.TFRecordDataset读取数据，但当我试图解析这个例子时，我得到了一个异常："TypeError:无法将类型的对象转换为张量“，而只尝试使用

代码是：

def _parse_function(example_proto):
    features = {"var_len_feature": tf.VarLenFeature(tf.float32),
                 "FixedLenFeature": tf.FixedLenFeature([10], tf.int64),
                 "label": tf.FixedLenFeature((), tf.int32default_value=0)}

parsed_features = tf.parse_single_example(example_proto, features)
return parsed_features["image"], parsed_features["label"]

filenames = ["/var/data/file1.tfrecord", "/var/data/file2.tfrecord"]
dataset = tf.contrib.data.TFRecordDataset(filenames)
dataset = dataset.map(_parse_function)

Answer 1

TensorFlow在v1.5中添加了对此的支持

https://github.com/tensorflow/tensorflow/releases/tag/v1.5.0

"tf.data现在支持数据集元素中的tf.SparseTensor组件。“

Answer 2

张量流编程guide中的教程有不同的缩进。

# Transforms a scalar string `example_proto` into a pair of a scalar string and
# a scalar integer, representing an image and its label, respectively.
def _parse_function(example_proto):
  features = {"image": tf.FixedLenFeature((), tf.string, default_value=""),
              "label": tf.FixedLenFeature((), tf.int32, default_value=0)}
  parsed_features = tf.parse_single_example(example_proto, features)
  return parsed_features["image"], parsed_features["label"]

# Creates a dataset that reads all of the examples from two files, and extracts
# the image and label features.
filenames = ["/var/data/file1.tfrecord", "/var/data/file2.tfrecord"]
dataset = tf.contrib.data.TFRecordDataset(filenames)
dataset = dataset.map(_parse_function)

错误的缩进可能导致TypeError，从而由pyton解释器处理不需要的控制流。

Answer 3

tf.VarLenFeature创建SparseTensor。并且大多数情况下SparseTensors与小批量相关联。你能像下面这样试试吗？

dataset =tf.contrib.data.TFRecordDataset(文件名)

dataset = dataset.batch(batch_size=32)

dataset = dataset.map(_parse_function)