文档建议反馈控制台
首页
学习
活动
专区
圈层
工具
MCP广场
文章/答案/技术大牛
发布
社区首页 >问答首页 >我正在接收Tensorflow错误"ValueError:无值不受支持“。当试图填充tf数据集时。有人能帮我吗？

问我正在接收Tensorflow错误"ValueError:无值不受支持“。当试图填充tf数据集时。有人能帮我吗？
EN

Data Science用户
提问于 2020-08-18 08:41:15
回答 1查看 3.2K关注 0票数 0

我已经将我的数据存储在2(Nx3)张量中，并且我试图收集这个张量的特定行，以生成单个示例的数据，以输入Keras训练循环。基本上，这个想法很简单:索引Ib (例如，100 )和Ie (例如，207 )对每个示例都是唯一的，指定我的输入数据是来自第76、77、78、.、100、105、110、115、……205、207、208、209、.行的300 x3张量。数据张量(直至每例总共300指数)的数据张量。注意到Ib和Ie之间的5的步调。虽然附加的代码在手动获取数据时工作得很好(如代码示例中所示)，但是当Tensorflow稍后尝试执行同样的操作时会发生一些事情。此外，我知道，如果我只是在Ib之前和之后选择了固定数量的行，下面的代码就可以工作。在使用符号张量Ib (?)调用时，行updates = tf.range(start=Ib, limit=Ie, delta=5, dtype=tf.int32)似乎不能产生任何合理的输出，从而导致代码在下一行上中断。我使用Python3.7/ Tensorflow 2.3.0，附带的代码生成输出。

代码语言：javascript
复制
2020-08-18 10:25:51.384252: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2020-08-18 10:25:51.394210: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ff732d46110 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-08-18 10:25:51.394226: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version

trainINDX =   0, X.shape = [ 300, 3 ]
trainINDX =   1, X.shape = [ 300, 3 ]
trainINDX =   2, X.shape = [ 300, 3 ]
trainINDX =   3, X.shape = [ 300, 3 ]
trainINDX =   4, X.shape = [ 300, 3 ]
trainINDX =   5, X.shape = [ 300, 3 ]
trainINDX =   6, X.shape = [ 300, 3 ]
trainINDX =   7, X.shape = [ 300, 3 ]
trainINDX =   8, X.shape = [ 300, 3 ]
trainINDX =   9, X.shape = [ 300, 3 ]

Traceback (most recent call last):
  File "testData.py", line 93, in <module>
    main( )
  File "testData.py", line 80, in main
    trainData = trainData.map(fetchData, num_parallel_calls=tf.data.experimental.AUTOTUNE)
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 1702, in map
    preserve_cardinality=True)
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 4084, in __init__
    use_legacy_function=use_legacy_function)
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 3371, in __init__
    self._function = wrapper_fn.get_concrete_function()
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/eager/function.py", line 2939, in get_concrete_function
    *args, **kwargs)
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/eager/function.py", line 2906, in _get_concrete_function_garbage_collected
    graph_function, args, kwargs = self._maybe_define_function(args, kwargs)
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/eager/function.py", line 3213, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/eager/function.py", line 3075, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/framework/func_graph.py", line 986, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 3364, in wrapper_fn
    ret = _wrapper_helper(*args)
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 3299, in _wrapper_helper
    ret = autograph.tf_convert(func, ag_ctx)(*nested_args)
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/autograph/impl/api.py", line 258, in wrapper
    raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:

    testData.py:57 fetchData  *
        indices    = tf.range(start=sampleCount, limit=sampleCount+updates.shape[0], dtype=tf.int32)
    /Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/ops/variables.py:1074 _run_op
        return tensor_oper(a.value(), *args, **kwargs)
    /Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/ops/math_ops.py:1125 binary_op_wrapper
        return func(x, y, name=name)
    /Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/util/dispatch.py:201 wrapper
        return target(*args, **kwargs)
    /Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/ops/math_ops.py:1443 _add_dispatch
        y = ops.convert_to_tensor(y, dtype_hint=x.dtype.base_dtype, name="y")
    /Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/framework/ops.py:1499 convert_to_tensor
        ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
    /Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/framework/constant_op.py:338 _constant_tensor_conversion_function
        return constant(v, dtype=dtype, name=name)
    /Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/framework/constant_op.py:264 constant
        allow_broadcast=True)
    /Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/framework/constant_op.py:282 _constant_impl
        allow_broadcast=allow_broadcast))
    /Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/framework/tensor_util.py:444 make_tensor_proto
        raise ValueError("None values not supported.")

    ValueError: None values not supported.
代码语言：javascript
复制
import numpy as np
import tensorflow as tf

def main( ):

    inputData      = np.zeros((1000, 3), dtype=np.float32)
    inputData[:,0] = np.sin(np.arange(1000)/360)
    inputData[:,1] = np.cos(np.arange(1000)/360)
    inputData[:,2] = np.sin(np.arange(1000)/360) * np.cos(np.arange(1000)/360) # Generate some input data
    inputData      = tf.convert_to_tensor(inputData, dtype=tf.float32)

    outputData     = np.random.randint(low=0, high=3, size=1000, dtype=np.int32) # Generate random output data

    eventData      = np.zeros((1000,2), dtype=np.int32)
    eventData[:,0] = np.arange(1000)                                                                  # Begin index of sparse sampling
    eventData[:,1] = np.arange(1000) + np.random.randint(low=80, high=121, size=1000, dtype=np.int32) # End   index of sparse sampling
    eventData      = tf.convert_to_tensor(eventData, dtype=tf.int32)

    totalSampleCount = int(1000)
    eventCount       = int(1000)
    inputDim         = int(300)
    outputDim        = int(3)
    batchSize        = int(256)
    epochCount       = int(5)
    stepsPerEpoch    = np.floor(666/batchSize)

    trainINDX      = np.arange(666)
    validationINDX = np.arange(666,1000)

    model = tf.keras.Sequential([
        tf.keras.layers.Conv1D(filters=15, kernel_size=15, strides=1, padding='same', dilation_rate=1, activation='relu', use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', input_shape=(inputDim,3)),
        tf.keras.layers.MaxPooling1D(pool_size=5, strides=None, padding='valid'),
        tf.keras.layers.Flatten(),
        tf.keras.layers.Dense(256, activation='relu', use_bias=True),
        tf.keras.layers.Dropout(0.50),
        tf.keras.layers.Dense(outputDim, activation='softmax')
    ])

    model.compile( optimizer=tf.keras.optimizers.Adam(learning_rate=1e-3, beta_1=0.9, beta_2=0.999, amsgrad=True),
                   loss=tf.keras.losses.SparseCategoricalCrossentropy(),
                   metrics=[tf.keras.metrics.SparseCategoricalAccuracy()])

    def fetchData(eventINDX, y): # This function picks up the requested elements from inputData

        sampleINDX = tf.zeros([inputDim], dtype=tf.int32) # Initialize sampleINDX to zero

        Ib = tf.gather_nd(eventData, [tf.cast(eventINDX, dtype=tf.int32), 0]) # Begin of sparse sampling
        Ie = tf.gather_nd(eventData, [tf.cast(eventINDX, dtype=tf.int32), 1]) # End   of sparse sampling

        indices    = tf.range(start=0, limit=24, dtype=tf.int32)
        updates    = tf.range(start=Ib-24, limit=Ib, dtype=tf.int32)
        sampleINDX = tf.tensor_scatter_nd_update(sampleINDX, tf.expand_dims(indices, axis=1), updates)

        sampleCount = tf.Variable(24, dtype=tf.int32)

        updates    = tf.range(start=Ib, limit=Ie, delta=5, dtype=tf.int32)
        indices    = tf.range(start=sampleCount, limit=sampleCount+updates.shape[0], dtype=tf.int32)
        sampleINDX = tf.tensor_scatter_nd_update(sampleINDX, tf.expand_dims(indices, axis=1), updates)

        sampleCount.assign_add(updates.shape[0])
           
        remainingSampleCount = tf.math.subtract(tf.constant(inputDim, dtype=tf.int32), sampleCount)

        indices    = tf.range(start=sampleCount, limit=inputDim, dtype=tf.int32)
        updates    = tf.range(start=Ie, limit=Ie+remainingSampleCount, dtype=tf.int32)
        sampleINDX = tf.tensor_scatter_nd_update(sampleINDX, tf.expand_dims(indices, axis=1), updates)

        X = tf.gather(inputData, tf.math.floormod(sampleINDX, totalSampleCount), axis=0)

        return X, y

    print('')
    for i in range(10):
        X, y = fetchData(i,outputData[i])
        print('trainINDX = %3d, X.shape = [ %3d, %d ]' % (i, X.shape[0], X.shape[1]))
    print('')

    trainData = tf.data.Dataset.from_tensor_slices((trainINDX, outputData[trainINDX]))
    trainData = trainData.shuffle(buffer_size=trainINDX.size, reshuffle_each_iteration=True) 
    trainData = trainData.map(fetchData, num_parallel_calls=tf.data.experimental.AUTOTUNE)
    trainData = trainData.repeat()
    trainData = trainData.batch(batchSize, drop_remainder=True)
    trainData = trainData.prefetch(buffer_size=tf.data.experimental.AUTOTUNE)

    validationData = tf.data.Dataset.from_tensor_slices((validationINDX, outputData[validationINDX]))
    validationData = validationData.map(fetchData, num_parallel_calls=tf.data.experimental.AUTOTUNE)
    validationData = validationData.batch(batchSize, drop_remainder=False)
    validationData = validationData.prefetch(buffer_size=tf.data.experimental.AUTOTUNE)

    history = model.fit(x=trainData, steps_per_epoch=stepsPerEpoch, validation_data=validationData, verbose=1, epochs=epochCount)

if __name__== "__main__":
  main( )

任何帮助解决所描述的问题都是非常感谢的。提前谢谢你！

machine-learning
keras
tensorflow
dataset
python-3.x
EN

回答 1

Data Science用户

发布于 2020-08-21 06:02:17

将"updates.shape0“替换为”tf.shape(更新)0“似乎解决了这个特殊问题。然而，这将导致下文所述的另一个问题。

代码语言：javascript
复制
import numpy as np
import tensorflow as tf

def main( ):

    inputData      = np.zeros((1000, 3), dtype=np.float32)
    inputData[:,0] = np.sin(np.arange(1000)/360)
    inputData[:,1] = np.cos(np.arange(1000)/360)
    inputData[:,2] = np.sin(np.arange(1000)/360) * np.cos(np.arange(1000)/360) # Generate some input data
    inputData      = tf.convert_to_tensor(inputData, dtype=tf.float32)

    outputData     = np.random.randint(low=0, high=3, size=1000, dtype=np.int32) # Generate random output data

    eventData      = np.zeros((1000,2), dtype=np.int32)
    eventData[:,0] = np.arange(1000)                                                                  # Begin index of sparse sampling
    eventData[:,1] = np.arange(1000) + np.random.randint(low=80, high=121, size=1000, dtype=np.int32) # End   index of sparse sampling
    eventData      = tf.convert_to_tensor(eventData, dtype=tf.int32)

    totalSampleCount = int(1000)
    eventCount       = int(1000)
    inputDim         = int(300)
    outputDim        = int(3)
    batchSize        = int(256)
    epochCount       = int(5)
    stepsPerEpoch    = int(np.floor(666/batchSize))

    trainINDX      = np.arange(666)
    validationINDX = np.arange(666,1000)

    model = tf.keras.Sequential([
        tf.keras.layers.Conv1D(filters=15, kernel_size=15, strides=1, padding='same', dilation_rate=1, activation='relu', use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', input_shape=(inputDim,3)),
        tf.keras.layers.MaxPooling1D(pool_size=5, strides=None, padding='valid'),
        tf.keras.layers.Flatten(),
        tf.keras.layers.Dense(256, activation='relu', use_bias=True),
        tf.keras.layers.Dropout(0.50),
        tf.keras.layers.Dense(outputDim, activation='softmax')
    ])

    model.compile( optimizer=tf.keras.optimizers.Adam(learning_rate=1e-3, beta_1=0.9, beta_2=0.999, amsgrad=True),
                   loss=tf.keras.losses.SparseCategoricalCrossentropy(),
                   metrics=[tf.keras.metrics.SparseCategoricalAccuracy()])

    def fetchData(eventINDX, y): # This function picks up the requested elements from inputData

        sampleINDX = tf.zeros([inputDim], dtype=tf.int32) # Initialize sampleINDX to zero

        Ib = tf.gather_nd(eventData, [tf.cast(eventINDX, dtype=tf.int32), 0]) # Begin of sparse sampling
        Ie = tf.gather_nd(eventData, [tf.cast(eventINDX, dtype=tf.int32), 1]) # End   of sparse sampling

        indices    = tf.range(start=0, limit=24, dtype=tf.int32)
        updates    = tf.range(start=Ib-24, limit=Ib, dtype=tf.int32)
        sampleINDX = tf.tensor_scatter_nd_update(sampleINDX, tf.expand_dims(indices, axis=1), updates)

        sampleCount = tf.Variable(24, dtype=tf.int32)

        updates    = tf.range(start=Ib, limit=Ie, delta=5, dtype=tf.int32)
        indices    = tf.range(start=sampleCount, limit=sampleCount+tf.shape(updates)[0], dtype=tf.int32)
        sampleINDX = tf.tensor_scatter_nd_update(sampleINDX, tf.expand_dims(indices, axis=1), updates)

        sampleCount.assign_add(tf.shape(updates)[0])
           
        remainingSampleCount = tf.math.subtract(tf.constant(inputDim, dtype=tf.int32), sampleCount)

        indices    = tf.range(start=sampleCount, limit=inputDim, dtype=tf.int32)
        updates    = tf.range(start=Ie, limit=Ie+remainingSampleCount, dtype=tf.int32)
        sampleINDX = tf.tensor_scatter_nd_update(sampleINDX, tf.expand_dims(indices, axis=1), updates)

        X = tf.gather(inputData, tf.math.floormod(sampleINDX, totalSampleCount), axis=0)

        return X, y

    print('')
    for i in range(10):
        X, y = fetchData(i,outputData[i])
        print('trainINDX = %3d, X.shape = [ %3d, %d ]' % (i, X.shape[0], X.shape[1]))
    print('')

    trainData = tf.data.Dataset.from_tensor_slices((trainINDX, outputData[trainINDX]))
    trainData = trainData.shuffle(buffer_size=trainINDX.size, reshuffle_each_iteration=True) 
    trainData = trainData.map(fetchData, num_parallel_calls=tf.data.experimental.AUTOTUNE)
    trainData = trainData.repeat()
    trainData = trainData.batch(batchSize, drop_remainder=True)
    trainData = trainData.prefetch(buffer_size=tf.data.experimental.AUTOTUNE)

    validationData = tf.data.Dataset.from_tensor_slices((validationINDX, outputData[validationINDX]))
    validationData = validationData.map(fetchData, num_parallel_calls=tf.data.experimental.AUTOTUNE)
    validationData = validationData.batch(batchSize, drop_remainder=False)
    validationData = validationData.prefetch(buffer_size=tf.data.experimental.AUTOTUNE)

    history = model.fit(x=trainData, steps_per_epoch=stepsPerEpoch, validation_data=validationData, verbose=1, epochs=epochCount)

if __name__== "__main__":
  main( )

现在附加文件的输出是

代码语言：javascript
复制
trainINDX =   0, X.shape = [ 300, 3 ]
trainINDX =   1, X.shape = [ 300, 3 ]
trainINDX =   2, X.shape = [ 300, 3 ]
trainINDX =   3, X.shape = [ 300, 3 ]
trainINDX =   4, X.shape = [ 300, 3 ]
trainINDX =   5, X.shape = [ 300, 3 ]
trainINDX =   6, X.shape = [ 300, 3 ]
trainINDX =   7, X.shape = [ 300, 3 ]
trainINDX =   8, X.shape = [ 300, 3 ]
trainINDX =   9, X.shape = [ 300, 3 ]

Epoch 1/5
Traceback (most recent call last):
  File "testData.py", line 93, in <module>
    main( )
  File "testData.py", line 90, in main
    history = model.fit(x=trainData, steps_per_epoch=stepsPerEpoch, validation_data=validationData, verbose=1, epochs=epochCount)
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/keras/engine/training.py", line 108, in _method_wrapper
    return method(self, *args, **kwargs)
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/keras/engine/training.py", line 1098, in fit
    tmp_logs = train_function(iterator)
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/eager/def_function.py", line 780, in __call__
    result = self._call(*args, **kwds)
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/eager/def_function.py", line 840, in _call
    return self._stateless_fn(*args, **kwds)
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/eager/function.py", line 2829, in __call__
    return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/eager/function.py", line 1848, in _filtered_call
    cancellation_manager=cancellation_manager)
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/eager/function.py", line 1924, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/eager/function.py", line 550, in call
    ctx=ctx)
  File "/Users/relaxation82/Library/Python/3.7/lib/python/site-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
    inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.FailedPreconditionError:  Error while reading resource variable _AnonymousVar24 from Container: localhost. This could mean that the variable was uninitialized. Not found: Resource localhost/_AnonymousVar24/N10tensorflow3VarE does not exist.
     [[{{node range_3/ReadVariableOp}}]]
     [[IteratorGetNext]] [Op:__inference_train_function_1569]

Function call stack:
train_function
```
代码语言：javascript
复制
票数 1
EN
页面原文内容由Data Science提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：

https://datascience.stackexchange.com/questions/80433

复制
相关文章

相似问题

领券

腾讯云开发者

扫码关注腾讯云开发者

扫码关注腾讯云开发者

领取腾讯云代金券

热门产品

热门推荐

更多推荐

Copyright © 2013 - 2026 Tencent Cloud. All Rights Reserved. 腾讯云 版权所有 

深圳市腾讯计算机系统有限公司 ICP备案/许可证号：粤B2-20090059 粤公网安备44030502008569号

腾讯云计算（北京）有限责任公司 京ICP证150476号 |  京ICP备11018762号

问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档