如何在yolov3模型上进行多幅图像批量推理？

Question

我已经在TF服务器上部署了yolov3对象检测模型。我可以成功地对单个图像进行推断，现在我想测试多批图像的服务器容量，但是当我试图传递多个图像时，我会得到一个错误：“无法挤压昏暗，预期维数为1，得到6\n\t [{{node 3/yolo_nms/挤压}]”saved_model签名如下

The given SavedModel SignatureDef contains the following input(s):
  inputs['input'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, -1, -1, 3)
      name: serving_default_input:0

当我对单个大小的图像(1,416,415,3)模型进行推理时，接受并对其执行推断，但是当我传递一批图像数组时，例如对于6幅形状为(6,416,416,3)的图像，我会得到一个错误

details = "Can not squeeze dim[0], expected a dimension of 1, got 6
[[{{node yolov3/yolo_nms/Squeeze}}]]"
debug_error_string = "{"created":"@1623239327.766680379","description":"Error received from peer ipv4:127.0.0.1:8500","file":"src/core/lib/surface/call.cc","file_line":1061,"grpc_message":"Can not squeeze dim[0], expected a dimension of 1, got 6\n\t [[{{node yolov3/yolo_nms/Squeeze}}]]","grpc_status":3}"

加载一批图像的代码如下所示

load_imgs = load_images_from_dir("/content/yolov3-tf2/image_data/",416,6)
print(load_imgs.shape)
(6, 416, 416, 3)

通过gRPC请求将此图像批处理发送到tf服务器的代码如下所示

request.inputs["input"].CopyFrom(
      tf.make_tensor_proto(
          load_imgs,
          dtype= types_pb2.DT_FLOAT ,
          shape=load_imgs.shape
      )
  )

模型描述显示它接受任意数量的图像作为维度声明-1，但是当我传递多个图像时，它会抛出一个错误，如上面所示。我使用此代码中的模型回购模型码源代码将模型转换为saved_model格式，如下所示

!wget https://pjreddie.com/media/files/yolov3.weights -O data/yolov3.weights
!python convert.py --weights ./data/yolov3.weights --output ./checkpoints/yolov3.tf
!python ./tools/export_tfserving.py --output serving_model/yolov3/1/

!saved_model_cli show --dir serving_model/yolov3/1/ --tag_set serve --signature_def serving_default

2021-06-10 19:40:53.843200: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
The given SavedModel SignatureDef contains the following input(s):
  inputs['input'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, -1, -1, 3)
      name: serving_default_input:0
The given SavedModel SignatureDef contains the following output(s):
  outputs['yolo_nms'] tensor_info:
      dtype: DT_FLOAT
      shape: (1, -1, 4)
      name: StatefulPartitionedCall:0
  outputs['yolo_nms_1'] tensor_info:
      dtype: DT_FLOAT
      shape: (1, -1)
      name: StatefulPartitionedCall:1
  outputs['yolo_nms_2'] tensor_info:
      dtype: DT_INT64
      shape: (1, -1)
      name: StatefulPartitionedCall:2
  outputs['yolo_nms_3'] tensor_info:
      dtype: DT_INT32
      shape: (1)
      name: StatefulPartitionedCall:3
Method name is: tensorflow/serving/predict

我做错什么了？

我可以将任意数量的图像传递给不同批次大小的模型吗？或者是硬编码到特定的批次大小？还是我用错误的方式称呼它？

Answer 1

不你说的完全错了。任何模型一般都有输入形状BATCH_SIZE，FEATURE_SIZE，例如8,640,640,3。因此，批量图像在yolo中是独立的。您可以使用批处理概念对图像进行推理。