使用多幅图像进行批处理推理cppflow C++

Question

我试图在cppflow库 10 x64机器上使用VS2019 C++。我想推断我的batch of images (vector <cv::Mat> )模型。我为single image编写了一个简单的代码，如下所示，它运行correctly

string im_path{ "..." };
string model_path{ "...\\ocr_model" };

cv::Mat tmp, im;

cv::resize(cv::imread(im_path, cv::IMREAD_GRAYSCALE), tmp, cv::Size(127, 25), 0, 0, cv::INTER_CUBIC);

cv::transpose(tmp, im);

int rows = im.rows; int cols = im.cols; int channels = im.channels();

// Put image in tensor 
std::vector<uint8_t> img_data;
    
auto e = std::end(img_data);
img_data.insert(e, im.data, im.data + im.total() * channels);

auto input = cppflow::tensor(img_data, {rows, cols, channels});
       
input = cppflow::cast(input, TF_UINT8, TF_FLOAT);
auto t = input.get_data<float>();
 
input = input / 255.f;
input = cppflow::expand_dims(input, 0);
    
cppflow::model model{ model_path };
auto output = model({ {"serving_default_input:0", input}}, { "StatefulPartitionedCall:0"});

我想要load multiple images (在下面的代码中，我使用一个克隆的图像作为第二个映像)。以下是我真正想做的事：

string im_path{ "..." };
string model_path{ "...\\ocr_model" };

cv::Mat tmp, im;
cv::resize(cv::imread(im_path, cv::IMREAD_GRAYSCALE), tmp, cv::Size(127, 25), 0, 0, cv::INTER_CUBIC);

cv::transpose(tmp, im);
    
int rows = im.rows; int cols = im.cols; int channels = im.channels();

// Put image in tensor 
std::vector<uint8_t> img_data; 

auto im_clone = im.clone();
auto e = std::end(img_data);
img_data.insert(e, im.data, im.data + im.total() * channels);
e = std::end(img_data);
img_data.insert(e, im_clone.data, im_clone.data + im_clone.total() * channels);

auto input = cppflow::tensor(img_data, {2, rows, cols, channels});
       
input = cppflow::cast(input, TF_UINT8, TF_FLOAT);
 
input = input / 255.f;
input = cppflow::expand_dims(input, 0);
    
cppflow::model model{ model_path };
auto output = model({ {"serving_default_input:0", input}}, { "StatefulPartitionedCall:0"});

正如您所看到的，difference between the codes是img_data preparation和tensor definition，但不幸的是，我得到了以下错误：

Cppflow_Test.exe中0x00007FFFF4514ED9处的未处理异常: C++异常:内存位置为0x00000031F72FDBD 8的std::runtime_error。

如何将多个图像(vector< cv::Mat >)加载到张量并使用其相应的输出？换句话说，我需要一个batch inference using cppflow library示例。

Answer 1

尝试使用std::copy将图像(或多个图像)的多个副本插入到img_data向量中。

int data_size = rows * cols * channels;
std::vector<uint8> img_data; 
for (size_t i = 0; i < batch_size; i++)
{
    std::copy(im.data, im.data + data_size, std::begin(img_data) + i * data_size);
}
auto input = cppflow::tensor(img_data, {batch_size, rows, cols, channels});

Answer 2

使用cppflow::concat连接要发送到模型的n个张量：

auto input1 = cppflow::decode_jpeg(file1);
input1 = cppflow::expand_dims(input1, 0);
auto input2 = cppflow::decode_jpeg(file2);
input2 = cppflow::expand_dims(input2, 0);

cppflow::tensor dim({0});
std::vector<cppflow::tensor> values;
values.push_back(input1);
values.push_back(input2);

auto inputs = cppflow::concat(dim, values);

auto output = model({ {"serving_default_input:0", inputs}}, { "StatefulPartitionedCall:0"});