OpenVINO如何处理CT分割

Question

我尝试用新冠肺炎CT分割数据集进行NCS2的CT切片分割。

我引用这个GitHub并使用它的pth文件来转换为ONNX。

我使用python mo_onnx.py --input_model model.onnx --input_shape [1,1,512,512] --data_type FP16生成IR文件。

我的预期推断结果如下图所示。

​

​

我尝试过segmentation_demo.py的openvino_2021.4.689版本，但是错误消息显示如下。

[ INFO ] Initializing Inference Engine...
[ INFO ] Loading network...
[ INFO ] Reading network from IR...
Traceback (most recent call last):
  File "C:\Program Files (x86)\Intel\openvino_2021.4.689\deployment_tools\inference_engine\demos\segmentation_demo\python\segmentation_demo.py", line 270, in <module>
    sys.exit(main() or 0)
  File "C:\Program Files (x86)\Intel\openvino_2021.4.689\deployment_tools\inference_engine\demos\segmentation_demo\python\segmentation_demo.py", line 176, in main
    model, visualizer = get_model(ie, args)
  File "C:\Program Files (x86)\Intel\openvino_2021.4.689\deployment_tools\inference_engine\demos\segmentation_demo\python\segmentation_demo.py", line 160, in get_model
    return SegmentationModel(ie, args.model), SegmentationVisualizer(args.colors)
  File "C:\Program Files (x86)\Intel\openvino_2021.4.689\deployment_tools\open_model_zoo\demos\common\python\models\segmentation.py", line 27, in __init__
    self.input_blob_name = self.prepare_inputs()
  File "C:\Program Files (x86)\Intel\openvino_2021.4.689\deployment_tools\open_model_zoo\demos\common\python\models\segmentation.py", line 43, in prepare_inputs
    raise RuntimeError("3-channel 4-dimensional model's input is expected")
RuntimeError: 3-channel 4-dimensional model's input is expected

然后，我还尝试运行classification_sample_async.py并遇到下面类似的问题。

[ INFO ] Creating Inference Engine
[ INFO ] Reading the network: C:\Users\Hsien\Desktop\Segmentation-COVID-19-master\Sample_Testing\model.xml
[ INFO ] Configuring input and output blobs
[ INFO ] Loading the model to the plugin
[ INFO ] Starting inference in asynchronous mode
Traceback (most recent call last):
  File "C:\Program Files (x86)\Intel\openvino_2021.4.689\inference_engine\samples\python\classification_sample_async\classification_sample_async.py", line 168, in <module>
    sys.exit(main())
  File "C:\Program Files (x86)\Intel\openvino_2021.4.689\inference_engine\samples\python\classification_sample_async\classification_sample_async.py", line 111, in main
    exec_net.requests[i].async_infer({input_blob: input_data[i]})
  File "ie_api.pyx", line 1222, in openvino.inference_engine.ie_api.InferRequest.async_infer
  File "ie_api.pyx", line 1224, in openvino.inference_engine.ie_api.InferRequest.async_infer
  File "ie_api.pyx", line 1363, in openvino.inference_engine.ie_api.InferRequest._fill_inputs
ValueError: could not broadcast input array from shape (1,3,512,512) into shape (1,1,512,512)

这两条消息都可以发现演示代码不支持灰度图像。

但是如何正确地处理灰度CT切片的分割呢？

我还在记事本 on GitHub中引用了210 ct扫描-live-ference.ipynb和110-ct分段-quze.ipynb，但都是徒劳的。

(我的环境是带有openvino_2021.4.689版本的Windows 11。)

更新新的测试结果。

下面我开发了一个简单的推理代码。

from openvino.inference_engine import IECore, Blob, TensorDesc
import cv2
import numpy as np

IMG_PATH = "20.jpg"
XML_PATH = "model.xml"
BIN_PATH = "model.bin"

ie_core_handler = IECore()
network = ie_core_handler.read_network(model=XML_PATH, weights=BIN_PATH)
input_blob_name = next(iter(network.input_info))
network.input_info[input_blob_name].precision = 'FP16'
executable_network = ie_core_handler.load_network(network, device_name='MYRIAD', num_requests=1)
inference_request = executable_network.requests[0]

image = cv2.imread(IMG_PATH)
re_img = cv2.resize(src=image, dsize=(512, 512))
input_data = np.expand_dims(np.transpose(re_img, (2, 0, 1)), 0).astype(np.float16)
tensor_description = TensorDesc(precision="FP16", dims=(1, 1, 512, 512), layout='NCHW')
input_blob = Blob(tensor_description, input_data)

input_blob_name = next(iter(inference_request.input_blobs))
inference_request.set_blob(blob_name=input_blob_name, blob=input_blob)
inference_request.infer()
output_blob_name = next(iter(inference_request.output_blobs))
output = inference_request.output_blobs[output_blob_name].buffer
print(output)

这是可行的，我的其他三个渠道的项目。

但是这个错误来自于这个1通道模型。

Traceback (most recent call last):
  File "C:\Users\Hsien\Desktop\seg_test.py", line 20, in <module>
    input_blob = Blob(tensor_description, input_data)
  File "ie_api.pyx", line 214, in openvino.inference_engine.ie_api.Blob.__cinit__
AttributeError: Number of elements in provided numpy array 786432 and required by TensorDesc 262144 are not equal

所以这意味着OpenVINO不能支持灰度模型什么的？

Answer 1

通常，在用您自己的模型替换模型之后，您不能期望任何演示仍能工作，因为不同的模型对输入格式布局和前后处理有不同的要求。

对于这个AnamNet，我首先将模型导出为onnx格式，方法是向test.ipynb添加以下行(可能您必须确保模型位于test.ipynb上，而不是GPU上，我不太确定)：

torch.onnx.export(net, (inp1, ), 'model.onnx')

然后将其加载到openvino并运行推断(sampledata.mat来自https://github.com/NaveenPaluru/Segmentation-COVID-19/)：

import scipy.io as io
import openvino.runtime as ov
import torchvision.transforms as transforms
import matplotlib.pyplot as plt
import numpy as np

core = ov.Core()
model = core.read_model('model.onnx')

device = 'CPU'
model_cpu = core.compile_model(model, device)
k0 = model_cpu.output(0)

data = io.loadmat('sampledata.mat')

fig, axs = plt.subplots(1, 4, figsize=(50, 50))
for i, name in enumerate(['im1','im2','im3','im4']):
    inp = data[name]
    inp = np.reshape(inp,(512,512,1))
    inp = transforms.ToTensor()(inp).unsqueeze(dim=0)
    out = model_cpu.infer_new_request([inp])[k0]
    axs[i].imshow(out)
plt.show()

它复制了类似于我在test.ipynb中看到的结果。

至于为什么您的代码(基于OpennVINO API1.0的代码)不能工作，请从您显示的错误日志中看到：

AttributeError: Number of elements in provided numpy array 786432 and required by TensorDesc 262144 are not equal

我猜OpenCV自786432=512x512x3以来仍然产生3通道，并且模型只接受262144=512x512输入，因此您可以按照此链接中描述的指令来确保在输入到OpenVINO之前生成灰度数组。

Answer 2

目前，开放模型动物园演示和样本(segmentation_demo.py，classification_sample_async.py)只支持3个输入通道，这是按BGR通道顺序。

1开放模型动物园演示和模型的输入通道可能在以后的版本中得到支持和启用。

Answer 3

如果模型是为3通道RGB/BGR输入而训练的，那么您需要将3通道内容输入到模型中。

您是否尝试过将1通道输入转换为3通道输入，即将“纯灰度图像”转换为彩色图像(“mycolor=cv2.cvtColor(灰色，cv2.COLOR_GRAY2RGB)")？还是使用NUMPY添加额外的通道/维度？

你会有“足够”的那些灰度图像来重新训练一个模型吗？

你可能想看看仍然开放拉/默格training "延期/拉动/546“的乳房超声(模拟)训练。