NetcatH.264视频从Raspivid到OpenCV

Question

其目标是通过网络将Raspberry Pi (Raspivid/H.264)的视频流到一个运行在笔记本上的OpenCV应用程序中。

公开的CV捕获如下(C++)：

cv::VideoCapture cap;
cap.open("cam_1"); // cam_1 is a FIFO 

cv::Mat frame;

while(1){
    cap >> frame;
    cv::imshow("", frame);
    cv::waitKey(10);
}

FIFO流创建如下：

mkfifo cam_1

一旦OpenCV程序运行，netcat侦听器就会启动：

ncat --recv-only --keep-open --verbose --listen 5001 > cam_1

一旦netcat侦听器在笔记本上运行，流就从Raspberry Pi启动

raspivid --verbose --nopreview -b 2000000 --timeout 0 -o - | ncat 192.168.LAPTOP.IP 5001

或者，为了调试目的，笔记本电脑上的本地文件可以流到netcat中：

cat video.h264 | nc 192.168.LAPTOP.IP 5001 

这两种方法都会产生以下错误：

无法停止流:不适当的ioctl用于设备(错误)icvOpenAVI_XINE()：无法初始化视频驱动程序。

有趣的是，如果我在笔记本上启动Netcat侦听器，然后用CTRL+C关闭它，然后在启动视频流之前再次启动它，使用任何一种方法.，然后视频正确播放。

我不明白为什么启动netcat侦听器，然后杀死它，然后再次启动有影响或影响是什么。我已经考虑到，可能我需要回音EOF或BOF到FIFO之前的视频，我不知道这个语法会是什么。

我试过各种口味的Netcat。

Answer 1

我刚刚用下面的https://stackoverflow.com/a/48675107/2355051解决了这个问题

我最终适应了这个巨蟒配方

树莓Pi：(createStream.py)

import io
import socket
import struct
import time
import picamera

# Connect a client socket to my_server:8000 (change my_server to the
# hostname of your server)
client_socket = socket.socket()
client_socket.connect(('10.0.0.3', 777))

# Make a file-like object out of the connection
connection = client_socket.makefile('wb')
try:
    with picamera.PiCamera() as camera:
        camera.resolution = (1024, 768)
        # Start a preview and let the camera warm up for 2 seconds
        camera.start_preview()
        time.sleep(2)

        # Note the start time and construct a stream to hold image data
        # temporarily (we could write it directly to connection but in this
        # case we want to find out the size of each capture first to keep
        # our protocol simple)
        start = time.time()
        stream = io.BytesIO()
        for foo in camera.capture_continuous(stream, 'jpeg', use_video_port=True):
            # Write the length of the capture to the stream and flush to
            # ensure it actually gets sent
            connection.write(struct.pack('<L', stream.tell()))
            connection.flush()

            # Rewind the stream and send the image data over the wire
            stream.seek(0)
            connection.write(stream.read())

            # Reset the stream for the next capture
            stream.seek(0)
            stream.truncate()
    # Write a length of zero to the stream to signal we're done
    connection.write(struct.pack('<L', 0))
finally:
    connection.close()
    client_socket.close()

在处理流的机器上：(processStream.py)

import io
import socket
import struct
import cv2
import numpy as np

# Start a socket listening for connections on 0.0.0.0:8000 (0.0.0.0 means
# all interfaces)
server_socket = socket.socket()
server_socket.bind(('0.0.0.0', 777))
server_socket.listen(0)

# Accept a single connection and make a file-like object out of it
connection = server_socket.accept()[0].makefile('rb')
try:
    while True:
        # Read the length of the image as a 32-bit unsigned int. If the
        # length is zero, quit the loop
        image_len = struct.unpack('<L', connection.read(struct.calcsize('<L')))[0]
        if not image_len:
            break
        # Construct a stream to hold the image data and read the image
        # data from the connection
        image_stream = io.BytesIO()
        image_stream.write(connection.read(image_len))
        # Rewind the stream, open it as an image with opencv and do some
        # processing on it
        image_stream.seek(0)
        image = Image.open(image_stream)

        data = np.fromstring(image_stream.getvalue(), dtype=np.uint8)
        imagedisp = cv2.imdecode(data, 1)

        cv2.imshow("Frame",imagedisp)
        cv2.waitKey(1)  #imshow will not output an image if you do not use waitKey
        cv2.destroyAllWindows() #cleanup windows 
finally:
    connection.close()
    server_socket.close()

这个解决方案的结果与我在最初的问题中引用的视频相似。较大的分辨率帧会增加提要的延迟，但对于我的应用程序来说，这是可以容忍的。

首先您需要运行processStream.py，然后在Raspberry上执行createStream.py。如果这不起作用，用sudo执行python脚本

Answer 2

如果您在OpenCV试图读取它之后，但在您开始使用它之前，触摸FIFO，那么它将工作。