GStreamer管道+ OpenCV RTSP VideoCapture在Docker容器中不起作用

Question

我正在尝试让GStreamer + NVIDIA视频捕获在基于OpenCV PyTorch图像的Docker容器中工作。我最终不得不从源代码构建OpenCV来支持GStreamer集成，我在Dockerfile中这样做：

FROM nvcr.io/nvidia/pytorch:19.12-py3

# OpenCV custom build instructions from:
# https://medium.com/@galaktyk01/how-to-build-opencv-with-gstreamer-b11668fa09c
# https://github.com/junjuew/Docker-OpenCV-GStreamer/blob/master/opencv3-gstreamer1.0-Dockerfile

# Install base dependencies + gstreamer
RUN pip uninstall -y opencv-python
RUN apt-get update
RUN apt-get -y install build-essential
RUN apt-get -y install pkg-config
RUN apt-get install -y libgstreamer1.0-0 \
            gstreamer1.0-plugins-base \
            gstreamer1.0-plugins-good \
            gstreamer1.0-plugins-bad \
            gstreamer1.0-plugins-ugly \
            gstreamer1.0-libav \
            gstreamer1.0-doc \
            gstreamer1.0-tools \
            libgstreamer1.0-dev \
            libgstreamer-plugins-base1.0-dev \
            cmake \
            protobuf-compiler \
            libgtk2.0-dev \
            ocl-icd-opencl-dev

# Clone OpenCV repo
WORKDIR /
RUN git clone https://github.com/opencv/opencv.git
WORKDIR /opencv
RUN git checkout 4.2.0

# Build OpenCV
RUN mkdir /opencv/build
WORKDIR /opencv/build
RUN ln -s /opt/conda/lib/python3.6/site-packages/numpy/core/include/numpy /usr/include/numpy
RUN cmake -D CMAKE_BUILD_TYPE=RELEASE \
    -D INSTALL_PYTHON_EXAMPLES=ON \
    -D INSTALL_C_EXAMPLES=OFF \
    -D PYTHON_EXECUTABLE=$(which python) \
    -D BUILD_opencv_python2=OFF \
    -D CMAKE_INSTALL_PREFIX=$(python -c "import sys; print(sys.prefix)") \
    -D PYTHON3_EXECUTABLE=$(which python3) \
    -D PYTHON3_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \
    -D PYTHON3_PACKAGES_PATH=$(python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \
    -D WITH_GSTREAMER=ON \
    -D BUILD_EXAMPLES=ON ..
RUN make -j$(nproc)

# Install OpenCV
RUN make install
RUN ldconfig

 python -c 'import cv2; print(cv2.getBuildInformation());'

 /* snip */    

 Video I/O:
    DC1394:                      NO
    FFMPEG:                      NO
      avcodec:                   NO
      avformat:                  NO
      avutil:                    NO
      swscale:                   NO
      avresample:                NO
    GStreamer:                   YES (1.14.5)
    v4l/v4l2:                    YES (linux/videodev2.h)

但是，只要我尝试在Docker容器中使用带有cv2.VideoCapture()的GStreamer管道，它就会立即失败：

import cv2
video = cv2.VideoCapture('gst-launch-1.0 rtspsrc location=<<rtsp URL>> latency=0 ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink', cv2.CAP_GSTREAMER)

我得到这个“警告”(例如，错误)。我无法从RTSP源中提取帧。

[ WARN:0] global /opencv/modules/videoio/src/cap_gstreamer.cpp (713) open OpenCV | GStreamer warning: Error opening bin: unexpected reference "gst-launch-1" - ignoring
[ WARN:0] global /opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created

如果我在Docker容器之外做这件事，它就像一个护身符。还要注意，如果我从Docker容器内的命令行运行GStreamer管道，我会得到合理的输出，这与在Docker容器外运行相同的命令相同：

root:/# gst-launch-1.0 rtspsrc location=<<rtsp URL>> latency=0 ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to <<rtsp URL>>
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Redistribute latency...
Redistribute latency...

Answer 1

我不认为您应该在cv2管道描述中包含gst-launch-1.0命令行工具。

它只想要唯一的GStreamer管道，而不是控制台命令。例如：

  gst_str = ('v4l2src device=/dev/video{} ! '
    'video/x-raw, width=(int){}, height=(int){} ! '
    'videoconvert ! appsink').format(dev, width, height)
  return cv2.VideoCapture(gst_str, cv2.CAP_GSTREAMER)

因此，在您的情况下，请尝试：

import cv2
video = cv2.VideoCapture('rtspsrc location=<<rtsp URL>> latency=0 ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink', cv2.CAP_GSTREAMER)

Answer 2

我有几个建议。但请注意，我没有太多使用Gstreamer的经验。

1. 会不会与提要的分辨率有关？来自here

它与docker无关。我使用我的Android手机作为网络摄像头(使用adb-ffmpeg-v4l2loopback)，并且我刚刚使用了错误的分辨率640x360而不是1280x720 (从我的bash历史记录中选择错误的行)。

1. 你有没有试过安装opencv_contrib？这可能无关紧要，但我看到通过安装它可以修复很多opencv问题。

1. 可能会尝试添加参数format=(string)NV12，来源：OpenCV VideoCapture not working with GStreamer plugin

这里是来自Python cv2.CAP_GSTREAMER Examples的第一个示例

def open_cam_rtsp(uri, width, height, latency):
    """Open an RTSP URI (IP CAM)."""
    gst_elements = str(subprocess.check_output('gst-inspect-1.0'))
    if 'omxh264dec' in gst_elements:
        # Use hardware H.264 decoder on Jetson platforms
        gst_str = ('rtspsrc location={} latency={} ! '
                   'rtph264depay ! h264parse ! omxh264dec ! '
                   'nvvidconv ! '
                   'video/x-raw, width=(int){}, height=(int){}, '
                   'format=(string)BGRx ! videoconvert ! '
                   'appsink').format(uri, latency, width, height)
    elif 'avdec_h264' in gst_elements:
        # Otherwise try to use the software decoder 'avdec_h264'
        # NOTE: in case resizing images is necessary, try adding
        #       a 'videoscale' into the pipeline
        gst_str = ('rtspsrc location={} latency={} ! '
                   'rtph264depay ! h264parse ! avdec_h264 ! '
                   'videoconvert ! appsink').format(uri, latency)
    else:
        raise RuntimeError('H.264 decoder not found!')
    return cv2.VideoCapture(gst_str, cv2.CAP_GSTREAMER) 

它们似乎检查Gstreamer元素的subprocess.check_output('gst-inspect-1.0')输出，并相应地设置gst_str。

1. 在上面的链接中的13个示例中，我没有看到gst-launch-1.0命令。会不会是它造成了这个问题？

这个链接会有帮助吗？Docker Error Could not capture frame on Ubuntu 18.04

抱歉，这些都帮不上忙。我希望你能尽快找到解决方案！