如何在ROS中使用OpenCV-Python

Question

在ROS环境中使用yolo和opencv-python，，我想在ROS中使用yolo和Opencv-python来控制摄像机和实现目标检测。现在我已经知道如何在Windows中运行yolo，但我不知道如何在ROS中运行它。如何在ROS中移植我的代码？

Answer 1

ROS是一个用于轻松组合不同库的框架，这些库提供的接口不是定义为头文件，而是由“启动文件”(xml脚本)调用的“节点”。这意味着您希望在视频/摄像机提要上运行Yolo，但希望它与其他库或代码接口。如果你不需要，那你就不需要罗斯。

目前，ROS(v1)在Ubuntu上运行得最好。它可以在本地或在virtualbox中工作。ROS2支持windows，但是如果您对它有问题，那就不同了。

要为您的python代码创建一个ROS节点，首先将其放在一个单独的类/模块中；ROS节点应该在“真正的代码”和通信之间只有接口样板。假设您使用usb_cam_node获取相机馈送，数据将在“topic”<camera_name>/image [sensor_msgs/Image]上发布，其中<camera_name>是usb_cam_node参数。主题就像所有ROS节点之间的全局变量，由Subscriber读取(带有回调)并由Publisher发布。

然后你必须决定从它中发布什么。因为它是yolo，也许你需要每个检测的边界框。有一堆预定义的ROS消息(这些是‘主题’的(静态和强类型)“类型”)。一个是geometry_msgs/PolygonStamped，它允许您指定一个盒子的角，并给它盖章。

下面是一些示例代码，摘自维基

# yolo_boxes_node.py
import rospy
from std_msgs.msg import Header
from geometry_msgs.msg import PolygonStamped, Point32
from sensor_msgs.msg import Image
# This is your custom yolo code
import my_yolo as yolo # Assuming a method like as follows:
# yolo.evaluate(img_frame) -> 
#   boxes ([x,y,width,hight] list), confidences (list), classids (list)

# Subscribers
#     img_sub (sensor_msgs/Image): "webcam/image" #Comment: we document a name for the sub, the type, and the default topic for it

# Publishers
#     boxes_pub (geometry_msgs/PolygonStamped): "webcam/yolo/boxes"

# Publishers
boxes_pub = None

# Parameters
frequency = 100.0 # Hz

# Global Variables
img_frame = None
header = None

def img_callback(data): # data of type Image
    global img_frame
    global header
    img_frame = data.data
    header = data.header

def timer_callback(event): # This is to process data at a fixed rate, perhaps different from camera framerate
    # Convert img_frame somehow if needed
    if img_frame is None or boxes_pub is None:
        return
    boxes, confidences, classids = yolo.evaluate(img_frame)
    for b in boxes:
        msg = PolygonStamped()
        msg.header = header # You could use the header differently
        msg.polygon.points.append(Point32(x=b[0],y=b[1]))
        msg.polygon.points.append(Point32(x=b[0]+b[2],y=b[1]))
        msg.polygon.points.append(Point32(x=b[0],y=b[1]+b[3]))
        msg.polygon.points.append(Point32(x=b[0]+b[2],y=b[1]+b[3]))
        boxes_pub.publish(msg)

# In your main function, you subscribe to topics
def yolo_boxes_node():
    # Init ROS
    rospy.init_node('yolo_boxes_node', anonymous=True)

    # Parameters
    if rospy.has_param('~frequency'):
        frequency = rospy.get_param('~frequency')

    # Subscribers
    # Each subscriber has the topic, topic type, AND the callback!
    rospy.Subscriber('webcam/image', Image, img_callback)
    # Rarely/never need to hold onto the object with a variable: 
    #     img_sub = rospy.Subscriber(...)
    rospy.Timer(rospy.Duration(1.0/frequency), timer_callback)

    # Publishers
    boxes_pub = rospy.Publisher('webcam/yolo/boxes', PolygonStamped, queue_size = 100)
    # queue_size increases as buffer for msgs; if you have 1000s of boxes, might need bigger

    # spin() simply keeps python from exiting until this node is stopped
    # This is an infinite loop, the only code that gets ran are callbacks
    rospy.spin()
    # NO CODE GOES AFTER THIS, NONE! USE TIMER CALLBACKS!
    # unless you need to clean up resource allocation, close(), etc when program dies

if __name__ == '__main__':
    yolo_boxes_node()

因此，xml启动文件示例可能是：

<?xml version="1.0"?>
<!-- my_main_program.launch -->
<launch>
  <!--
    Pub: <camera_name>/image [sensor_msgs/Image]
  -->
  <node name="usb_cam_node" type="usb_cam_node" pkg="usb_cam" output="screen" restart="true">
    <param name="camera_name" value="webcam"/>
    <param name="video_device" value="/dev/video0"/>
  </node>

  <!--
    img_sub: webcam/image [sensor_msgs/Image]
    boxes_pub: webcam/yolo/boxes [geometry_msgs/PolygonStamped]
  -->
  <node name="yolo_boxes_node" type="yolo_boxes_node" pkg="my_pkg" output="screen">
    <param name="frequency" value="30.0"/>
  </node>
</launch>