从Detectron2中提取对象特征

Question

我对目标检测真的很陌生，如果这个问题看起来太明显的话，我很抱歉。

我在FasterRCNN上有一个经过训练的detectron2模型，用于检测对象，并且我正在尝试提取每个检测对象的特征，以用于模型预测的输出。可用的教程似乎是为了寻找ROI，并做出新的预测，以及它们的框和特性。我有来自推理的框，我只需要提取每个框的特征。我添加了下面的代码。谢谢

# Preprocessing
images = predictor.model.preprocess_image(inputs)  # don't forget to preprocess
# Run Backbone Res1-Res4
features = predictor.model.backbone(images.tensor)  # set of cnn features
        
#get proposed boxes + rois + features + predictions
# Run RoI head for each proposal (RoI Pooling + Res5)
proposal_boxes = [x.proposal_boxes for x in proposals]
features = [features[f] for f in predictor.model.roi_heads.in_features]
proposal_rois = predictor.model.roi_heads.box_pooler(features, proposal_boxes)
box_features = predictor.model.roi_heads.box_head(proposal_rois)
predictions = predictor.model.roi_heads.box_predictor(box_features)#found here: https://detectron2.readthedocs.io/_modules/detectron2/modeling/roi_heads/roi_heads.html        
pred_instances, pred_inds = predictor.model.roi_heads.box_predictor.inference(predictions, proposals)
pred_instances = predictor.model.roi_heads.forward_with_given_boxes(features, pred_instances)


# output boxes, masks, scores, etc
pred_instances = predictor.model._postprocess(pred_instances, inputs, images.image_sizes)  # scale box to orig size
# features of the proposed boxes
feats = box_features[pred_inds]
proposal_boxes = proposals[0].proposal_boxes[pred_inds]

Answer 1

这个问题已经在这个问题上得到了很好的讨论：https://github.com/facebookresearch/detectron2/issues/5。

文档还解释了如何实现这一目标。