使用AVCaptureOutput的CIDetector

Question

我正在尝试用AVFoundation和CIDetector从视频输入中一帧一帧地检测微笑。在captureOutput中使用时，CIDetector在准确性方面非常不一致。同样的面部表情可以根据光线、离相机的距离和照片中的其他面孔快速地从微笑到不微笑。有没有一种方法可以在不降低CIDetector速度的情况下减少它的抖动？

@interface CameraViewController () {
    BOOL smiling;
}

...

- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection {

    CVPixelBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    CFDictionaryRef attachments = CMCopyDictionaryOfAttachments(kCFAllocatorDefault, sampleBuffer, kCMAttachmentMode_ShouldPropagate);
    CIImage *ciImage = [[CIImage alloc] initWithCVPixelBuffer:pixelBuffer options:(__bridge NSDictionary *)attachments];

    if (attachments)
        CFRelease(attachments);

    //CIImage translates all the pixels -90 deg; set orientation taking this into account

    NSNumber *orientation;
    switch ([UIDevice currentDevice].orientation) {
        case UIDeviceOrientationPortrait:
            orientation = @6;
            break;
        case UIDeviceOrientationPortraitUpsideDown:
            orientation = @8;
            break;
        case UIDeviceOrientationLandscapeLeft:
            orientation = @3;
            break;
        case UIDeviceOrientationLandscapeRight:
            orientation = @1;
            break;
        default:
            orientation = @6;
            break;
    }

    NSArray *faces = [[[SmileDetector sharedDetector] detector] featuresInImage:ciImage
                                                                        options:@{CIDetectorSmile: @(YES), CIDetectorImageOrientation:orientation}];
    smiling = NO;
    for (CIFaceFeature* face in faces)
        if (face.hasSmile) {
            smiling = YES;
            break;
        }
}

SmileDetector是在其他ViewControllers中使用相同的CIDetector实例的单例。我将探测器精度设置为Low，因为High太慢了。

- (id)init {
    if (self = [super init]) {
        detector = [CIDetector detectorOfType:CIDetectorTypeFace
                                      context:[CIContext contextWithOptions:nil]
                                      options:@{CIDetectorAccuracy:CIDetectorAccuracyLow}];
    }
    return self;
}

Answer 1

我目前正在处理同样的问题，我发现在CIDetector工作之前对帧进行预处理以稳定对比度和亮度的效果非常好。我的项目是静态帧，我不知道这将如何工作与视频帧…