使用OpenGL ES 2.0进行iOS视频解码

Question

我一直在尝试将.h264视频解码成opengl，但我显示的帧是黑色的。我看不到任何错误，如果我将帧从CMSampleBufferRef导出到相机胶卷中，它们是正常的。

也许是在openGL这一边吧？但是当我显示图像而不是视频时，它工作得很好，所以不知道去哪里看。

以下是初始化视频解码器的代码：

- (void)initVideo {

    glActiveTexture(GL_TEXTURE0);
    glGenTextures(1, &_glTextureHook);

    NSURL * url = [NSURL URLWithString:[self.videoMedia getFilePath]];
    self.videoAsset = [[AVURLAsset alloc] initWithURL:url options:NULL];

    dispatch_semaphore_t sema = dispatch_semaphore_create(0);

    [self.videoAsset loadValuesAsynchronouslyForKeys:@[@"tracks"] completionHandler:^{

        self.videoTrack = [self.videoAsset tracksWithMediaType:AVMediaTypeVideo][0];

        NSString *key = (NSString *) kCVPixelBufferPixelFormatTypeKey;
        NSNumber *value = @(kCVPixelFormatType_32BGRA);
        NSDictionary *settings = @{key : value};

        self.outputVideoTrackOuput = [[AVAssetReaderTrackOutput alloc]
                                      initWithTrack:self.videoTrack outputSettings:settings];

        self.assetReader = [[AVAssetReader alloc] initWithAsset:self.videoAsset error:nil];
        [self.assetReader addOutput:self.outputVideoTrackOuput];
        [self.assetReader startReading];

        dispatch_semaphore_signal(sema);
    }];

    dispatch_semaphore_wait(sema, DISPATCH_TIME_FOREVER);
}

以及检索opengl纹理的代码，视频的每一帧：

- (void)play {

    if (self.assetReader.status == AVAssetReaderStatusReading) {
        CMSampleBufferRef sampleBuffer = [self.outputVideoTrackOuput copyNextSampleBuffer];
        if(sampleBuffer) {
            CVImageBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);

            if(pixelBuffer) {
                CVPixelBufferLockBaseAddress(pixelBuffer, 0);

                glBindTexture(GL_TEXTURE_2D, _glTextureHook);
                glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 1920, 1080, 0, GL_BGRA_EXT, GL_UNSIGNED_BYTE, CVPixelBufferGetBaseAddress(pixelBuffer));
                CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);
            }

            CFRelease(sampleBuffer);
        }
    }
}

顶点着色器和碎片对于图像和视频是相同的(适用于图像)。我看到的唯一区别是在glTexImage2D中，两种格式都是GL_RGBA格式的图像。

我确保视频解码器的_glTextureHook被很好地发送到着色器管理器，上下文在线程上被激活等。

这是片段着色器的代码(它非常基本，顶点着色器也是如此)：

precision lowp float;

uniform sampler2D Texture;

varying vec4 DestinationColor;
varying vec2 TexCoordOut;

void main() {
    gl_FragColor = DestinationColor * texture2D(Texture, TexCoordOut);
}

Answer 1

我只是错过了创建纹理后的那些线条：

glBindTexture(GL_TEXTURE_2D, _glTextureHook);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);