Java - Xuggle -获取帧的最佳方法

Question

我使用xuggle已经有一周了，我写了一个方法来获取视频的帧，但是如果视频很长，这个方法需要太多的时间：

public static void getFrameBySec(IContainer container, int videoStreamId, IStreamCoder videoCoder, IVideoResampler resampler, double sec) 
{ 
    BufferedImage javaImage = new BufferedImage(videoCoder.getWidth(), videoCoder.getHeight(), BufferedImage.TYPE_3BYTE_BGR); 
    IConverter converter = ConverterFactory.createConverter(javaImage, IPixelFormat.Type.BGR24); 
    IPacket packet = IPacket.make(); 
    while(container.readNextPacket(packet) >= 0) 
    { 
        if (packet.getStreamIndex() == videoStreamId) 
        { 
            IVideoPicture picture = IVideoPicture.make(videoCoder.getPixelType(), videoCoder.getWidth(), videoCoder.getHeight()); 
            int offset = 0; 
            while(offset < packet.getSize()) 
            { 
                int bytesDecoded = videoCoder.decodeVideo(picture, packet, offset); 
                if (bytesDecoded < 0) 
                    throw new RuntimeException("got error decoding video"); 
                offset += bytesDecoded; 
                if (picture.isComplete()) 
                { 
                    IVideoPicture newPic = picture; 
                    if (resampler != null) 
                    { 
                        newPic = IVideoPicture.make(resampler.getOutputPixelFormat(), picture.getWidth(), picture.getHeight()); 

                        if (resampler.resample(newPic, picture) < 0) 
                            throw new RuntimeException("could not resample video from"); 
                    } 
                    if (newPic.getPixelType() != IPixelFormat.Type.BGR24) 
                            throw new RuntimeException("could not decode video as RGB 32 bit data in"); 

                    javaImage = converter.toImage(newPic); 
                    try 
                    { 
                        double seconds = ((double)picture.getPts()) / Global.DEFAULT_PTS_PER_SECOND; 
                        if (seconds >= sec && seconds <= (sec +(Global.DEFAULT_PTS_PER_SECOND ))) 
                        { 

                            File file = new File(Config.MULTIMEDIA_PATH, "frame_" + sec + ".png"); 
                            ImageIO.write(javaImage, "png", file); 
                            System.out.printf("at elapsed time of %6.3f seconds wrote: %s \n", seconds, file); 
                            return; 
                        } 
                    } 
                    catch (Exception e) 
                    { 
                        e.printStackTrace(); 
                    } 
                } 
            } 
        } 
        else 
        { 
            // This packet isn't part of our video stream, so we just 
            // silently drop it. 
        } 
    } 
    converter.delete(); 
} 

你知道更好的方法吗？

Answer 1

通过阅读你的代码，我看到了一些可以优化的地方。

第一次读取整个文件时，创建一个字节偏移量和秒数的索引。然后，该函数可以从给定的秒数查找字节偏移量，您可以在该偏移量处解码视频，并执行代码的其余部分。

另一种选择是使用您的方法，每次读取整个文件，而不是调用所有的重采样器、newPic和java图像转换器代码，首先检查秒数是否匹配。如果是，则将图像转换为要显示的新图片。

所以

if(picture.isComplete()){

try { 

  double seconds = ((double)picture.getPts()) / Global.DEFAULT_PTS_PER_SECOND; 
  if (seconds >= sec && seconds <= (sec +(Global.DEFAULT_PTS_PER_SECOND ))) 
  {
    Resample image
    Convert Image
    Do File stuff
  }

Answer 2

使用seekKeyFrame选项。您可以使用此函数查找视频文件中的任何时间(时间以毫秒为单位)。

double timeBase = 0;
int videoStreamId = -1;

private void seekToMs(IContainer container, long timeMs) {
    if(videoStreamId == -1) {
        for(int i = 0; i < container.getNumStreams(); i++) {
            IStream stream = container.getStream(i);
            IStreamCoder coder = stream.getStreamCoder();
            if (coder.getCodecType() == ICodec.Type.CODEC_TYPE_VIDEO) {
                videoStreamId = i;
                timeBase = stream.getTimeBase().getDouble();
                break;
            }
        }
    }

    long seekTo = (long) (timeMs/1000.0/timeBase);
    container.seekKeyFrame(videoStreamId, seekTo, IContainer.SEEK_FLAG_BACKWARDS);
}

然后，您可以使用经典的while(container.readNextPacket(packet) >= 0)方法将图像保存到文件中。

注意：它不会寻求精确的时间，而是近似的时间，所以你仍然需要检查数据包(但当然比以前少得多)。