无缝连接AVAssets

Question

我有一些简单的AVFoundation代码可以将一堆长达4秒钟的mp4文件连接在一起，如下所示：

func
compose(parts inParts: [Part], progress inProgress: (CMTime) -> ())
    -> AVAsset?
{
    guard
        let composition = self.composition,
        let videoTrack = composition.addMutableTrack(withMediaType: .video, preferredTrackID: kCMPersistentTrackID_Invalid),
        let audioTrack = composition.addMutableTrack(withMediaType: .audio, preferredTrackID: kCMPersistentTrackID_Invalid)
    else
    {
        debugLog("Unable to create tracks for composition")
        return nil
    }

    do
    {
        var time = CMTime.zero
        for p in inParts
        {
            let asset = AVURLAsset(url: p.path.url)
            if let track = asset.tracks(withMediaType: .video).first
            {
                try videoTrack.insertTimeRange(CMTimeRange(start: .zero, duration: asset.duration), of: track, at: time)
            }
            if let track = asset.tracks(withMediaType: .audio).first
            {
                try audioTrack.insertTimeRange(CMTimeRange(start: .zero, duration: asset.duration), of: track, at: time)
            }

            time = CMTimeAdd(time, asset.duration)
            inProgress(time)
        }
    }

    catch (let e)
    {
        debugLog("Error adding clips: \(e)")
        return nil
    }

    return composition
}

不幸的是，每隔四秒你就能听到断音片刻，这表明这不是一个完全无缝连接。我能做些什么来改善这一点吗？

解决方案

感谢NoHalfBits下面的出色回答，我已经用以下方法更新了上面的循环，它运行得非常好：

        for p in inParts
        {
            let asset = AVURLAsset(url: p.path.url)

            //  It’s possible (and turns out, it’s often the case with UniFi NVR recordings)
            //  for the audio and video tracks to be of slightly different start time
            //  and duration. Find the intersection of the two tracks’ time ranges and
            //  use that range when inserting both tracks into the composition…

            //  Calculate the common time range between the video and audio tracks…

            let sourceVideo = asset.tracks(withMediaType: .video).first
            let sourceAudio = asset.tracks(withMediaType: .audio).first
            var commonTimeRange = CMTimeRange.zero
            if sourceVideo != nil && sourceAudio != nil
            {
                commonTimeRange = CMTimeRangeGetIntersection(sourceVideo!.timeRange, otherRange: sourceAudio!.timeRange)
            }
            else if sourceVideo != nil
            {
                commonTimeRange = sourceVideo!.timeRange
            }
            else if sourceAudio != nil
            {
                commonTimeRange = sourceAudio!.timeRange
            }
            else
            {
                //  There’s neither video nor audio tracks, bail…

                continue
            }

            debugLog("Asset duration: \(asset.duration.seconds), common time range duration: \(commonTimeRange.duration.seconds)")

            //  Insert the video and audio tracks…

            if sourceVideo != nil
            {
                try videoTrack.insertTimeRange(commonTimeRange, of: sourceVideo!, at: time)
            }
            if sourceAudio != nil
            {
                try audioTrack.insertTimeRange(commonTimeRange, of: sourceAudio!, at: time)
            }

            time = time + commonTimeRange.duration
            inProgress(time)
        }

Answer 1

在mp4容器中，每个轨道都可以有自己的启动时间和持续时间。特别是在录制的材料中，音频和视频音轨的时间范围略有不同并不少见(在CMTimeRangeShow(track.timeRange)附近插入一些insertTimeRange来查看这一点)。

要克服这一问题，不要盲目地从CMTime.zero插入和整个资产的持续时间(所有轨道的最大结束时间)：

• 获取源音频和视频轨道的timeRange
• 从这些计算出公共时间范围(CMTimeRangeGetIntersection为您做了这件事)
• 在将段从源轨道插入到目标轨道时使用公共时间范围
• 按公共时间范围的持续时间递增time