使用MediaStreamSource和MediaTranscoder进行TimedMetadata编码

Question

我正在构建一个C# .NET核心应用程序，它收集用户交互数据并将其与视频提要一起存储。我打算利用MPEG容器中的定时元数据来存储交互数据和h264编码的视频数据。

在使用MediaTranscoder进行代码转换操作期间，似乎可以对元数据流进行编码。我使用的代码是从screen recorder example修改的。它的工作方式是创建一个支持元数据的自定义MediaEncodingProfile和一个同时包含VideoStreamDescriptor和TimedMetadataStreamDescriptor的MediaStreamSource，然后再使用它们对MediaTranscoder执行编码。

public class Encoder
{
    private MediaEncodingProfile GetEncodingProfile()
    {
        var profile = new MediaEncodingProfile();
        var containerEncoding = new ContainerEncodingProperties
        {
            Subtype = MediaEncodingSubtypes.Mpeg4
        };

        var videoEncoding = new VideoEncodingProperties
        {
            Subtype = MediaEncodingSubtypes.H264,
            Width = configuration.Width,
            Height = configuration.Height,
            Bitrate = configuration.BitsPerSecond,
            FrameRate = { Denominator = 1, Numerator = configuration.FramesPerSecond },
            PixelAspectRatio = { Denominator = 1, Numerator = 1 }
        };

        profile.Container = containerEncoding;
        profile.Video = videoEncoding;

        return profile;
    }

    private Tuple<VideoStreamDescriptor, TimedMetadataStreamDescriptor> GetStreamDescriptors()
    {
        var videoEncoding = VideoEncodingProperties.CreateUncompressed(MediaEncodingSubtypes.Bgra8,
                                                                       configuration.InputWidth, configuration.InputHeight);

        var videoStreamDescriptor = new VideoStreamDescriptor(videoEncoding);

        var metadataEncoding = new TimedMetadataEncodingProperties
        {
            Subtype = "{36002D6F-4D0D-4FD7-8538-5680DA4ED58D}"
        };

        byte[] streamFormatData = GetMetadataStreamFormatData(); // This just sets some arbitrary bytes

        metadataEncoding.SetFormatUserData(streamFormatData);

        var metadataStreamDescriptor = new TimedMetadataStreamDescriptor(metadataEncoding);

        return new Tuple<VideoStreamDescriptor, TimedMetadataStreamDescriptor>(
                videoStreamDescriptor, metadataStreamDescriptor);
   }

   private MediaStreamSource GetMediaStreamSource(IMediaStreamDescriptor videoStreamDescriptor,
                                                  IMediaStreamDescriptor metadataStreamDescriptor)
   {
        var mediaStreamSource = new MediaStreamSource(videoStreamDescriptor, metadataStreamDescriptor)
        {
            BufferTime = TimeSpan.FromSeconds(0)
        };

        mediaStreamSource.Starting += OnStart;
        mediaStreamSource.SampleRequested += OnSampleRequested;

        return mediaStreamSource;
    }

    private void OnStart(MediaStreamSource sender, MediaStreamSourceStartingEventArgs args)
    {
        // Intentionally omitted
    }

    private void OnSampleRequested(MediaStreamSource sender, MediaStreamSourceSampleRequestedEventArgs args)
    {
        // This only gets called for the video stream, not the metadata stream
    }

    private MediaTranscoder GetTranscoder()
    {
        var transcoder = new MediaTranscoder { HardwareAccelerationEnabled = true };
        return transcoder;
    }

    public async Task TranscodeAsync()
    {
        var transcoder = GetTranscoder();

        var (videoStreamDescriptor, metadataStreamDescriptor) = GetStreamDescriptors();

        var mediaStreamSource = GetMediaStreamSource(videoStreamDescriptor, metadataStreamDescriptor);
        var encodingProfile = GetEncodingProfile();
        await using var destinationFile = File.Open(configuration.FilePath, FileMode.Create);

        var prepareTranscodeResult = await transcoder.PrepareMediaStreamSourceTranscodeAsync(
                    mediaStreamSource, destinationFile.AsRandomAccessStream(), encodingProfile);
        await prepareTranscodeResult.TranscodeAsync().AsTask();
    }
}

我面临的问题是，不会为定时元数据流引发SampleRequested事件，而只会引发视频流事件。在测试期间，我用音频流替换了计时的元数据流，并且视频流和音频流都正确地引发了SampleRequested事件。我怀疑可能有一种不同的方法将数据添加到定时元数据流中，可能使用TimedMetadataTrack和DataCue，但我的努力到目前为止被证明是不成功的。

在使用MediaTranscoder (以及可能的MediaStreamSource)时，在编码期间将仅在每个样本基础上可用的定时元数据添加到流的正确方法是什么？

Answer 1

在与工程师沟通后，现在这个问题有了结果。

虽然MediaTranscoder确实支持计时元数据流，但当在MF转换引擎上设置配置文件时，它将被删除以仅支持1个音频流和1个视频流。所以使用MediaTranscoder是行不通的。

相关文档已更新：Transcode media files