检测MediaStreamTrack是否为黑色/空白

Question

我正在和peerjs一起创建videochat。

我使用以下功能切换摄像头(开/关)：

function toggleCamera() {
    localStream.getVideoTracks()[0].enabled = !(localStream.getVideoTracks()[0].enabled);
}

调用此函数后，视频变黑，接收器仅显示黑屏(按预期工作)。现在我想要检测黑屏/空白屏幕，这样我就可以向用户显示一些消息或图标，表明摄像头已禁用，并且没有流。

我如何检测到这一点？

Answer 1

过了一段时间，我终于找到了解决方案：

var previousBytes = 0;
var previousTS = 0;
var currentBytes = 0;
var currentTS = 0;

// peer - new Peer()
// stream - local camera stream (received from navigator.mediaDevices.getUserMedia(constraints))
let connection = peer.call(peerID, stream);

// peerConnection - reference to RTCPeerConnection (https://peerjs.com/docs.html#dataconnection-peerconnection)
connection.peerConnection.getStats(null).then(stats => {
    stats.forEach(report => {
        if (report.type === "inbound-rtp") {

            currentBytes = report.bytesReceived;
            currentTS = report.timestamp;

            if (previousBytes == 0) {
                previousBytes = currentBytes;
                previousTS = currentTS;
                return;
            }

            console.log({ previousBytes })
            console.log({ currentBytes })

            var deltaBytes = currentBytes - previousBytes;
            var deltaTS = currentTS - previousTS;

            console.log("Delta: " + (deltaBytes / deltaTS) + " kB/s")
            previousBytes = currentBytes;
            previousTS = currentTS;

        }
    });
});

这段代码实际上在每秒都会被调用的函数中。当摄像头打开但没有覆盖时，deltaBytes在100到250之间，当摄像头关闭(编程)或覆盖(用餐巾或其他东西)时，摄像头流是黑色/空白的，deltaBytes是1.5-3kbps。打开摄像头后，deltaBytes出现峰值，达到500kbps左右。

这是简短的控制台日志：

    124.52747252747253 kB/s
    202.213 kB/s
    194.64764764764766 kB/s
    15.313 kB/s (this is where camera is covered)
    11.823823823823824 kB/s
    11.862137862137862 kB/s
    2.164 kB/s
    2.005 kB/s
    2.078078078078078 kB/s
    1.99 kB/s
    2.059 kB/s
    1.992992992992993 kB/s
    159.89810189810188 kB/s (uncovered camera)
    502.669 kB/s
    314.7927927927928 kB/s
    255.0909090909091 kB/s
    220.042 kB/s
    213.46353646353646 kB/s

编辑:所以最后我照@Philipp Hancke说的做了。我创建了主连接，从页面加载到用户关闭它都是打开的。通过这个连接，我发送命令来启动视频通话，取消视频会话，打开/关闭摄像头，...然后在另一端，我解析这些命令并执行函数。

function sendMutedMicCommand() { masterConnection.send(`${commands.MutedMic}`); }
function sendUnmutedMicCommand() { masterConnection.send(`${commands.UnmutedMic}`); }
function sendPromptVideoCallCommand() { masterConnection.send(`${commands.PromptVideoCall}`); }
function sendAcceptVideoCallCommand() { masterConnection.send(`${commands.AcceptVideoCall}`); }
function sendDeclineVideoCallCommand() { masterConnection.send(`${commands.DeclineVideoCall}`); }

Function which handles data:
function handleData(data) {
    let actionType = data;
    switch (actionType) {
        case commands.MutedMic: ShowMuteIconOnReceivingVideo(true); break;
        case commands.UnmutedMic: ShowMuteIconOnReceivingVideo(false); break;
        case commands.PromptVideoCall: showVideoCallModal(); break;
        case commands.AcceptVideoCall: startVideoConference(); break;
        case commands.DeclineVideoCall: showDeclinedCallAlert(); break;
        default: break;
    }
}


const commands = {
    MutedMic: "mutedMic",
    UnmutedMic: "unmutedMic",
    PromptVideoCall: "promptVideoCall",
    AcceptVideoCall: "acceptVideoCall",
    DeclineVideoCall: "declineVideoCall",
}

然后当我收到mutedMic命令时，我会显示带有交叉麦克风的图标。当我收到AcceptVideoCall命令时，我创建了另一个具有随机ID的对等体videoCallPeer，然后将其发送到另一端。然后，另一端使用随机ID创建另一个对等体，并使用接收到的ID发起视频会话。

Answer 2

常见的方法是发送信令消息(通过正常路径或数据通道)。轮询getStats以检测黑帧是一种有效的方法，但计算成本较高。