Pepper Live双向音频流错误

Question

我正在尝试建立Pepper的平板电脑和我的PC之间的实时音频通信。我正在使用Gstreamer来建立它。从Pepper的麦克风到PC的音频正常工作，但似乎没有音频从我的PC到Pepper的平板电脑。我做错了什么？

PC端：

audio_pipeline = Gst.Pipeline('audio_pipeline')

audio_udpsrc = Gst.ElementFactory.make('udpsrc', None)
audio_udpsrc.set_property('port', args.audio)

audio_caps = Gst.caps_from_string('application/x-rtp,media=(string)audio, clock-rate=(int)44100, width=16, height=16, encoding-name=(string)L16, encoding-params=(string)1, channels=(int)1, channel-positions=(int)1, payload=(int)96')
audio_filter = Gst.ElementFactory.make('capsfilter', None)
audio_filter.set_property('caps',audio_caps)

audio_depay = Gst.ElementFactory.make('rtpL16depay', None)
audio_convert = Gst.ElementFactory.make('audioconvert', None)

audio_sink = Gst.ElementFactory.make('alsasink', None)
audio_sink.set_property('sync',False)

audio_pipeline.add(audio_udpsrc,audio_filter,audio_depay,audio_convert,audio_sink)
audio_udpsrc.link(audio_filter)
audio_filter.link(audio_depay)
audio_depay.link(audio_convert)
audio_convert.link(audio_sink)

机器人(Choregraphe)：

audio_src = gst.element_factory_make('autoaudiosrc')

audio_convert = gst.element_factory_make('audioconvert')
audio_caps = gst.caps_from_string('audio/x-raw-int,channels=1,depth=16,width=16,rate=44100')
audio_filter = gst.element_factory_make('capsfilter')
audio_filter.set_property('caps',audio_caps)
# audio_enc = gst.element_factory_make('mad')
audio_pay = gst.element_factory_make('rtpL16pay')
audio_udp = gst.element_factory_make('udpsink')
audio_udp.set_property('host',user_ip)
audio_udp.set_property('port',int(user_audio_port))

self.audio_pipeline.add(audio_src,audio_convert,audio_filter,audio_pay,audio_udp)
gst.element_link_many(audio_src,audio_convert,audio_filter,audio_pay,audio_udp)

或

机器人端(Python SDK)：

GObject.threads_init()
Gst.init(None)

audio_pipeline = Gst.Pipeline('audio_pipeline')

audio_src = Gst.ElementFactory.make('autoaudiosrc')

audio_convert = Gst.ElementFactory.make('audioconvert')
audio_caps = Gst.ElementFactory.make('audio/x-raw-int,channels=2,depth=16,width=16,rate=44100')
audio_filter = Gst.ElementFactory.make('capsfilter')
audio_filter.set_property('caps',audio_caps)
audio_pay = Gst.ElementFactory.make('rtpL16pay')
audio_udp = Gst.ElementFactory.make('udpsink')
audio_udp.set_property('host',user_ip)
audio_udp.set_property('port',int(user_audio_port))

audio_pipeline.add(audio_src,audio_convert,audio_filter,audio_pay,audio_udp)
audio_src.link(audio_convert)
audio_convert.link(audio_filter)
audio_filter.link(audio_pay)
audio_pay.link(audio_udp)

audio_pipeline.set_state(Gst.State.PLAYING)

计算机的麦克风对Pepper：

audio_port = 80

s_audio_pipeline = Gst.Pipeline('s_audio_pipeline')

s_audio_src = Gst.ElementFactory.make('autoaudiosrc')

s_audio_convert = Gst.ElementFactory.make('audioconvert')
s_audio_caps = Gst.ElementFactory.make('audio/x-raw-int,channels=2,depth=16,width=16,rate=44100')
s_audio_filter = Gst.ElementFactory.make('capsfilter')
s_audio_filter.set_property('caps',audio_caps)
s_audio_pay = Gst.ElementFactory.make('rtpL16pay')
s_audio_udp = Gst.ElementFactory.make('udpsink')
s_audio_udp.set_property('host',ip)
s_audio_udp.set_property('port',int(audio_port))

s_audio_pipeline.add(s_audio_src,s_audio_convert,s_audio_filter,s_audio_pay,s_audio_udp)
s_audio_src.link(s_audio_convert)
s_audio_convert.link(s_audio_filter)
s_audio_filter.link(s_audio_pay)
s_audio_pay.link(s_audio_udp)

Pepper接收：

audio = 80

    r_audio_pipeline = Gst.Pipeline('r_audio_pipeline')

    #defining audio pipeline attributes
    r_audio_udpsrc = Gst.ElementFactory.make('udpsrc', None)
    r_audio_udpsrc.set_property('port', audio)

    r_audio_caps = Gst.caps_from_string('application/x-rtp,media=(string)audio, clock-rate=(int)44100, width=16, height=16, encoding-name=(string)L16, encoding-params=(string)1, channels=(int)2, format=(string)S16LE, channel-positions=(int)1, payload=(int)96')
    r_audio_filter = Gst.ElementFactory.make('capsfilter', None)
    r_audio_filter.set_property('caps',r_audio_caps)

    r_audio_depay = Gst.ElementFactory.make('rtpL16depay', None)
    r_audio_convert = Gst.ElementFactory.make('audioconvert', None)

    r_audio_sink = Gst.ElementFactory.make('alsasink', None)
    r_audio_sink.set_property('sync',False)

    #linking the various attributes
    r_audio_pipeline.add(r_audio_udpsrc,r_audio_filter,r_audio_depay,r_audio_convert,r_audio_sink)
    r_audio_udpsrc.link(r_audio_filter)
    r_audio_filter.link(r_audio_depay)
    r_audio_depay.link(r_audio_convert)
    r_audio_convert.link(r_audio_sink)

    r_audio_pipeline.set_state(Gst.State.PLAYING)

我想可能是胡椒的接收端口号有问题...我尝试了不同的端口号(包括9559)，但似乎都不起作用。源ID是否错误？

是否可以在同一管道中运行双向流？

我看了一下其他的库，比如ffmpeg和PyAudio，但是我不能使用任何直播的方法。

Answer 1

确保在机器人上运行Python脚本。

还有，你有没有运行GMainLoop？

Choregraphe行为在NAOqi中运行，并且NAOqi已经在后台运行GMainLoop。也许这就是您的独立脚本中缺少的内容。

最后，在您的代码片段中没有显示用于将PC的音频传输到网络，也没有从网络传输到Pepper的扬声器的代码。