FsAppTransmitter / dummy call in nullprpl
David Woodhouse
dwmw2 at infradead.org
Fri Feb 2 08:31:15 EST 2018
On Mon, 2017-12-11 at 10:13 +0000, David Woodhouse wrote:
> On Sun, 2017-12-10 at 21:02 +0000, David Woodhouse wrote:
> > I'm thinking of following the precedent set by FsShmTransmitter and
> > abusing the host and username fields of the FsCandidate, except not
> > with filenames as FsShmTransmitter does it, but with pipeline
> > descriptions to be handled by gst_parse_launch().
>
> OK, so that much works, but I can't used named elements; those have to
> have been created in the *same* pipeline description. So this code
> doesn't work because there's no way to make it find "MyTestSrc" AFAICT:
So I got this working. I can create the named element in the
FsAppTransmitter pipeline, then I can find it again from the outside
with gst_bin_get_by_name().
My own protocol code can now feed Opus frames up from the GstAppSrc,
and receive them from the GstAppSink, and I can actually have a
conversation. Yay!
However... I need to do a bunch of stuff for myself. There are
timestamp fields in the wire protocol, which I need to manually keep in
sync with the timestamps on the GstBuffers, and I also need a jitter
buffer.
Now that now my life isn't *entirely* dominated by CPU vulnerability
work I'm starting to take a look again. It occurred to me that I'd
actually be better off faking RTP, and just copying my protocol's
timestamps into the corresponding field of an RTP packet. That way I
can avoid reinventing the wheel.
Before I do the step of faking RTP myself, I started with purely adding
rtpopuspay/rtpopusdepay in my pipelines and switching to
fsrtpconference. And now I'm back to "I am clueless and I don't know
why this doesn't work mode" and asking for help again... :)
I'm still working on getting permission to actually publish this PRPL
under LGPL, but below is a pseudo-patch which shows what was working,
and what I've done. I can probably do this in the dummy PRPL as I did
before, if anyone really wants to be able to test it locally.
The GStreamer pipeline is at http://david.woodhou.se/conf.svg
When I press a key it does correctly send DTMF. But although the audio
from the microphone *looks* like it's hooked up, nothing is arriving.
And the incoming audio doesn't seem to be hooked up at all. And I never
see 'farstream-recv-codec-changed'.
Any further clues would be, as before, most gratefully received.
static void on_audio_state(ChimeCall *call, ChimeAudioState audio_state, struct chime_chat *chat)
{
purple_debug(PURPLE_DEBUG_INFO, "chime", "Audio state %d\n", audio_state);
const gchar *name = chime_call_get_alert_body(chat->call);
if (audio_state == CHIME_AUDIO_STATE_AUDIO_MUTED && chat->media) {
purple_media_stream_info(chat->media, PURPLE_MEDIA_INFO_MUTE, "chime", name, FALSE);
} else if (audio_state == CHIME_AUDIO_STATE_AUDIO && chat->media) {
purple_media_stream_info(chat->media, PURPLE_MEDIA_INFO_UNMUTE, "chime", name, FALSE);
} else if (audio_state == CHIME_AUDIO_STATE_AUDIO && !chat->media) {
PurpleMediaManager *mgr = purple_media_manager_get();
chat->media = purple_media_manager_create_media(purple_media_manager_get(),
chat->conv->account,
- "fsrawconference",
+ "fsrtpconference",
name,
TRUE);
if (chat->media) {
- const gchar *caps = "audio/x-raw,format=(string)S16LE,layout=(string)interleaved,channels=(int)1";
+ const gchar *caps = "application/x-rtp,media=(string)audio,payload=(int)96,encoding-name=(string)OPUS";
gboolean r = purple_media_add_stream(chat->media, "chime", name,
PURPLE_MEDIA_AUDIO, TRUE,
"app", 0, NULL);
gchar *srcname = g_strdup_printf("chime_src_%p", call);
gchar *sinkname = g_strdup_printf("chime_sink_%p", call);
/* Without the capsfilter (even though there's another capsfilter almost immediately after it in the pipeline, it doesn't work */
- gchar *srcpipe = g_strdup_printf("appsrc name=%s format=time caps=audio/x-opus,channel-mapping-family=0 ! opusdec ! capsfilter caps=%s",
- srcname, caps);
- gchar *sinkpipe = g_strdup_printf("opusenc bitrate=16000 bitrate-type=vbr ! appsink name=%s async=false", sinkname);
+ gchar *srcpipe = g_strdup_printf("appsrc name=%s format=time caps=audio/x-opus,channel-mapping-family=0 ! rtpopuspay ! capsfilter caps=%s", srcname, caps);
+ gchar *sinkpipe = g_strdup_printf("rtpopusdepay ! appsink name=%s async=false", sinkname);
PurpleMediaCandidate *cand =
purple_media_candidate_new(NULL, 1,
PURPLE_MEDIA_CANDIDATE_TYPE_HOST,
PURPLE_MEDIA_NETWORK_PROTOCOL_UDP,
sinkpipe, 0);
g_object_set(cand, "username", srcpipe, NULL);
g_free(sinkpipe);
g_free(srcpipe);
GList *cands = g_list_append (NULL, cand);
GList *codecs = g_list_append(NULL,
- purple_media_codec_new(1, caps, PURPLE_MEDIA_AUDIO, 0));
+ purple_media_codec_new(96, "OPUS", PURPLE_MEDIA_AUDIO, 0));
+// g_object_set(codecs->data, "channels", 1, NULL);
+ purple_media_codec_add_optional_parameter(codecs->data, "farstream-recv-profile", "rtpopusdepay ! opusdec");
+ purple_media_codec_add_optional_parameter(codecs->data, "farstream-send-profile", "opusenc ! rtpopuspay");
+// purple_media_codec_add_optional_parameter(codecs->data, "sprop-stereo", "0");
+// purple_media_codec_add_optional_parameter(codecs->data, "stereo", "0");
+ purple_media_get_local_candidates(chat->media, "chime", name);
purple_media_add_remote_candidates(chat->media, "chime", name, cands);
+ purple_media_set_send_codec(chat->media, "chime", codecs->data);
purple_media_set_remote_codecs(chat->media, "chime", name, codecs);
GstElement *pipeline = purple_media_manager_get_pipeline(mgr);
GstElement *appsrc = gst_bin_get_by_name(GST_BIN(pipeline), srcname);
GstElement *appsink = gst_bin_get_by_name(GST_BIN(pipeline), sinkname);
g_free(srcname);
g_free(sinkname);
gst_app_src_set_size(GST_APP_SRC(appsrc), -1);
gst_app_src_set_max_bytes(GST_APP_SRC(appsrc), 100);
gst_app_src_set_stream_type(GST_APP_SRC(appsrc), GST_APP_STREAM_TYPE_STREAM);
// gst_base_src_set_live(GST_BASE_SRC(appsrc), TRUE);
chime_call_install_gst_app_callbacks(chat->call, GST_APP_SRC(appsrc), GST_APP_SINK(appsink));
g_signal_connect(chat->media, "state-changed", G_CALLBACK(call_media_changed), chat);
g_signal_connect(chat->media, "stream-info", G_CALLBACK(call_stream_info), chat);
purple_media_stream_info(chat->media, PURPLE_MEDIA_INFO_ACCEPT, "chime", name, FALSE);
GST_DEBUG_BIN_TO_DOT_FILE(GST_BIN(purple_media_manager_get_pipeline(mgr)), GST_DEBUG_GRAPH_SHOW_ALL, "chime conference graph");
}
}
}
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 5213 bytes
Desc: not available
URL: <https://pidgin.im/pipermail/devel/attachments/20180202/00a5b218/attachment.bin>
More information about the Devel
mailing list