AppSinkStream is difficult to use
I'm using Gstreamer to add a watermark to a video and then upload it to a video streaming service using a pipeline that looks something like this:
appsrc ! decodebin ! rsvgoverlay ! encodebin ! appsink
where the appsrc
is being pushed data from a stream using AppSrcSink
and then AppSinkStream
is pushing data to the streaming service via reqwest
.
However, I found that AppSinkStream
will immediately yield None
unless the appsink
is actively streaming data (by looking at AppSink::is_eos()
), which makes it difficult to use because I have to build the Gstreamer pipeline and start pushing to it to get that to start, but I'm doing something like this:
// build pipeline
// ...
// rsvgoverlay and its following elements aren't linked until `decodebin` emits the `pad-added` signal
tokio::try_join! {
appsrc.sink().send_all(&mut source_stream),
reqwest_client.post("some-url").body(appsink.stream().map(/* convert `Sample` to `bytes::Bytes` */)).send()
}
So the result is that the reqwest
call uploads 0 bytes while the appsrc
element is still being fed data.
I found a partial solution, to use tokio::sync::Notify
to make the upload task wait until triggered:
let data_available = Arc::new(tokio::sync::Notify::new());
tokio::try_join! {
appsrc.sink().send_all(&mut source_stream),
async {
data_available.notified().await;
reqwest_client.post("some-url").body(appsink.stream().map(/* convert `Sample` to `bytes::Bytes` */)).send()
}
}
However, I can't find a suitable signal or callback to watch to know when to call data_available.notify()
.
Connecting to the pad-linked
signal on appsink
's sink
pad doesn't quite get me there as data hasn't started flowing yet.
I thought of connecting to the notify signal for the eos
property since I'd expect that to have a true->false transition as data starts being queued, but the code of appsink
doesn't emit that signal internally.
I can set the new-sample
callback but that would override what's set by AppSink::stream()
, although I could just implement my own stream using a channel which wouldn't be too hard and is probably what I'll end up doing.
I could also set emit-signals: true
connect to the new-sample
signal but I don't want to take the performance hit (although maybe I could set emit-signals: false
in the handler so it's only called once? would that break things?).
Any advice here? I don't know what I would change about AppSinkStream
to make this easier. I think to fix it for my case would require it to always return Pending
until it actually gets an eos
callback, but would that be a breaking change to its behavior?