Skip to content

ocvr: Added original content video rate plugins

Jochen Henneberg requested to merge jh-hsd/gst-plugins-rs:ocvr into main

Original content video rate detection

This is a set of two plugins that can be used to minimize host load when using framegrabber devices or for desktop frame grabbing. E. g. if a set-top-box provides video it will typically do this at 60Hz, but if movies a played the movie content is often recorded at 24Hz so we grab, encode and stream and decode (on the other side) a lot of duplicate frames without need. To avoid this, these plugins try to detect the original content video rate within the grabbed frames and drop duplicates. In addition, we can also downsample 60Hz -> 30Hz and 50Hz -> 25Hz. We need the original content rate detection for this as well or we will end up with stuttering video, e. g. in case of 24Hz in 60Hz.

There are two plugins:

  • ocvrhint: Tries to detect the original content video rate from the encoder output by correlating frame sizes with pre-defined vectors for each combination of input rate and original content rate. On match a custom upstream event is sent to ocvrctrl as a hint. Ocvrhint will then stop working (because frames might be dropped now and the correlation will not work anymore) until ocvrctrl sends a custom downstream event that no sync on the given framerate happens anymore.

  • ocvrctrl: This plugin does accurate or fuzzy (needed for watermarked video) compare of frames and uses this to confirm the framerate hint from ocvrhint. It can also try to detect the framerate itself but at much higher cost. Once a framerate sync has been found all duplicate frames are dropped, a new caps event is sent (if not disabled from properties) and the timestamps and duration are adjusted. If the framerate monitoring detects an unexpected mismatch ocvrctrl may try to repeat syncing if configured of give up and return to the capture rate.

A typical simplified pipeline will look like this: v4l2src ! ocvrctrl ! vaapih265enc ! h265parse ! ocvrhint ! rtph265pay ! udpsink

Please let me know what you think.

Merge request reports