Skip to content

Draft: Create Rust bindings to GStreamer Vulkan architecture

Marijn Suijten requested to merge MarijnS95/gstreamer-rs:vulkan into main

Closes #389

These are my Vulkan Rust bindings that have been in the making for well over 3 years. This is still a draft while working out the best way to map various structures, but is already being used by external contributors for a few months (#389).

I have previously tried to merge these changes piecemeal, but that approach was rejected in favour of having the complete, "finished" bindings MR'd in one go, so here they are.

I will dedicate my efforts towards cleaning up these bindings, as well as compare them with a recent alternative that was strangely oblivious to my efforts over the past 3 years. Understandably there is some bit-rot in this branch (with the amount of churn in common files like lib.rs/Gir.toml/Cargo.toml that gstreamer-rs sees) but the bindings still compile and the example code is still running.

As usual upstream improvements could have simplified the bindings, but this is where we are now.

Just like GL this includes a vkupload example (still a very crude "testing ground" while experimenting how GStreamer's Vulkan bindings work), showcasing how to use an AppSink to copy + present a Vulkan image. Unfortunately ways to work with Vulkan barriers and pipeline sync flags in GStreamer are yet to be clearly documented, and seem to have changed with the addition of VulkanOperation in 1.24. Specifically, when pulling a Vulkan texture out of an AppSink we need to transition it to a TRANSFER_SRC (for this specific example/test-case) with another barrier, and likewise no clear clue how to update the "current" layout and semaphore-to-wait-on before "recycling" the texture back into the "pool" (assuming there is one).

Relevant MRs and discussions that have contributed towards these bindings, or are required for these bindings:

I will look into ways to reduce dependencies, though ultimately improving upstream documentation/annotations and regenerating bindings with newer spec definitions is a perfect way to get more correct and complete bindings.

Edited by Marijn Suijten

Merge request reports