RFC: Introduce Rust as second language for writing modules

This RFC PR introduce the Rust programming language as second language for writing modules.

The goal of this initiative is to be able to use a safer language than C that still as comparable (or better) performance than C/C++.

Current implementation

The support is added by using cargo (the Rust package manager and build system) and the integration adds 3 crates (Rust "library"):

1. vlcrs-core-sys: The bindings crate, it automatically generate from the C headers the corresponding Rust definitions
2. vlcrs-core: The safe-abstraction crate, where all the Rust safe abstractions lives (stream_t, vlc_tick_t, ...)
3. vlcrs-core-macros: The proc-macro crate, where all the procedural macro lives (like the module! macro)

The integration is the current build system is not ideal but works. The main challenge is that libtool doesn't support Rust nor does it have a have to simply add support for another system. The integration is therefor done by emulating part of it's works by generating via a custom script libtool_cargo.sh a libtool convenience library, that is then used normally by the rest of the system.

A meson integration has also been done and requires less "hack".

The overall Rust support can be enable with --enable-rust in configure (that internally binds to HAVE_RUST) or with -Drust=true with meson.

There is currently no support for anything other than x86_64, this can be fixed relatively simply but isn't yet done;

Overall the current implementation only supports (for now) stream_filter and demux but every other kind of module are planned to be supported.

Problematics

No va_list support in Rust

Rust doesn't support at all va_list and berley supports ... (only for calling not implementing). However many callbacks works by using a single va_list callback, to make it able to safely implement those callbacks the Rust side use a typed controls, this works well for Rust but adds some complexity for both Rust and C side because the typed controls must be kept synchronized. This complexity is inevitable and manageable due to the very low changes in those controls.

Fallible allocation

Rust currently has limited support for fallible allocations. Works is currently being done to improve and fix the situation but it's still lacking. This RFC currently doesn't do anything to mitigate this.

Internet connection

Status: Partially fixed (building offline works, packaging still need to be done)

Cargo as the package manager needs to retrieve the dependencies from the internet. This should not be a problem when developing but when packaging the application in a tar.gz or other we should be able to build without. This isn't a priority and could be added later without hopefully to much problems but is nevertheless a issue to consider.

Proof of concept

The RFC includes a POC that this works and is a viable long-term strategy.

inflate-rs is a module that has the same capability as the C version inflate.c, it's is build using pure Rust including for the decoding part by using the flate2 Rust crate. See loic/vlc@216b4e4a for a complete view of the module.

Status

• Automatic generation of Rust bindings
• Integration into the build system
  • Support other arch than x86_64
  • Support offline building (required for tarball)
• Modules support
  • stream_filter
  • demux
  • intf
  • ...
• VLC Subsystem
  • stream_t
  • demux_t
  • vlc_fourcc_t
  • frame_t/block_t
  • tick_t
  • date_t
  • vlc_url_t
  • input_item_t
  • ...

Thanks

Thanks for your contribution!

When all of the following conditions are fulfilled, your MergeRequest will be reviewed by the Team:

• the check pipeline passes
• the MR is considered as 'mergeable' by gitlab

You can find more details about the acceptance process here.

This message was automatically generated by homer-bot.

added MRStatus::Reviewable label

added MRStatus::InReview label and removed MRStatus::Reviewable label

added 14 commits

• 24a4b56a - vlcrs-core: add automatic VLC_FOURCC generation
• 8d17fa73 - vlcrs-core: generate bindings for all FOURCC
• 15e342bd - vlcrs-core: add abstraction for vlc_frame_t
• dd151824 - vlcrs-core: add abstraction for vlc_block_t
• 8574412d - vlcrs-core: add abstraction for stream_t usage
• b03e7863 - vlcrs-core: add dummy input_item_t abstraction
• 8a532395 - vlcrs-core: add stream read interface
• 566617cd - vlcrs-core: add typed stream_t control callbacks
• f5708188 - vlcrs-core: add needed re-export for Rust module macro
• 026f59ba - vlcrs-core: add stream_filter module abstraction
• d1db8acf - vlcrs: add macro crates
• 59e570dc - vlcrs-macros: add Rust module macro
• 5741782d - vlcrs: add infra for build and linking Rust modules
• c398ac03 - inflate-rs: add first Rust module

Compare with previous version

resolved all threads

added 2 commits

• 8a40b65c - vlcrs: add infra for build and linking Rust modules
• cb264d40 - inflate-rs: add first Rust module

Compare with previous version