Skip to content

How to rustify libmultiprocess ? #56

Open
@ariard

Description

I would like to enable libmultiprocess across different languages where ProxyClient is in a C++ process and ProxyServer is in a Rust process (or vice-versa), while still keeping the interface description reconciliation safe and straightforward.

After a bit of research, I think there is 2 different approaches

  • libmultiprocess.rs : rewriting libmultiprocess in rust, including a new rust-mpgen binary relying on Cap'n Proto rust compiler plugin
  • rust-libmultiprocess.rs : writing Rust bindings for libmultiprocess API (src/mp/proxy.h, src/mp/util.h) and a Rust FFI generator for the artifacts of mpgen

In the case of the first approach, I think API consistency across process would be guaranteed by consuming the data definition (*.capnp). If the calculator.rs/calculator.capnp doesn't match a error should happen at Rust process compilation. This approach comes also
with the benefit that if libmultiprocess features want to be used across Rust only client-server processes, the C++ code doesn't come as a dependency. That said a new dependency is added on the rust compiler plugin and it's a lot of development code and maintenance
to guarantee behavior consistency and feature compatibility of the both library.

For this reason, I think the second approach is wiser.

AFAICT, you have 6 files output by mpgen:

  • calculator.capnp.c++ (generated by the capnp proto compiler though called by mpgen)
  • calculator.capnp.h
  • calculator.proxy-client.c++
  • calculator.proxy-server.c++
  • calculator.proxy-types.c++
  • calculator.proxy.h

I think they're mostly stub code relying on code in include/mp/proxy-types.h and as such should be okay to generater FFI interface for them. I've a doubt if there is a need to cover the calculator.capnp.c++, as I'm not sure if will be consumed by the Rust code anyway.
W.r.t to libmultiprocess API, it's nice to have coverage for proxy.cpp/util.cpp and as such rebuild the process orchestration API available in Bitcoin Core's ipc/* on the Rust-side to have consistent process behavior.

Once you have *.rs output by a Rust FFI generator you can integrate them in your Rust build system. Of course, the C++ generated code must still be build by a C++ compiler, and the outcome of which linked in your Rust binary, otherwise you'll have missing symboles throw at you by the linker.

Here a graphical illustration of the second rust-libmultiprocess approach.



             ---|                   | - calculator.capnp.c++                 
        API     |                   | - calculator.capnp.h                   
                |-----> mpgen ----> | - calculator.capnp.proxy-client.c++   --------------------------> c++-compiler ----------------------------------------------------------------> cpp.o ----|
        data    |                   | - calculator.capnp.proxy-server.c++                                                                                                                        | 
             ---|                   | - calculator.capnp.proxy-types.c++                                                                                                                         |-----> linker ----> calculator.bin
                                    | - calculator.capnp.proxy.h            --------|                               | - calculator.capnp.rs                                                      |
                                                                                    |                               | - calculator.capnp.proxy-client.rs                                         |
                                                                                    |                               | - calculator.capnp.proxy-server.rs  ---------> rust-compiler --> rust.o ---|
                                                                                    |---------> rust-mpgen -------> | - calculator.capnp.proxy-types.rs                   ^
                                                                                                                    | - calculator.capnp.proxy.h.rs                       |
                                                                                                                                                                          |
                                                                                                                                                                          |
                                                                                                                                                proxy.rs -----------------|
                                                                                                                                                util.rs

Do you think the approach advocated is reasonable ? Feel free to raise all relevant points I'm missing.

This is in the context of the Altnet project where I would like to offer the choice between C++/Rust to the pluggable transports daemon writers. Starting by myself :)

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions