Rust ffi questions

[jacobrosenthal]

Hey all

Was asking @mattgreg about future rust directions and design decisions and he referred me here.

I see there's active Rust commits so Im curious what your current direction is. I saw you've recently had some recently abandoned work around no_std rust api. Is your goal no_std no alloc? As a no_std rust embedded fan myself, that is the gold standard though I know it has implementation difficulties and tends to make your std api worse if you're not careful. Tony in the Rust Crypto org is doing good work keeping no_std apis and Whitequark's managed is a pattern I've seen for sharing heapless and heapfull apis. I'd have to dig around more but I think I've seen some wire protocol implementations just ditch std::io the biggest std leaking culprit for their own no_std implementation that gets used on both the no_std and std code pretty seemlessly.

Is there any discussion you can link me to around the choice of katai for your cross language schema/wire protocol? It seems an interesting choice given it doesn't seem to get you a seralizer side. Digging around I found an old pr attempting to bring rust support but the author appears to have abandoned the work advising to go for nom binary parsing instead. Sadly many of the wire protocols you might have heard of don't have much better no_std rust support. Even Google's rust implementation of their schemaless zero copy flexbuffer just used std and thus allocates and is thus its not zero copy!

[jdspdx]

Hi @jacobrosenthal!

We've been revisiting the design of our Rust crates, and I thought you might be interested in some updates related to your comment.

As you mentioned, having a primary goal of supporting no_std without alloc is very challenging, but also a useful and valid target. Part of our crate re-organization was to split our support into three features:

1. no_std without alloc
2. no_std with alloc
3. std

The Ockam repository now has a crate hierarchy which allows higher level features to benefit from std, but also make lower level libraries available to a broader ecosystem. Our goal is to use the minimum support necessary, within reason. For example, our initial releases of the std Ockam Node will depend on tokio 1.0. At the same time, our Ockam Vault crate remains no_std, no alloc compatible.

Kaitai Structs was used to document the wire format initially, but as you noted it is not a good fit for rust serialization. We have recently settled on the BARE protocol, which has great serde support.