Launching wasm.rs: a collection of crates, a community
About a week ago I got seriously bothered to figure out whether it is possible to run BPXE (a workflow automation engine I am developing) in a browser. I mean, theoretically it was always a possibility (Rust can target wasm32-unknown-unknown after all), but how does this translate to having something more than an in-and-out library that takes the heavy lifting away from JavaScript? So I set out on a journey.
It’s been a frantic week, suffice to say! The problem is that this engine is more than just a library, it’s rather a long-running engine disguised as a library so that it can be precisely customized to do what needs to be done. It uses Rust’s async a lot. It also has an embedded scripting capability (👋 Rhai!). It has a lot of dependencies.
As a result of my journey, I’ve published a couple of crates under the umbrella of wasm.rs.
First one is really simple. I just wanted a better way to see my debugging output. So I wrote a drop-in replacement for dbg! macro that works both in the browser and other targets. I do realize that WebAssembly is no longer just for “web” and I’ll see if anything needs to be done about other WebAssembly targets. Nevertheless, please welcome wasm-rs-dbg!
Next one was really important. I’ve tried running BPXE and its own tests on a bunch of async executors (from Tokio, futures crate, async-executor, etc.) to no avail. They all kept breaking when I was running them in the browser. Mostly due to unavailability of synchronization primitives in the default standard library. Even though some of these were explicitly “local” (one thread) executors.
So I ended up developing my own executor, which is (at this time, since there’s no good multi-threading support for WebAssembly in Rust) strictly single-threaded. I am not yet 100% confident it’s perfect, but it’s been able to run all of BPXE. In the future, should better support for multi-threading materialize, I’ll be happy to grow it in that direction as well. You can find it in the wasm-rs-async-executor crate.
Lastly, but not least importantly, I had to figure out how to make long-running processes work well. As we all know, the main thread of the browser is not a place to do such things unless you’re doing it with callbacks, or async/await in JavaScript. I’ve tried to integrate with this from the Rust side, but it was rather a futile approach. So I set out on a different path.
I decided that I will run main BPXE engine on a worker thread. But how do we communicate between the main thread and a worker? In JavaScript world we simply postMessage between threads and have handlers ready. But I am running a full-time async executor in Rust already. I can’t have an onmessage handler.
However, there’s SharedArrayBuffer (that is currently coming out of limbo caused by the Spectre bug) and Atomics — these JavaScript primitives allow for sharing memory between threads. After searching for an appropriate crate, I realized I had to do this myself.
As a result, I developed the first version of wasm-rs-shared-channel which takes advantage of these primitives and gives you a SPSC (single publisher single consumer) channel so that your main thread can send messages to a worker waiting for messages in a thread (which can then post_message responses back or even use another channel if the other side can poll or wait). Check out the quick-and-dirty example of how it can be used.
There’s more to be done. For instance, I have not found a great way to launch a worker and pass a WebAssembly.Module to it when using wasm-bindgen. I had to settle on a small JavaScript shim and building two versions of bindings (for bundling and for web/no-module deployment to be used by workers). There’s no support for Webpack 5. I am pretty sure there are more issues just waiting around the corner…
Throughout this week I’ve found that it is not trivial to figure the state of things, best practices and ideas when it comes to running Rust on a WebAssembly target. There’s some rigidity in the tools, there’s a lack of explicit, tested support of this target. There’s a general lack of knowledge (I’ve been told that tokio::sync channel primitives should probably be patched to work; so far I used them straight from the source, and did seem to work — after I had my own executor). Suffice to say, it’s frustrating.
I want to improve the status quo. I want to meet other enthusiasts that are discovering or working on this part of the ecosystem. There’s a “#wg-wasm” channel on Rust’s Discord, but I didn’t get a sense of a community there. It’s a “work group” channel after all.
So, I’ve launched a Discord “server” for wasm.rs — with the express intent for it to become a place to share experience and collaborate on making working with WebAssembly in Rust easy and pleasant.
See you there! I hope this can be of help to others.
