In principle I would love to have something like this working for arbitrarily large ETL flows; ie. write the logic you want and it tries to figure out a good distributed algorithm to get the outputs. At the moment flowing data through the cluster is the biggest bottleneck. We generally assume that action inputs & outputs themselves are pretty small at the boundary even if their containerized context is quite large.

Where did you hit the most snags with caching?

We have two things, both custom: bytecode (what our VM uses) and IR (what Python is parsed into); there's no stable ABI yet since we practically don't need it, yet, IR is specified as a protobuf schema and bytecode is a specified via rust-native types in our framework for building instruction sets.

In principle, where WASM would go is it would be our compilation target instead of our own bytecode. We've thought about using WASM and, independent of that, about using LLVM compiler facilities. They don't have to go together, and, arguably, tapping into LLVM would be more interesting than WASM (I'm mentioning them together one of the intuitive perks from using WASM would seemingly be using WASM compilers too).

The thing is, we would, most likely, not have too much benefits from using LLVM or existing compilers from various languages into WASM; we compile things in a slightly different manner, and, I suspect, it would be possible to re-use some of the existing optimizations for control graph - but not the low-level ones.

As for WASM - we use a stackless VM, and WASM has an execution stack as a requirement - so we can't seemingly use WASM. That said, in practice we could use the same technique rust uses for compiling futures, and run on a WASM VM with a single stack frame holding our state machine and just doing everything in it. That's pretty close conceptually to how we have implemented in our custom VM now.

What this means is when you have an async fn that you spawn in a loop where each iteration also spawns a ton of async fns which in turn call actions - there's no separate bookkeeping to do parallel to the execution itself. No replays (unlike other tools like Temporal), meaning the thing just goes, as-is, and when it crashes it can load the state and resume from where it left off - without the need to match the state of the interpreter to the persisted state of execution graph. We just store the real, actual runtime state.

- Each LLM call and tool call becomes a checkpoint. Without durable execution you're pretty much stuck doing db saves/restores if you detect that this agent might have a previous context window. Agent frameworks are fundamentally just harnesses around the underlying model: you `while loop` until some server gives you some payload back, you add that to your context window list, and you keep going until it terminates. Durable execution moves that loop into the workflow body itself (a quick example here https://www.waymark.sh/python/tutorials/building-a-from-scra...).

- Long waits. Sometimes you do want your agent to wait for an arbitrary amount of time (ie. wait for this compilation job to finish - look again in 5mins, or maybe 5 hours). Durable execution makes sure that sleep timer will wake up regardless of what happens in the system state.

- Easier monitorability. Instead of having to sift through interleaved logs to figure out what was going on, you can see every large function call with a clean input/output contract. Useful especially in tracing down raised Exceptions and the data flow that it took to get there. Good for agents and regular code alike.

Of course, if your system is largely idempotent or your agents are mostly classifiers that are working in short internals - a simple db save/restore can work perfectly well too.

I like this one, as in the context of workloads like these, going with anything more complex than a Postgres is overkill.

> This is an appeal to authors of such posts: please give a motivation that provides the necessary background to understand the problem, and ideally do so in a way that a technical but non specialist reader can understand.

I agree with this. For Show HN posts, this would really help broaden the audience as well as lift up the audience’s knowledge and enthusiasm.

Consider a microservices architecture (for the sake of example), where a user sign up needs to create an account in three services: auth-svc, avatars-svc and emails-svc.

Your options to do it are either an event-sourcing architecture where your services have to orchestrate together via somewhat-awkward message-passing, or write another reconciling service where you write the logic more-or-less directly, but with a ton of code-overhead to make the execution reliable.

Waymark would help with the second approach, because it soaks in all the durable execution complexity, and gives you a way to just write the logic code and handle the edge cases - with a fraction of the code and systems you'd need otherwise.

I'd even dare to say - a tool like waymark, the second approach is pretty much impractical altogether, hence why the event sourcing microserivce architectures dominate today.

But, yet again, that's just one example of where waymark would be useful. If you can write the code where you can split the orchestration logic from the execution of the individual side effects - then with waymark your orchestration code can be just the logic you need, without the extra steps to persist the progress - thus making it easier to reason and mental model simpler.

As for CIL - we should add it at the "What should we build next?" section on the website :)

With the way our current Python workers are implemented you'd have we'd have to place the sync code onto threads manually.

We could make it happen though, if there is a demand for sync code; we can allow actions to be non-async, and from that run them in sync mode - either with an sdk-level offload to threads off-of async reactor, or on a separate worker pools that run fork-per-action. From the perspective of our VM, we don't really have a difference there...

As for async - background jobs are better modeled as async because of their natural parallelism across nodes imo. Logic in actions themselves are free to be as synchronous as you want. Why the preference?

This is a platform for running workflows distributed on multiple machines, sometimes long-running and with lots of IO. Every single contender in the space advertises the same code, from temporal to hatchet to dagster. Because it makes sense to write it this way.

When we compile a workflow into bytecode we get an executable; we store it and assign it an executable id. In-flight workflows are technically VM instances, and they always pin to a concrete executable id. When the VM goes out of residency (i.e. evicted from a node that executes it and unloaded from memory) we will eventually gc the executable bytecode from memory if it's not used by any VM for a while - but it's kept in postgres so VM can be revived.

There are ways to make workflows upgradeable - but don't have this yet. Ideally, we'd want to build it once we have enough feedback on how people want to do this - cause it's really about choosing the trade-offs more than anything else.