This is a platform for running workflows distributed on multiple machines, sometimes long-running and with lots of IO. Every single contender in the space advertises the same code, from temporal to hatchet to dagster. Because it makes sense to write it this way.
[–] dangistan202611 1 day ago
[–] whilenot-dev 1 day ago
You'd use asyncio.to_thread[0].
I also don't quite get your complaint... asyncio provides a high-level concurrency primitive with coroutines, and that's a good enough primitive for workflow engines. Even with asyncio you could fork/spawn processes easily with aiomultiprocess[1], if you'd ever want or need to.
What's your use case for having more low-level concurrency primitives (threads/processes) available, having to deal with their synchronization, and running workflows sync?
[0]: https://docs.python.org/3/library/asyncio-task.html#asyncio....
[–] dangistan202611 16 hours ago
No such thing is typically necessary. If modeled correctly, the need for synchronization is minimal.
With sync code, tasks run on different threads/interpreters/processes. Message passing via queues is used to communicate which tasks need to run. Many asyncio users seem to wholly unaware of how well it works. Go-lang doesn't even have or need async.
I actually like asyncio; I am just not a religious zealot about it as many here seem to be, also without awareness of the struggles they face with loop-hogging coroutines and multicore parallelism.
Also, the claim made that something is "better modeled as async" is wholly unqualified.