Durable Python workflows with Pynenc and Pynmon
Most task queues are excellent at starting work. They are less opinionated about what happens when one unit of work is really a business process.
What search algorithms look like when every step is a task
Please don’t try this at home.
Declarative event-driven tasks in Python
Most distributed task frameworks give you a queue. Put work in, workers pull it out. That covers the easy half of background processing.
Why pynenc uses an invocation state machine
Pynenc models every task invocation as a finite-state machine. The file pynenc/invocation/status.py declares the invocation states, the allowed transitions between them, ownership rules, and dedicated recovery states.
Per-account task concurrency without a lock service
Many background jobs call an external system on behalf of separate accounts, tenants, or installations. The external system allows parallel calls across different accounts, but it does not allow two calls for the same account to run at the same time.
Distribute a Python function without changing callers
You have a Python function that processes one item. You call it in a loop. The list grows, and the loop slows down because each call waits on I/O: an API request, a database query, a scrape, an embedding, a model call....
Recover Python tasks after a worker crash
I wanted a small failure test: start a few long-running tasks, kill the worker, then see whether the work still finishes.