Douglas profile picture

Douglas Amoo-Sargon

Software Engineer at RetailEdge Software Labs

Full-stack software engineer with 5+ years of experience designing, building, and shipping production systems across healthcare, ISP management, and e-commerce. Rooted in a biomedical engineering background, I bridge traditional engineering discipline with modern software craft to build applications that solve real problems in the real world.
I'm particularly drawn to event-driven architectures and durable workflow design — systems that stay resilient when things inevitably go wrong. Outside of building, I enjoy teaching and sharing what I've learned with others.

Abstract

When Step 3 Fails: Reliable Multi-Step Workflows in Celery Using the Saga Pattern

In real systems, “step 3 failed” is not an edge case, it is normal. A workflow might charge a card, reserve inventory, and then fail calling a third-party shipping provider. Celery can retry tasks and report failures, but it cannot undo the earlier side effects for you. Teams typically respond with ad-hoc compensation code scattered across tasks, which becomes brittle, hard to test, and difficult to reason about as the workflow graph grows. The saga pattern is an established solution: each step defines a **compensation** (a semantic undo), and if any step fails, compensations run automatically in reverse order. In this talk, I will: - Show the problem concretely using a familiar order workflow (payment → inventory → shipping) and where Celery primitives fall short for rollback coordination. - Explain the saga pattern (orchestration vs choreography) and what “compensation” means in practice (refund is not a database rollback). - Demonstrate how to implement sagas for Celery workflows, using **`celery-saga` (a lightweight library I built)** as the concrete reference implementation: define steps + compensations, execute successfully, then intentionally fail and observe reverse-order compensation with step-level visibility. - Cover the hard parts production systems hit: parallel steps, partial success, idempotency (webhook retries), and fast CI testing without a broker. - Close with guidance on when *not* to use sagas (when retries, idempotency, or single-DB transactions are enough). Attendees will leave with a clear mental model for compensation-based rollback and a practical structure they can apply to their own Celery workflows immediately.

Short Talk Intermediate