Skip to content
VECTOR WIREAI INTELLIGENCE
UTC
Home

Airbnb open-sources Skipper, its embedded durable-workflow engine for JVM services

Apache 2.0 licensed, ships with core engine, MySQL and SQLite adapters, admin UI, and test utilities.

1Skipper, Airbnb’s embedded workflow engine, is now open sourcemedium.com/airbnb-engineeringHigh11·10 18:01Z

Airbnb released Skipper, the embedded workflow engine it runs in production across multiple internal teams, as an open-source project under the Apache 2.0 license on November 101. The code, documentation, and runnable examples are available on GitHub.

Skipper lets developers express multi-step business processes as Java or Kotlin classes, persist their progress, and recover from crashes and transient failures — all inside the host service, with no separate orchestration cluster. On the normal execution path, a workflow runs directly in the service process, backed by persisted state and scheduled recovery work, eliminating round-trips to an external orchestration service between actions.

What ships in the release

The open-source package includes the core engine, MySQL and SQLite adapters, test utilities, and an admin UI. It also includes Skipper State Machine, a Kotlin DSL for processes described as states, events, and transitions. Example projects cover Java, Kotlin, Spring Boot, and Dropwizard.

The programming model exposes signals, queries, retries, compensation, and workflow-evolution primitives. Skipper supports Saga-style compensation when a process needs explicit recovery actions for previously completed steps. Developers define which errors are retryable and choose retry policies; permanent failures and exhausted retries are surfaced for handling.

Execution and determinism model

Skipper requires deterministic workflow code. When it resumes an interrupted workflow, it re-executes the workflow method from the top; actions with durably persisted checkpoints return their saved results rather than re-running side effects. By default, checkpoint writes are batched until execution completes or pauses for a wait or retry, though actions can request immediate checkpointing to narrow the replay window. Side effects remain at-least-once because a downstream effect and its checkpoint are not one atomic operation; each action must make its own side effects idempotent.

The `waitUntil` primitive hibernates a workflow without holding a thread, and a redeploy during the wait does not discard the persisted workflow. A `WorkflowTest` harness provides an isolated in-memory runtime with controllable time, exercising waits, signals, retries, and completion while supplying fakes for downstream collaborators.

Planned additions

The team plans to add more storage plugins, including Postgres and DynamoDB, and to improve tooling for workflow evolution and replay debugging. The post notes that open-source contributions around automated compatibility checks, migration assistance, and clearer replay visualization could help developers work through replay-based engine problems.

ANALYSIS Skipper occupies a distinct point in the durable-execution space: it embeds inside the application process rather than requiring a standalone orchestration cluster, trading operational overhead for tighter coupling to the host service's deployment lifecycle. The initial storage-adapter set — MySQL and SQLite — limits adoption outside those databases until the planned Postgres and DynamoDB plugins arrive.

Ricardo Gamba and Andriy Sergiyenko authored the release post.

Was this useful?

The day’s vector, in your inbox.

Vector Wire Daily: one dispatch each publishing day.