Fintech Development

Resumable Batch Settlement Job Prompt

Structure a batch settlement job that can safely resume after a crash mid-run without double-processing or missing any transactions.

Anselm Fowel / Public collection27 Sept 2026

Your prompt

The prompt

You are a fintech engineer who has operated overnight batch settlement jobs processing large transaction volumes, and has been paged at 3am for a job that crashed halfway through and needed to resume without corrupting the day's settlement.

My settlement job runs in [LANGUAGE/FRAMEWORK] on a [SCHEDULE: NIGHTLY / HOURLY] basis, processes [VOLUME] transactions from [DATABASE], and produces [OUTPUT: A SETTLEMENT FILE FOR A BANKING PARTNER / PAYOUT INSTRUCTIONS / BOTH]. It currently [DESCRIBE CURRENT STATE: HAS NO RESUME CAPABILITY / RESTARTS FROM SCRATCH ON FAILURE].

First, design how the job tracks its own progress so it can resume rather than restart: a checkpoint or watermark strategy that records which records have been fully processed, and why this must be committed transactionally alongside the actual settlement action rather than as a separate best-effort update.

Next, design how the job guarantees each transaction is settled exactly once even if it crashes immediately after processing a record but before the checkpoint is saved. Specify whether the design tolerates at-least-once processing with a downstream idempotency check, or requires true exactly-once semantics, and how that's achieved given [OUTPUT] may be sent to an external party with no idempotency guarantee of its own.

Then design chunking and ordering: whether transactions must be processed in a strict order preserved on resume (e.g., because of running balance dependencies) or can be safely parallelized and resumed out of order, and how the job partitions [VOLUME] transactions into manageable batches so a single crash doesn't require a full reprocess. Also design observability: what's logged or emitted at each checkpoint so an on-call engineer can tell, at 3am, exactly how far the job got and whether it's safe to simply restart it.

Finally, output the job's core loop with checkpointing logic in [LANGUAGE/FRAMEWORK] with comments, followed by a runbook-style checklist for what an on-call engineer checks and does when this job fails mid-run.

2,045 characters

Your prompt

Recent articles