Engineering
You Sent the Charge and the Connection Died. Did It Charge?
By Clasify Team · · 5 min read
Every payment integration eventually meets the same question, and it is not a question about money. You sent a charge request. The connection died before the response came back. Did it charge?
You cannot know from your side. Retrying might double-charge a customer. Not retrying might leave a paid subscription unrenewed. And whichever you choose, you will choose it again on the next timeout, and the one after that.
This is the part of payments that is genuinely hard. The card form is not hard. Reconciling a distributed system with somebody else's ledger, across crashes and retries and duplicate webhooks, is.
Decide before you charge, not after
The reflex is to record the payment when it succeeds. That is exactly backwards, because the moment you need a record is the moment you do not know whether it succeeded.
So the record comes first. Before any renewal is attempted, we create an anchor row keyed by a deterministic idempotency key — deterministic meaning it is derived from the subscription and the billing period, so the same renewal always produces the same key no matter which process computes it or when.
That key carries a unique constraint, and the uniqueness is doing the real work. Creating the anchor is an atomic claim. If a previous pass already created it — because it crashed halfway, or a duplicate job fired, or a retry arrived — the insert conflicts, and we fetch the existing row instead of creating a second one.
The important consequence is what happens next. Having found an existing anchor, the code does not blindly charge again. It looks at the state that anchor is in and decides accordingly: charge, reconcile against the provider, or reapply a result that was already obtained. A crashed pass leaves evidence, and the evidence is what the next pass reads.
The moment you most need a record of the charge is the moment you cannot tell whether the charge happened.
Concurrency is won with conditional writes, not locks
Once there is a row, the remaining races are all the same shape: two processes look at the same anchor and both decide to act.
Reading and then writing loses that race. Both read `failed`, both decide to retry, both charge. The fix is not a mutex — it is to make the decision and the write a single conditional statement, and let the database say who won.
Retrying a failed payment is a conditional update that only matches a row currently in the failed state and flips it to pending. Two dunning retries run it; exactly one updates a row; the one that matched is the one allowed to charge. Settling an unpaid payment works the same way, and so does refunding a paid one. The count of rows the statement matched *is* the answer to "am I the winner?"
The subtlest case is recovering an anchor that has been stuck pending. You want a recovery pass to rescue a charge that genuinely died, and you absolutely do not want it stealing one that is still in flight. We match on a timestamp: only an anchor that is pending, still carrying its placeholder reference, and untouched for longer than a staleness cutoff. A live pass's anchor was updated moments ago, so it fails the predicate and is never stolen mid-charge. And because the recovery write itself bumps the timestamp, a second recovery pass running at the same instant matches zero rows.
Nothing here is a distributed lock. It is all unique constraints and conditional updates, which is the same thing your database is already very good at.
Webhooks arrive more than once, by design
Providers retry webhooks. That is not a bug in their system; it is how they guarantee delivery. Which means your handler will see the same event twice, and it must not act twice.
We record every incoming webhook against the provider's transaction id with a unique constraint on it. The insert is the deduplication: a repeat conflicts and is answered as a duplicate rather than reprocessed. The replay lock is the same mechanism as the renewal anchor, applied to a different problem.
Two more things are worth doing at that boundary. Verify the signature before you trust any field in the body, and store the verdict — we keep the HMAC validity and an overall pass/fail alongside the event, so an unsigned or tampered callback is a row you can look at later rather than something that vanished.
And store a summary, not the payload. We keep a compact set of safe fields for audit and deliberately no card data. A webhook log is a table people forget to think about, and the cheapest way to never leak card data from it is to never put card data in it.
What we would tell another team
- Write the record before the side effect, keyed deterministically. If you can only take one idea, take this one.
- Make "did I win this race?" a value the database returns, not a decision your code makes after a read.
- Treat every webhook as a duplicate until a unique constraint says otherwise.
- Make recovery jobs prove staleness with a timestamp, or they will eventually steal a live operation.
- Assume every process can die between any two lines, and ask what the next process will see. The anchor exists so the answer is never "nothing".
We are candid about scope here: this design is what our billing runs on, and the failure modes above are the ones it was built to survive. Payment correctness is not something you retrofit after the first double-charge — the whole point is that the expensive incident is the one you never have.
We do this kind of work for other teams as payments and checkout engineering, which sits alongside our SaaS development work.
All articles