The Lost Update Problem: Where Did My $50 Go?

Part 1 of a 3-part series on concurrency in payment systems.

Here's a system design interview question that sounds almost too easy:

MegaStore has a balance of $100. Alice and Bob both pay MegaStore $50 at the exact same millisecond. How do you update the database so MegaStore ends up with $200?

Before reading on, take 30 seconds and sketch your answer. Most of us (me included, the first time) reach for something like this:

balance = db.query("SELECT balance FROM accounts WHERE id = 'MegaStore'")
db.execute("UPDATE accounts SET balance = ? WHERE id = 'MegaStore'", balance + 50)

It reads the balance, adds 50 and writes it back. It passes every unit test you'll ever write for it, and in production it will lose money.

Watching the money disappear

Let's slow time down and watch two requests run that code concurrently:

Both threads read the same value, both compute the same answer, and both write it. Bob's write lands on top of Alice's and erases it.

The final balance is $150. Fifty dollars has vanished, and nothing failed. No exception, no error log, no alert. Both requests return 200 OK. Alice and Bob both got receipts, and MegaStore is $50 short.

This is the Lost Update problem, and the "silent" part is what makes it dangerous. A crash gets noticed and fixed. A lost update gets found weeks later in a reconciliation report, if at all.

Two problems hiding in one bug

When you explain this in an interview, it helps to name both halves, because they're distinct:

1. The race window. Both threads read the same stale value before either has committed. The gap between "read" and "write" is where another request sneaks in.
2. The lost update. One thread's write silently overwrites the other's. The database did exactly what it was told, and what it was told was wrong.

The window is the cause and the lost update is the damage.

"Just use balance = balance + 50"

The sharp reader is already shouting this. And for this exact example, they're right:

UPDATE accounts SET balance = balance + 50 WHERE id = 'MegaStore';

This is safe. The database reads and writes the value inside a single statement, taking a row lock for the duration, so two concurrent increments are serialized and you get $200. There's no gap for anyone to sneak into.

So why isn't that the end of the article?

Because real payment systems almost never get to write without reading first. Before you touch a balance, you usually need to check something:

• Does the payer have sufficient funds?
• Would this exceed a daily transfer cap?
• Does a fraud rule flag this amount for this account?
• Is the account frozen, closed or under review?

Every one of those checks is a read whose result decides the write. The moment your logic looks like "read the row, think in application code, then write," you've reopened the race window, no matter how atomic your final UPDATE looks.

The interview trap is confusing an atomic update with a read-modify-write cycle. balance = balance + 50 is atomic. "Read in code, decide in code, write in SQL" is not.

A partial escape hatch: push the check into the WHERE

For simple invariants you can sometimes fold the check into the update itself:

UPDATE accounts
SET balance = balance - 50
WHERE id = 'alice' AND balance >= 50;
-- 1 row affected → success
-- 0 rows affected → insufficient funds

This is atomic and race-free, and it's worth knowing. But it only works when the rule fits in a single row's WHERE clause. "Sum of today's transfers across three tables is under the cap" or "the external fraud service says OK" doesn't fit there. For those, you need a real concurrency-control strategy, which is what Part 2 covers.

It's not just money: same bug, many names

Once you see the shape (read, assume nothing changed, write), you start seeing it everywhere:

They're all symptoms of one root cause: the system assumed nothing would change between the read and the write. On a single thread on a single machine, that assumption mostly holds. In a distributed system with many app servers sharing one database, it never does.

Why "it's rare" is not a defense

It's tempting to argue that two payments to the same merchant in the same millisecond is an edge case. For a small merchant, maybe.

Now picture a flash sale. Thousands of requests per second, all updating the same merchant's balance row. The read-to-write window might be only a few milliseconds, but at that volume windows overlap constantly. Every overlap is a lost transaction. At scale, this goes from rare to guaranteed.

And because nothing errors, you won't see it in your dashboards. You'll see it when finance asks why the ledger doesn't match the bank.

The mindset shift

The most useful takeaway from this problem isn't any particular fix. It's a change in the question you ask while writing code.

Every time you write a SELECT whose result feeds an UPDATE, ask the second question.

Recap

1. An atomic in-database update (SET balance = balance + 50) is safe only when you don't need to read the value first.
2. Read-modify-write cycles in application code create a race window between the read and the write.
3. The result is a Lost Update: silent, undetected, and inevitable at scale.
4. The same root cause shows up as double bookings, negative inventory, double charges and TOCTOU bugs.

So how do you close the window when you do need to read first? There are two classic answers, optimistic locking and pessimistic locking, and choosing between them is where the interesting trade-offs live.

Next: Part 2: Optimistic vs. Pessimistic Locking: Retry or Wait?