The Expense That Showed Up Twice
A duplicate bug I couldn't reproduce, an AI that wanted to paper over it, and why the extra machinery you didn't ask for is where the concurrency bugs live.
Sections
The bug report came in on the first day of the beta. Two testers, same complaint: an expense was showing up twice in their list. One of them sent a screenshot. Same merchant, same comment, same amount, same date, stacked on top of each other like the app was trying to make a point.
I couldn’t reproduce it. I tried on my phone, on the simulator, on bad hotel wifi, with the app killed mid-upload. One expense, every time.
Some context. The app is an internal tool for a field sales team. I’d just added receipt capture: scan a receipt, review the merchant and amount, tap Save, and get on with your day while the upload happens in the background. Simple feature, except for the last three requirements, which were “keep working offline,” “retry when the network drops,” and “never, ever create a duplicate expense.” Those three turn a camera and a form into a small distributed system. There’s durable state on the phone, a background transfer that iOS manages on its own schedule, a Node server, a MySQL table, a receipt file on disk, a Core Data cache, and a SwiftUI list watching that cache. Any seam between those can fail, and now one of them apparently had.
The AI had theories
I was building this with Claude Code in the loop, and it had theories immediately. Maybe the testers double-tapped Save. Maybe they didn’t notice the “uploading in background” message and entered the receipt again. Maybe a request timed out and the queue sent it twice. Maybe the list rendered the same object twice.
All plausible. Then came the fixes, and this is the part worth writing down. The first suggestion was a debounce on the Save button. The second was to deduplicate expenses by merchant, amount, and date before inserting them.
I said no to both, and no to the second one a little louder.
A debounce is harmless as polish, but it isn’t a fix. It treats one way a duplicate could happen as if it were the only way. The content hash is worse than useless. Two people from the same team eating at the same taco place on the same day for the same amount is not a bug. It’s Tuesday. Silently dropping one of those expenses would turn a visible duplicate into an invisible missing reimbursement, which is a much worse bug that nobody would ever report.
Neither suggestion was stupid. They were the kind of thing a smart engineer proposes in the first five minutes of a bug triage. The problem was that they were proposed as answers, before anyone had asked the question.
The question
Here it is:
Are there two rows on the server, or is one row being shown twice on the phone?
That sounds obvious written down. It wasn’t obvious in the moment, because “duplicate expense” reads like one problem. It’s at least five. Between tapping Save and seeing a row in the list, “the expense” exists as five different things, and any one of them could be the one that doubled.
So instead of a fix, I shipped a debugging aid. The expense detail screen got one extra line at the bottom, in small grey type: the transaction ID. Now when a tester said “it’s doubled,” I could say “open both, read me the IDs.” Different IDs would mean the server made two expenses. Same ID would mean the server made one and the phone was showing it twice. Ten minutes of work, and it split the problem in half.
The server was already fine
While I waited on testers, I went through the server path, half expecting to find the bug there. I didn’t.
Every upload job gets a UUID the moment it’s created on the phone, before anything is sent. It’s saved to disk with the job, and every retry sends the same one. The server hashes it into the 25-character transaction ID the existing schema wants, and that column has a unique index. If two requests with the same UUID race, MySQL picks a winner. The loser catches the duplicate-key error and returns the winner’s row.
Controllers/nativeExpenseRouter.js
const transactionid = transactionIDFor(uploadID); // sha256(uuid) → "cp-" + 22 hex
const existingExpense = await Expensify.findCanonicalOwned(transactionid, username, req.dbPool);
if (existingExpense) {
// Same UUID as an earlier attempt. Hand back the same expense.
return res.status(200).json(canonicalExpenseResponse(existingExpense));
}
try {
createdExpense = await helpers.withTransaction(req.dbPool, async (connection) => {
await Expensify.createUploadedExpense({ transactionid, username, ...metadata }, connection);
return Expensify.findCanonicalOwned(transactionid, username, connection);
});
} catch (error) {
// Two requests raced past the check above. The unique index is the referee.
if (error.code === "ER_DUP_ENTRY") {
const duplicate = await Expensify.findCanonicalOwned(transactionid, username, req.dbPool);
if (duplicate) return res.status(200).json(canonicalExpenseResponse(duplicate));
}
throw error;
}
res.status(201).json(canonicalExpenseResponse(createdExpense));
That’s the whole idempotency story, and it’s the thing the debounce suggestion was trying to reinvent in the wrong layer. A timeout on the phone doesn’t tell you whether the server failed before writing the row or wrote it and lost the response. Retrying is the only sane move, so retrying has to be safe, so the identity has to be fixed before the first attempt. The database gets the final say because it’s the only thing both server processes can see.
Given all that, the server producing two rows with different IDs would have needed two separate taps on Save, which the testers were fairly sure they hadn’t done. So I looked at the phone.
One lunch, two objects
The app uses Core Data as a cache of what the server said. Not a source of truth, a cache. The list is a @FetchRequest, which means it watches Core Data, not the server.
The function that refreshes the expense list is old. It’s been in the app for years: download every expense, parse the JSON on a private background context, merge into the main context. For all of those years it had exactly one caller, the list screen itself, and nothing ever ran it twice at once.
Then the new upload queue, which Claude Code wrote, needed the list to show a freshly uploaded expense. Its answer was to call that same full refresh after every completed upload. Re-download the whole list to make one row appear. It works, and I let it through, because it was a small line at the end of a large feature and I was looking at the upload logic, not the cleanup.
Now there were two callers with no coordination. Pull to refresh while an upload finished in the background, and two refreshes ran side by side. Each one fetched the cache, didn’t see the new expense yet, and inserted its own managed object for it. Both objects had the same transaction ID. Both got merged up into the main context. The list did exactly what it was told and showed both.
The server had one lunch. The phone had two.
Nobody wrote a race. The AI added a moving piece that sounded reasonable, a full refresh after each upload, and pointed it at code that had never been asked to run concurrently. This is the shape of most concurrency bugs I’ve seen: not a mistake in the hard part, but an extra mechanism added for a reason that sounded fine, with nobody owning the question “what happens when two of these overlap?” An AI will add that mechanism in seconds. It will not ask the question.
The fix was in three layers, and I think the order matters. Plus one deletion: the upload endpoint now returns the created expense, and the queue upserts that single object instead of re-downloading the list. The extra caller is gone. First, the model got a uniqueness constraint on the transaction ID, so Core Data itself refuses to hold two objects with the same one. Second, refreshes were serialized through a single in-flight task on the main context, so the race can’t happen in the first place. Third, because the constraint can’t be added to a store that already has duplicates in it, the app rebuilds the Core Data store once on first launch after the update.
That last step made the AI nervous, and it should make you nervous too, in general. “Delete Core Data” is a terrible idea if Core Data holds anything the user created and hasn’t synced. Here it holds nothing of the kind. Pending uploads live in a JSON manifest. Credentials live in the keychain. Core Data is a copy of the server, and the server is still there. Deleting a cache is not data loss. It’s a cache miss.
What I actually learned
The bug was a race, and the fix was a uniqueness constraint. Neither is interesting on its own. What I keep coming back to is how close I came to shipping a debounce and a content hash, calling it fixed, and waiting for the next report.
Here’s the thing about building with an AI in the loop. It doesn’t write bad code. It writes plausible code, fast, for the first theory anyone states out loud, and it has no instinct for leaving things out. Over the course of this one feature it proposed a second expense table when the schema already had one, a thumbnail-and-CDN pipeline for receipts nobody would look at twice, hashing the username into an idempotency key that was already unique, storing the server’s response in the upload queue as a second source of truth to cover a crash window the retry already handled, and the full list re-download after every upload that caused this bug. Every one of those was defensible in isolation. Every one of them was another moving piece, another place where two things could overlap, another surface for exactly the kind of bug this post is about.
The debounce and the content hash were the same instinct pointed at a symptom. More mechanism, offered as an answer, before anyone asked what the mechanism was for.
So the job isn’t to review the AI’s code. It’s to direct it, which mostly means saying no, and saying it early, with a reason. The discipline it doesn’t have, and that I had to supply, is a short list of questions that come before any fix:
- Did this failure actually happen, or does it merely sound like it could?
- Which invariant would it break?
- Which layer already owns that invariant?
- What is the smallest thing that closes the gap?
- How will we know it worked?
The transaction ID at the bottom of the detail screen answered the first question. The identity chain answered the second and third. The constraint was the fourth. A tester reading me two matching IDs was the fifth.
None of those questions produce code. That’s the point. The AI is very good at the step after them, and dangerous at the step before, because it will happily skip it. Somebody has to hold the line on “what is the smallest system whose behavior we can actually explain?” That somebody is still you.
If you want the whole system, every crash window and the code that recovers from each one, it’s written up as reference notes. That page is long and nobody should read it start to finish. It exists so I don’t have to keep this in my head.