Was Hired to Verify a "Ready" Website. It Wasn't.
This was supposed to be a two-week pre-launch QA audit.
"The website is basically ready."
That's what the client told me during our first call.
They had spent months rebuilding their ecommerce platform with an external development team. The demos had gone well. The relaunch was approaching.
Everything looked ready.
Still, the owner and the project manager had a gut feeling that something wasn't quite right.
That's why they brought me in.
Not to redesign anything.
Not to manage the project.
Just to spend two weeks doing a final QA audit before launch.
I expected to find a handful of issues.
Instead, by the end of my first day, I had logged 40+ bugs.
More than 20 of them were severe enough that I wouldn't have recommended launching.
The client looked genuinely shocked.
"How did none of us catch this?"
The answer became obvious almost immediately.
The demos weren't fake.
They were simply testing a perfect scenario.
Almost every serious issue appeared the moment I stopped behaving like someone giving a presentation and started behaving like an actual customer.
Customers don't follow scripts.
They change quantities.
They edit information.
They go back.
They switch shipping methods halfway through checkout.
They combine filters.
They upgrade subscriptions.
They browse on mobile.
That's where the product started falling apart.
A small sample of what I found
Shipping costs didn't update. Switch from standard delivery to express during checkout, and the total still charged for the previous shipping method.
Changing quantity broke pricing. Increase the quantity during checkout, and the total wasn't recalculated correctly. Customers could purchase multiple items for the price of one.
Subscription upgrades failed. Upgrading during onboarding resulted in errors or incorrect billing.
Product filters broke under real usage. Individual filters worked, but certain combinations returned no products even though matching items existed.
Mobile checkout became unusable. Important buttons disappeared on smaller devices, preventing customers from completing purchases.
Several checkout flows simply couldn't complete. Depending on the product type, users would hit validation errors or payment failures before reaching confirmation.
None of these appeared during the demo.
Every one of them would have affected real customers.
What surprised me wasn't how many bugs I found.
It was where I found them.
These weren't edge cases that required ten impossible clicks to reproduce.
They were normal customer journeys.
Buying a product.
Changing shipping.
Updating quantities.
Upgrading a subscription.
Checking out on a phone.
The kinds of things customers do every single day.
The original engagement was supposed to last two weeks.
Months later, I'm still working with the client.
The work changed from discovering issues to validating fixes, catching regressions, and helping investigate bugs whenever something unexpected appeared.
That taught me another lesson.
Finding bugs is only the first step.
Every fix creates the possibility of another problem somewhere else.
Good QA isn't about running through a checklist.
It's about continuously asking:
"What happens if the customer doesn't behave exactly like we expected?"
One thing I hear a lot is:
"We already tested it."
I'm sure they did.
The developers tested it.
The product team tested it.
The demos passed.
None of those things were false.
But they weren't enough.
Because there's a huge difference between proving that your product can work and proving that it will work once hundreds of real people start using it in ways nobody predicted.
That's where QA earns its value.
If this website had launched as it was, the biggest cost wouldn't have been fixing bugs afterward.
It would have been failed orders.
Incorrect payments.
Support tickets.
Frustrated customers.
And trust that had taken years to build.
The website wasn't "broken."
It simply hadn't been tested beyond the happy path.
And in my experience, that's exactly where the most expensive bugs tend to hide.
I'm curious—have you ever had a product that looked perfect during the demo, only for real users to uncover problems almost immediately after launch?
The part that stuck with me is that the demos weren't fake, they just tested a perfect scenario. You build muscle memory around using your own product the "right" way and stop seeing the paths a real person takes — changing quantity mid-checkout, switching shipping, hitting back. None of that is an edge case, it's just how people behave when they're not performing a demo. Did you find any way to systematically force yourself off the happy path, or does it really just take fresh eyes like yours coming in?
Great question. Fresh eyes definitely help, but over time I've found it's also a habit you can build.
I try to constantly ask myself: What if the user changes their mind? Goes back? Edits something?
Those questions naturally take you away from the happy path and closer to how real customers actually use the product.
I'm actually trying to build something right now. And I'm not sure werether it's good or I just want it to be that's the depressing part. And I'm betting a lot on this.
I can relate to that. One thing this audit reinforced for me is that getting fresh eyes on something isn't admitting uncertainty, it's a way of validating confidence before the stakes get higher. Sometimes the most valuable outcome is confirming you're on the right track.
A 2-week audit turning into months of retainer work is the ultimate QA origin story. Glad they actually listened to you instead of rushing the launch anyway.
Thanks! 😄 Credit to the client, they took the findings seriously instead of treating them as obstacles to launch. I think that's what turned a two-week audit into an ongoing collaboration.
The part that stuck with me is that the worst ones in your list are silent. Shipping cost not recalculating doesn't throw an error — it just quietly charges the wrong amount. No crash, no red flag, so it sails straight through the demo and the dev tests and only surfaces later as chargebacks and support tickets. That's the hardest QA case to make to a client pre-launch: you're not flagging things that look broken, you're flagging things that look completely fine and aren't. "The button doesn't work" sells itself. "This checkout silently overcharges 1 in 20 people" is the one nobody believes until it's live — and it's also the one that actually burns the trust you mentioned.
That's a great observation. The silent issues were definitely the most concerning.
A visible error gets attention immediately. A workflow that appears successful but produces the wrong outcome can go unnoticed for a long time and by then, the cost is usually measured in customer trust rather than just bug fixes.
exactly — the failure mode nobody budgets for is the one that returns a 200 and a wrong answer. loud errors get triaged same day; the silent, successful-looking ones get trusted until a customer finds them. that's why i treat "looks done" as a risk signal, not a finish line.
State transitions are where it breaks every time. Your shipping/quantity/subscription examples all follow the same pattern: the happy path is linear (select, confirm, done) but users are non-linear (change, reconsider, modify). When a system assumes A then B then C, and users actually do A then B then A-again, that's when recalculation fails. This gap between "tested in sequence" and "tested with reversals" probably explains half of production bugs. Great catch on it being about user behavior, not code quality.
Thanks! I think that's what surprised me most during the audit. The individual features usually worked fine on their own. The problems appeared once users started moving back and forth between them. That's when you're no longer testing features in isolation, you're testing how the whole system behaves.
The "happy path" concept is such a useful frame for something that kills so many launches. Teams spend months in a product and build such deep familiarity with it that they literally cannot see it the way a new user does.
The "fresh eyes" point resonates with me beyond just QA. I notice the same dynamic at goldenweeks, a 2-week deep work retreat in Zanzibar. Founders who step out of their day-to-day environment often catch things in their own product or strategy that they had been too close to see for months. Sometimes a change of context is the audit you did not know you needed.
Great write-up, Redion. The state transition testing advice alone is worth bookmarking.
Thanks! I really like that analogy. I think "change of context" is another way of creating fresh eyes.
The longer we work on something, the more assumptions become invisible to us. Whether it's a product, a strategy, or even a business, stepping back often reveals things that were there all along.
This is basically the tech version of what I do on the financial side. "It's basically ready" and "the numbers basically check out" have the same problem — they're only true until someone actually stress-tests it instead of watching the demo. Happy path testing and happy path financials both hide the expensive stuff until it's too late.
That's a great analogy. Different domain, same mindset.
Whether it's software or financials, the happy path gives confidence. The stress test reveals where the real risks are.
the through-line in your whole list is state transitions — change shipping after picking one, change quantity after adding to cart, upgrade after onboarding. that's the tell: code and demos both cover the forward "create" path (set it once), and almost nobody tests "now change it." recalculation-on-change is where it breaks, every time. the happy path proves a product can work; the mutation paths prove it does. cheapest fix is making "change every value after it's set" a required test path, not an afterthought. great write-up.
I like the way you framed it around state transitions. Looking back, that's exactly what connected most of the issues I found.
"Change every value after it's set" is such a simple rule, but it naturally pushes testing beyond the happy path and closer to real user behavior.
The "demos test the happy path, customers test reality" insight is the whole post, and the shipping-cost-not-updating and quantity-breaking-pricing examples are the kind that make a founder's stomach drop. Great catch.
The deeper pattern worth naming: every bug you found lives at a moment of change. Switch shipping mid-checkout, edit quantity, upgrade a subscription, combine filters. The happy path is linear (pick, buy, done), so that's what gets built and tested. Real users are non-linear, they change their mind mid-flow, and state-change is exactly where code that assumed a straight line breaks. Your value isn't "I test more thoroughly." It's "I test the transitions, not the destinations."
Which is the marketing angle hiding in your post: you frame this as QA, but what you sell is "I find the revenue-leaking bugs your team can't see because they built it." A dev testing their own code follows the path they designed, so they can't see the non-happy-path because their mental model IS the happy path. Not a skill gap, a structural blindness. Lead with the money ("multiple items for the price of one, shipped to production") not the process.
To your question, the most common version I've seen: the demo runs on clean seed data, then real users bring messy data (weird characters, huge inputs, empty fields) and it buckles. Same root cause, the demo tested the world as imagined, not as it is.
Are you productizing this into a repeatable audit offer, or keeping it bespoke?
That's a really interesting way to frame it, "I test the transitions, not the destinations." I hadn't thought of it that way, but it describes the pattern I kept seeing during this audit.
And yes, I'm gradually productizing it. Every audit is different, but the overall approach is becoming much more repeatable. The goal isn't just to find bugs, it is to validate the workflows that matter most before customers do.
A classic example, and very true. When you know exactly what a platform is designed to do, it’s easy to fall into following a set script.
We learned this the hard way. Now, we always bring in someone who has no prior context of the project journey and give them complete freedom to approach QA from both a subject expert’s and a user’s perspective.
That's been my experience as well.
One thing I've learned is that familiarity can become a blind spot. After working on a product for months, it's only natural to interact with it the way it was designed.
Fresh eyes don't have that context, so they're more likely to explore unexpected paths and question assumptions. That's often where the most valuable findings come from.
What stood out to me is that none of these were “bugs” in the traditional sense—they were behavior mismatches between how teams test and how customers actually behave. Most QA catches correctness. Very few systems catch behavioral divergence, where the product works technically but fails under real decision paths. That gap is usually what decides whether launch-day traffic converts or collapses silently.
That's a great way to put it. "Behavioral divergence" captures what I was seeing much better than simply calling them bugs.
Most of the issues weren't caused by a single broken feature. They appeared when users behaved differently than the happy path the product had been designed and demonstrated around.
That's one of the reasons I enjoy exploratory testing so much, you're not just verifying features, you're trying to think like someone who has never seen the product before.
That's exactly where my thinking went as well.
I think there's one strategic implication of that distinction which becomes much more important as products mature, but I don't think I can explain the reasoning properly in a thread without oversimplifying it.
If you're interested, what's the best email to reach you on?
Thanks! I appreciate the thoughtful discussion.
Feel free to reach out through qaura.io , my contact details are there if you'd like to continue the conversation.
Thanks! I've just sent it over.
Looking forward to hearing your thoughts whenever you have a chance.