Fire a conversion rate optimization consultant who loses sales

That's not a vendor relationship anymore. That's a hemorrhage. And your job — right now, this week — is to plug it before the quarter closes.

Lock the baseline before you lock the door

Here's the rule that 90% of e-commerce operators ignore until the money is already gone: if the KPIs aren't nailed down in writing before the first test ships, you cannot fire for cause. You can fire for convenience. You can fire because your gut says so. But you cannot point at a contract clause and say "performance failed" because the contract never defined performance.

A real CRO engagement defines, in writing, before any work begins:

• Conversion rate (CR) on the funnel pages being tested
• Average order value (AOV)
• Cart abandonment rate
• Revenue per visitor (RPV)
• Statistical significance threshold — 95% is the industry standard, anything below is a coin flip dressed up as science
• Test runtime minimums before a result can be called
• Reporting cadence and the exact data fields each report must include

Everything else is noise. Pageviews, scroll depth, time on site, "engagement score" — these are vanity metrics that look great in a monthly slide and convert exactly zero extra customers. If your consultant is leading with them in the kickoff deck, you already know how this engagement ends.

Here's where it gets expensive. Vanity metrics are safe. Nobody gets fired for reporting a 12% lift in scroll depth. But a consultant who leads with scroll depth while your cart abandonment creeps from 68% to 74% is hiding behind a metric that has no direct connection to revenue. The slide looks green. Your Stripe dashboard is red. And every month that gap widens, the compound effect on lost revenue becomes harder to recover — because you're not just losing the sales you would have made, you're losing the customer lifetime value that would have followed.

Pull the contract up tonight. If the success criteria are vague, your first move isn't termination — it's a written amendment that pins the numbers, the runtime, and the reporting format. That amendment is what protects you 90 days from now when the consultant's lawyer calls yours.

No KPI baseline in writing means you're paying a consultant to guess. And guesswork, at $10K/month, is the most expensive line item on your P&L.

Red flags in the reporting — the things they hide in the appendix

A consultant who is winning shows their work. A consultant who is losing buries the work. Here's what to scan for in the last 60 days of deliverables.

Statistical significance is missing or under 95%

If a test "won" at 87% confidence, it didn't win. It might have won. It also might have lost on the next 200 sessions. CXL, VWO, and every reputable testing platform will tell you the same thing: 95% is the floor for a callable result. Anything below is a hypothesis, not a verdict.

This isn't pedantry. A test that runs to 87% and gets called a winner has a roughly 1-in-7 chance of being noise. Run ten of those and you're building your optimization roadmap on one or two false positives — variants that look good today, ship to production, and quietly erode conversion when the traffic mix shifts. You don't feel it in week one. You feel it in month three when the cumulative lift you thought you'd banked turns out to be a cumulative drag.

Wins are cherry-picked, lifts are on secondary metrics

Watch for the consultant who proudly reports a 4% lift on "add to cart clicks" while revenue per visitor moves the wrong direction. CR up, RPV down, AOV collapsed — and they're celebrating the click. That's not optimization. That's moving deck chairs while the ship sinks.

The secondary-metric sleight of hand is the single most common way failing consultants stretch an engagement. They need a win on the slide, so they cherry-pick the one metric that moved in the right direction and build the narrative around it. Your job is to ask one question: "What happened to revenue per visitor?" If the answer is "well, that's a lagging indicator" — you have your answer.

"We need more time" appears more than twice

Traffic accumulates. Tests mature. But if the account manager keeps pushing the readout window — month three becomes month four, month four becomes "let's look at Q3" — they don't have a result. They have a story. And stories don't ship revenue.

Time is the consultant's best friend when the tests aren't working. Every extension is another retainer cycle. Every "let's revisit after the holiday season" is another invoice. A legitimate traffic shortage exists — a site doing 15,000 sessions a month can't power the same test cadence as one doing 500,000. But a consultant who proposed the engagement already knew your traffic volume. If they're citing it as an obstacle in month four, they either sandbagged the timeline to lock in a longer contract or they didn't do the math before they pitched.

The narrative summary replaces the data table

A real A/B test report shows: hypothesis, control variant, treatment variant, sample size per arm, confidence interval, lift, p-value, runtime, and segment cuts. If you're getting a one-pager with three bullets and a Loom, you're not being optimized. You're being managed.

Loom videos and narrative summaries have a place — they're useful for explaining the "why" behind a result to a non-technical stakeholder. But they are not a substitute for the raw output. When the narrative deck is the only deliverable, the data is being filtered through someone who has a financial incentive to frame it favorably. You need both. The narrative for the exec team, the data table for anyone who needs to verify the claim.

Experimental failure is not grounds for firing. Pattern failure is.

This is the line that saves you from a bad-faith termination lawsuit and keeps your replacement hire moving fast. CRO is inherently experimental. A single test that loses money is not negligence. It's the cost of doing business in a discipline where the majority of tests either don't reach significance or don't move the primary metric.

What is grounds for firing is a pattern:

• Three or more consecutive test cycles with no statistically significant winner on a primary KPI
• A sustained drop in CR over multiple weeks with no corrective plan
• Repeated methodology errors — running underpowered tests, peeking at the data, changing variants mid-test, ignoring QA
• Blaming external factors (seasonality, iOS updates, ad spend cuts) for every negative result without isolating the variable

The 3-to-6-month evaluation window isn't arbitrary. It takes traffic volume and a stack of test cycles to know whether the engagement is producing lift. CXL and VWO both publish guidance in this range, and Upwork's CRO contract templates default to it. If you've cleared six months and the dashboard is red, the engagement isn't working — it's costing you.

There's a subtlety here that separates a firing you can defend from one you can't. The difference between a bad consultant and an unlucky one is transparency. An unlucky consultant ships a test that misses significance, shows you the full data, explains why the sample was underpowered or the traffic mix shifted, and proposes a revised hypothesis with a tighter design. A bad consultant ships the same test, calls it "directionally positive," and moves on to the next variant without documenting why the first one failed. The test results may look identical. The professional behavior is night and day.

A cure period isn't a courtesy. It's your legal shield. Skip it and you pay out the contract, plus their legal fees, plus a great story for the next consultant who Googles your company.

The cure period playbook — 15 to 30 days, in writing

Most CRO contracts include a "termination for cause" clause. Read it tonight. The standard structure is:

1. Written notice of performance failure, with specific KPI references

2. A cure period — typically 15 to 30 days — during which the consultant must remediate

3. A final termination letter if remediation fails, citing the original notice and the expired cure period

What goes in the cure notice

• The KPIs from the contract or the amendment
• The actual numbers from the last 60–90 days, sourced from the testing platform — not the consultant's PDF
• The specific deliverables that are missing (significance reporting, segment cuts, documented hypotheses, SOW items)
• A dated remediation plan — what they will deliver, by when, with what success criteria

Send it on a day you can prove they received it. Email plus a tracked PDF. CC your legal counsel. Save the read receipt. This document is the artifact your attorney pulls when the invoice dispute lands.

What you do during the cure period

Do not pause the engagement. Do not stop the tests in flight. Do not coach them through it. The cure period is their chance to perform. Your job is to document, not to rescue.

• Pull a full export of all test data, variants, hypotheses, and learnings to date
• Identify which tests are statistically ready to call and which need more runtime
• Flag any winning variants currently live — those stay live through the transition
• Begin scoping the in-house CRO owner, fractional replacement, or senior PM who will run tests directly

If they turn it around in those 30 days, you keep the relationship. If they don't, you terminate with cause, and your exposure on the remaining contract value drops dramatically.

One operational note: during the cure period, start building your own institutional knowledge of the testing program. If you don't have login access to the testing platform, request it now — not as a signal of distrust, but as standard data governance. Any consultant who resists giving you admin-level access to your own testing infrastructure is a consultant you should have fired yesterday.

The post-mortem audit before the final invoice

Most operators skip this step because they're exhausted and want the vendor gone yesterday. Don't. The audit is what stops you from paying for work that was never delivered and what stops the next consultant from inheriting a mess they can't untangle.

Pull and preserve

• All A/B test records — variants, hypotheses, results, confidence intervals, segment cuts
• All implemented changes — which code shipped, which lives in the repo, which is a config flag
• All learnings documents, even partial drafts
• All access logs — QA accounts, analytics, CDP, ESP, tag manager, heatmap tool
• All reporting artifacts — decks, Looms, dashboards

Decide what stays live

Winning tests with statistical significance stay deployed. Losing tests get rolled back. Tests with no result get parked — they're a knowledge asset, not a launch decision. Do not let the consultant roll anything back themselves during transition; you control the timeline.

This matters more than it sounds. A "no result" test — one that ran to adequate sample size without reaching significance — is still valuable data. It tells you what doesn't move the needle on your audience, which narrows the hypothesis space for the next round. Rolling it back without documenting it is burning research dollars you already spent. Park it, label it, and hand it to the replacement with context.

Audit the final invoice against the SOW

Match deliverables against the statement of work. If the contract said 8 tests per quarter and they shipped 4, you don't owe the remainder. If they billed for "strategic roadmap" and never delivered one, that's a credit. If they billed for "monthly reporting" and delivered two reports in six months, that's a credit. Most consultants will not argue line items when the data is sitting in their own platform export.

Lock down access the day of termination

Analytics, ESP, testing platform, repo, CMS, tag manager. Rotate every credential. Audit every user list. The relationship ends on the day the termination letter is signed — not 30 days later when "transition" wraps up. The day after the letter, they have no business logging in.

This isn't paranoia. It's access hygiene. A disgruntled vendor with live credentials to your analytics stack, your email platform, and your testing tool is a liability no matter how professional the relationship was. Rotate first, audit second, reconcile third.

What to do tomorrow morning

The CRO consultant is a cost line, not a strategy. The strategy is your funnel, your offer, your brand, your list. The consultant is supposed to compound the strategy. When they subtract from it, the math is simple: you fire them and you fire them cleanly.

• Tonight: Pull the contract. Confirm KPIs are defined in writing. If not, draft the amendment and send it before any new tests ship.
• This week: Pull the last 90 days of test reports. Score each one against the red-flag checklist above. Count the pattern failures.
• Within 14 days: If pattern failure is confirmed, issue the cure notice in writing. Start the cure-period clock.
• Day 30: If remediation failed, send the termination letter. Match the final invoice to the SOW. Rotate every credential.
• Day 31: Stand up the replacement — in-house CRO lead, fractional consultant, or your senior PM running tests directly. Do not let the seat sit empty; your CR is bleeding right now.

Sustained losses from a bad vendor compound the way CAC compounds: every month you wait, the next month costs more. The cure period is your runway, not your excuse to procrastinate.

Protect the budget with the same discipline you'd apply to any other capital allocation — firm today, compounding tomorrow. Your CRO budget is exactly that. Spend it on lift, not on a vendor whose tests can't clear 95% and whose reporting can't survive a screenshot.