How to measure the ROI of AI in marketing when the baseline is a mess

Measure AI marketing ROI on one locked denominator: cost per booked qualified appointment, not cost per lead. In the worked example below, a single identical quarter reads as a 29.6% deterioration on cost per lead and a 16.7% improvement on cost per appointment. Same media plan, same records, opposite verdicts.

• The formula: ROI = (revenue from the cohort − total cost of the cohort) ÷ total cost of the cohort. A cohort of leads, not a calendar quarter.
• The denominator: cost per booked qualified appointment. Cost per lead moves the wrong way at exactly the moment AI starts working.
• The cost basis: media + platform fees + implementation + internal hours. Leaving the AI run cost out flattered the worked example by 14.3%.
• The cohort rule: credit revenue to the period the lead was created, not the period the deal closed. That one choice moved the example’s ROI from 290% to 239%.
• The evidence floor: at 1,200 leads a month, one quarter of data can prove a 3-percentage-point move in your lead-to-appointment rate and nothing smaller.

Cost per lead or cost per booked appointment — which am I actually measuring?

These two numbers agree only while your lead-to-appointment rate is constant — and changing that rate is precisely what AI qualification, instant follow-up and automated re-contact are for. The two denominators are guaranteed to disagree the moment the deployment works.

Cost per lead divides spend by volume at the top; cost per booked qualified appointment divides the same spend by a unit a salesperson can sell to. When AI lets you switch off a junk source, lead volume falls, cost per lead rises, and a dashboard built on cost per lead reports a failure while the sales diary fills.

If your marketing ROI report can go up and down without a single extra dollar of revenue changing hands, you are measuring the denominator, not the deployment.

A worked example: the same quarter, two denominators, two verdicts

Illustrative inputs — substitute your own. A B2B team runs 90 days before go-live, then 90 days with AI qualification and instant follow-up live on the same media plan, with one-off events excluded from both windows.

Run it yourself: total cost ÷ leads, then total cost ÷ booked appointments, on both windows. Cost per lead got 29.6% worse. Cost per booked appointment got 16.7% better. Cost per closed deal fell from $2,769 to $2,308 — the same 16.7%, because show rate and close rate did not move. That is the tell. When show rate and close rate are held constant, cost per appointment and cost per deal always move together; the divergence between cost per lead and everything downstream is the entire argument.

The four measurement choices that move your reported AI ROI most

Ranked by how far each one can move the number, worst first. The first three are computed from the example above; the fourth is what a single uncorrected one-off event does to a baseline of that size.

Three of the four are decisions, not measurements. They get made once by whoever built the dashboard and are never revisited — which is why two people in one company can produce ROI numbers 50 points apart from the same database.

“My baseline is a mess” — how to rebuild one you can defend

You almost certainly cannot get a clean before-state. Rebuild a defensible one instead, in this order:

1. Fix the window at 90 days ending the day before go-live. Not “last year”, not the last full quarter. Seasonality contaminates anything longer, and anything shorter will not clear the evidence floor below.
2. Exclude one-off events from both windows — a trade show, a launch, a competitor outage. If you cannot exclude the event, exclude the whole source from both sides.
3. Freeze source taxonomy. If lead-source values were renamed mid-window, the before and after are not comparable. Map old values to new ones and record the mapping.
4. Count sales-accepted leads, not form fills. Form fills include bots, duplicates and competitors; your rejection rate is your best estimate of how dirty the top of the funnel is.
5. Write the three locks down and date them — denominator, cost basis, cohort rule — before you see any results. This is the only step that survives a hostile CFO.

If the source tagging in your CRM is genuinely unrecoverable, the honest move is to declare the pre-period unusable and measure forward from a stamped cohort instead. The four-field first-touch stamp we use to credit pipeline to AI search referrals is the same mechanism, and it works for any channel.

How many leads do I need before a before/after means anything?

Use the standard two-proportion sample size at 80% power and 5% significance: n per arm ≈ 7.84 × [p₁(1−p₁) + p₂(1−p₂)] ÷ (p₂−p₁)², where p₁ is your current lead-to-appointment rate and p₂ is the rate you are trying to prove. From a 10% baseline:

At 1,200 leads a month, a single quarter can prove a 3-percentage-point move in your lead-to-appointment rate and nothing smaller. A pilot on 400 leads cannot detect anything under about a 10-point swing, which is why so many AI pilots end in an argument rather than a decision. If the effect you expect is small, do not run a bigger pilot — move the measurement to a step where the effect is large relative to the noise, such as speed to first contact or appointment set rate.

The Denominator Lock: three things to fix before go-live

A rule you can hand to a CFO in one line: lock the denominator, the cost basis and the cohort rule before the system goes live, in writing, and report every subsequent number against all three.

• Denominator lock. One unit, named. Cost per booked qualified appointment, with “qualified” defined by a written criterion, not a feeling.
• Cost basis lock. Every line that would disappear if you switched the system off, including the hours your own staff spend supervising it.
• Cohort lock. Revenue belongs to the period the lead was created. If your sales cycle is 60 days, your first honest read is at day 150, not day 90.

Our own headline figure is written to this rule and you can check it: the 7x average sales lift defined on our methodology page is trailing three-month closed-deal revenue at month six over the trailing three months before launch, averaged across clients who supplied both — and the same page discloses the median is closer to 4x. An average and a median that far apart is information, not an embarrassment. Ask any vendor quoting a multiple for both.

The denominator error we made ourselves

In our own editorial review on 11 September 2026, reviewers re-checked the numbers on 30 draft pages against their sources. Twenty of the thirty needed a correction before publishing, and one of those corrections was a denominator error of ours. One page had taken our Colliers-era database reactivation rate of 4.4% — which is appointments booked per dormant record contacted, set out on our database reactivation case record — and spent it as jobs won. Same number, wrong unit, wildly wrong business case.

We fixed it by stating the denominator in all four places the figure appears on the page that carried the error. A rate with no stated denominator is not a measurement, and the error stays invisible until someone builds a forecast on it. Look for it in your own AI ROI deck before your CFO does.

What this measurement work actually costs to run

The locks take about a day to write and agree. Rebuilding a defensible baseline from a contaminated CRM is the expensive part — budget 15 to 30 hours of analyst time for taxonomy mapping and one-off exclusion, plus 2 to 4 hours a month to keep the cohort report honest. It needs someone who can write SQL against your CRM and is allowed to say the pilot did not work.

What breaks at volume is not the arithmetic, it is the discipline: every new channel, renamed source and mid-quarter budget shift reopens the baseline question.

There is also a structural answer. On a pay-per-result appointment setting model rather than a retainer, the denominator is chosen for you: you pay on booked qualified appointments, so cost per booked appointment is the invoice. That removes an accounting problem, not a strategic one — you still have to decide whether the appointments are worth their price. Wider context: our overview of AI for business, and the Australian marketing ROI benchmarks for 2026 to compare yourself against.

Frequently asked questions

What is ROI in marketing, and how is it different from ROAS?

Marketing ROI is (revenue attributable to marketing minus total marketing cost) divided by total marketing cost, expressed as a percentage. ROAS is revenue divided by ad spend only, expressed as a multiple. ROAS ignores platform fees, agency costs, staff hours and gross margin, so the same campaign can show a 3x ROAS and a negative ROI.

Can I just use the attribution report in my CRM?

Not as proof. Observational attribution credits the channel that touched the buyer, not the channel that caused the purchase. In the eBay field experiments of Blake, Nosko and Tadelis (Econometrica, 2015), ordinary regression methods implied a paid-search ROI of over 4,100% without time and geographic controls, and over 1,400% with them, while the randomised version of the same measurement produced minus 63%, with a 95% confidence interval of minus 124% to minus 3%. When eBay switched off its own brand-keyword ads, 99.5% of the lost paid clicks were immediately picked up by natural search. Use the CRM report to steer weekly; use a holdout to decide.

How big does the lift have to be before I can prove it?

Bigger than most people assume. Lewis and Rao, reviewing 25 large field experiments with US retailers and brokerages in the Quarterly Journal of Economics, found the median confidence interval on advertising ROI was over 100 percentage points wide, and that informative experiments can easily require more than 10 million person-weeks. Most mid-market teams will never reach that scale, which is the argument for measuring AI at the funnel step it changed, where the effect is large relative to the noise, rather than at the top line.

How long before AI marketing ROI shows up?

Add your sales cycle to your measurement window. Process metrics such as speed to first contact and lead-to-appointment rate move within days. Revenue from the same cohort cannot appear until the cycle completes, so with a 60-day cycle the first honest read on a 90-day cohort is at day 150. Anyone reporting closed-won ROI at day 30 is reporting deals that were already in the pipeline.

Why does my AI marketing ROI look different in every report?

Usually three unrecorded decisions: which denominator, which costs are included, and whether revenue is credited to the close date or the lead-creation date. In the worked example above those three choices span roughly 50 ROI points on identical data. Write them down once, date them, and require every report to state them.

Should I run a holdout if my baseline is unusable?

Yes, if your volume supports it. A holdout replaces the contaminated before-period with a concurrent control, which removes seasonality and taxonomy drift in one move. Hold back 20% to 50% of comparable leads, keep the assignment random rather than by rep or region, and check the sample table above before you start — a holdout too small to detect your expected lift is worse than no test, because it produces a confident null.