Incrementality testing is the gold standard for attribution truth — and most operators avoid it because the holdout cost feels expensive. The math is more favorable than it appears: a 6-week 10% holdout on a $200K/month channel costs roughly $30-50K of foregone revenue, but the insight (knowing whether your channel is actually 100% incremental or only 60%) typically informs $200K+ of better budget allocation annually.
This calculator computes the test ROI: foregone revenue during holdout × duration = direct cost. Compare to insight value (the difference between reported and true incrementality applied to annual spend) — if the channel is actually 60% incremental and you\'re budgeting based on 100%, you\'re overspending by 40% which is the value of knowing the truth.
The strategic recommendation: run incrementality tests once your annual spend on a single channel exceeds $500K and you suspect attribution overstates impact. Geo-holdout is the cleaner test design (pause spend in 10-15% of regions for 4-8 weeks). Audience-holdout (Meta\'s Conversion Lift tool, Google\'s Geo Experiments) is easier to run but Meta/Google-managed. Run tests sequentially, not simultaneously across channels, to avoid interaction effects.
Pair with the MTA Platform Comparison calculator (the alternative measurement approach), Tracking Gap Estimator (post-iOS14 measurement loss), Channel Mix Modeler (the macro-level allocation framework), and Click vs View Attribution Gap calculator (the underlying problem incrementality testing solves). Most successful brands run 2-4 incrementality tests per year on their largest channels — the cost is meaningful but the resulting allocation decisions typically save 5-10× the test cost.
Frequently asked questions
What's an incrementality test?
You hold out spend in a controlled geography or audience for 4-8 weeks while continuing spend everywhere else. Compare revenue lift in the spending region vs holdout to measure true incremental impact (vs measured impact, which often overstates because of last-click bias).
How much does the holdout cost?
Direct cost: the revenue you would have generated in the holdout region. For a $100K monthly spend with 2× ROAS, holding out 10% of geos = $20K/month foregone revenue × 2 months = $40K cost. Plus 3-5% measurement infrastructure cost.
When is it worth doing?
When you spend $100K+/month on the channel being tested AND attribution feels suspicious (Meta says it's carrying 60% of revenue but you can't replicate that in pause tests). Below $100K spend, the holdout cost outweighs the insight value. Above $500K, it's usually mandatory.
Geo-holdout vs audience-holdout?
Geo-holdout: pause spend in 5-15% of geographic regions. Cleaner signal but bigger absolute cost. Audience-holdout: hold out a random 5-15% of users (Meta's "Conversion Lift" tool). Smaller cost but Meta-managed test, less control. Use geo for cross-channel tests; audience for single-channel (Meta or Google) tests.
How long should the test run?
Minimum 4 weeks per test. 8 weeks for high-AOV / long-cycle products. Below 4 weeks, conversion lag and attribution windows muddy the signal. Run multiple tests sequentially across different channels rather than testing them all at once (avoids interaction effects).