01. Introduction to Causal Inference: "Moving Together" Isn't the Same as "Because Of"

Two numbers moving together on a dashboard is the easiest thing in business to misread. This chapter is about the gap between "these two things move together" and "one of them happens because of the other" — and about building a way of thinking that keeps you from crossing that gap by accident.

Why This Matters

Most of the questions a business actually needs answered aren't prediction questions — they're causal questions. "Which users are likely to churn next month" is a prediction question: a model can answer it well using nothing but correlations. But "if we send this coupon, will it reduce churn" is a causal question, and no amount of correlational skill answers it on its own.

Here's a concrete way the two get confused. Suppose a dashboard shows that users who received a discount coupon last quarter have lower average revenue than users who didn't. Read causally, that looks like "discounts hurt revenue." But the far more likely story runs the other direction: the company tends to send discounts to users who were already spending less — at-risk users, lapsing users, low-engagement users. The coupon didn't cause the low revenue. The low revenue (or the risk of it) is why the coupon was sent in the first place. The correlation is real; the causal story implied by reading it naively is backwards.

Let's Start With One Dashboard Number

Say the marketing team ran a coupon campaign, and the post-campaign dashboard shows this: users who received the coupon spent an average of $550 afterward; users who didn't receive it spent an average of $100. The gap is $450, and it's tempting to write that down as "the coupon's effect."

That $450 is not the effect of the coupon. It's the difference between two groups of users, and those two groups were not assigned at random — the coupon almost certainly went to a specific kind of user (existing high-value customers, or re-engagement targets, depending on the campaign's targeting rule) rather than to a random half of the user base. Whatever kind of user tends to receive the coupon is exactly the kind of user whose spending would have looked different anyway, coupon or no coupon. The naive comparison bundles the coupon's real effect together with whatever pre-existing difference the targeting rule created — and there is no way to separate the two just by staring at this one number.

Thinking in Parallel Universes

Causal inference solves this with a specific mental move: imagine two parallel versions of the exact same user — one where they received the coupon, one where they didn't — and ask how much their spending actually differs between those two versions. That difference, for that one user, is their true individual effect. It has nothing to do with which other users received a coupon; it's a comparison of the same person against themselves in a world that didn't happen.

The obvious problem: we only ever get to observe one of the two universes for any given user. If they received the coupon, we see their coupon-world spending, and their no-coupon-world spending is forever hidden. If they didn't receive it, it's the reverse. This hidden half is what the field calls the counterfactual — literally, "contrary to fact." Causal inference, at its core, is a set of disciplined ways to reason about a number you will never directly observe.

If We Could See From God's Eye View

To make the idea concrete, imagine we secretly had access to both universes for four users — something no real dataset ever gives you. Here's what that impossible, complete table would look like:

Averaging the "true effect" column across all four users gives the Average Treatment Effect (ATE): (150 + 30 + 50 + 70) / 4 = $75. That's the effect of the coupon, averaged over everyone — as if the coupon had been handed out to the whole population.

But that's not usually the question a business actually needs answered. The more practical question is: among the users who actually received the coupon (A and C), what was the coupon's effect on them specifically? Averaging just their true effects gives (150 + 50) / 2 = $100. This is the Average Treatment Effect on the Treated (ATT) — the effect among exactly the population the company actually chose to target, which is usually the more relevant number for evaluating a campaign that was never going to be sent to everyone anyway.

Now here's the part that makes this table impossible to build from real data. In reality, we never see both columns for the same user — we only ever see the outcome in the world that actually happened. Erasing the hidden half turns the table above into this:

This is exactly the dashboard we started with: coupon recipients averaging $550, non-recipients averaging $100. The true individual effects, the ATE, and the ATT are all still sitting underneath this table — we just built it in the first version and know they're really there. In practice, no one ever gets to see the first table. They only ever see the second one, and have to reason their way back to something like the first.

So Where Did the $450 Actually Come From?

We can now decompose the naive $450 gap exactly, using the God's-eye table. The naive comparison is: (average observed spending of recipients) − (average observed spending of non-recipients) = $550 − $100 = $450.

Rewrite the recipients' side using the hidden column. Their true ATT is $100, so their average no-coupon-world spending (A and C's hidden values, $400 and $500) must average $450 — that's just $550 (observed) minus their true effect ($100). So the naive comparison is really:

$450 (naive gap) = $100 (the recipients' true effect, ATT) + $350 (the pre-existing gap between what recipients would have spent anyway, $450, and what non-recipients actually spent, $100).

That second term — $350 — is bias (also called selection bias). It has nothing to do with the coupon. It exists purely because the two groups were different kinds of users before the coupon was ever sent. The naive comparison can't tell you how much of the $450 is real effect and how much is bias, because it never sees the hidden column that would let it separate the two.

So How Do You Get Rid of the Bias?

The bias term above exists because which users received the coupon was correlated with what those users would have spent anyway. If that correlation is broken — if receiving the coupon has nothing to do with what kind of spender you are — the bias term disappears, and the naive comparison becomes the true effect.

The most reliable way to break that correlation is to decide who gets the coupon with a coin flip. A coin has no idea whether it's about to land on a heavy spender or a light one — so, on average, the group that gets heads and the group that gets tails start out identical in every way, including in ways nobody bothered to measure. This is the entire logic of A/B testing, and it's worth stating plainly: A/B testing isn't fundamentally a statistics trick. It's a bias-removal trick. The statistics that come afterward exist to handle sampling noise — a separate problem from the one randomization solves.

A Practical Checklist

Before trusting any comparison between a "treated" group and an "untreated" group, ask two questions:

• Who actually received the treatment, and how were they chosen? Was it a coin flip, or was it a rule — a targeting algorithm, a self-selected sign-up, an executive decision, an accident of timing?
• Would the treated group have looked different anyway, even without the treatment? If the answer is plausibly yes, a naive comparison is contaminated with bias, and the gap you're looking at is some unknown mix of real effect and pre-existing difference.

Terms We'll Use Going Forward

The rest of this book uses a small set of textbook terms for the ideas introduced above. Here's the translation:

With this vocabulary in place, the next chapter looks at exactly how randomization eliminates the bias term — and works through a full 3-arm coupon experiment end to end, including the checks you should always run before trusting a randomized result.