Confounders and the back door

Last time's quick check: in a fork, holding the middle fixed removes the link between the two outer boxes, if the fork is the whole story. Today we use that to deal with confounders. In Italy, researchers asked whether organic certification makes olive farms more efficient. But farm size gets in the way: it affects which farms go organic, and how efficiently a farm can run.

So there are two routes between organic and efficiency. The front door is the arrow we care about, from organic to efficiency. The back door starts with an arrow pointing into the treatment: organic, back to farm size, then on to efficiency. It's a fork, and association flows through it. Compare organic and conventional farms naively and you mix the two routes together.

The researchers closed the back door by matching: comparing organic farms with similar conventional farms. That's holding the middle of the fork fixed. They estimated technical efficiency about 10 per cent higher on organic farms. Careful, though: that's efficiency, not 10 per cent more olives.

The computer scientist Judea Pearl turned this into a rule in the 1990s. To estimate an effect from data you didn't control: One: list every route between the treatment and the outcome that starts with an arrow into the treatment. Those are the back doors. Two: close each one, by holding fixed a variable that sits on it. Three: don't hold fixed anything the treatment causes, and don't open a shut door by holding fixed a collider.

The variables you hold fixed are called the adjustment set. Often more than one set works, and you choose the one you can measure best. Sometimes none works, because a back door runs through something nobody measured. Then no amount of modelling will rescue the comparison.

Now a trap that catches experienced analysts. Open a farm data set, and every column that isn't the treatment or the outcome gets called a covariate. The habit is to put them all in the model, to be safe. But "covariate" is a label, not a role. Soil pH might be a confounder: put it in. Yield, measured after the nitrogen went on, is something the treatment changed: leave it out. Whether a sample was sent to the lab is a collider: never adjust for it. Same spreadsheet, three different instructions, and only the map tells you which is which.

Adding the wrong column doesn't just waste effort. It can create bias that wasn't there. This is how the Greek cotton study from the prologue worked. The researchers drew a causal map of the sowing decision first, used it to decide which differences between fields to balance, and only then compared fields, with matching and with weighting.

Across 171 fields, the estimates ranged from 12 to 17 per cent more yield when sowing followed the advice. Different methods, similar answers: that's reassuring. But every one of them could only close the back doors the researchers could see and measure. Next: a variable that can look like a confounder but sits somewhere very different on the map, between the treatment and the outcome.