What is collider bias, or selection bias?
Collider bias appears when you hold fixed, or select on, something that both the treatment and the outcome, or their causes, feed into. Choosing which data to look at can do it without anyone noticing. Analysing only animals that reached slaughter, only samples sent to a lab or only published studies can create a link that isn’t there.
Last time's quick check: adjust for a mediator and you're left at best with the direct effect, the part that doesn't travel through it, and only if nothing else links the mediator to the outcome. Today: the collider, the variable you must never hold fixed.
In 2.2 we met a collider. Farmers send grain to the lab when the soil is good or when they've tried something new. Look only at lab samples, and treated fields seem to have poorer soil. Statisticians call this Berkson's bias, after Joseph Berkson, who described it in 1946 using hospital patients. Two unrelated illnesses appeared linked, simply because either one could put a patient in hospital.
It matters for farming because so much farm data is selected: samples sent to labs, farms that join schemes, animals that reach slaughter. Here's a livestock version. Suppose a treatment helps weak calves survive. Now compare carcass weights at slaughter, treated against untreated. The treated group may come out lighter.
Did the treatment slow their growth? Not necessarily. It changed who reached slaughter. Without it, the weakest calves died and never appeared in the slaughter records. With it, they survived, and pulled the treated group's average down. On the map, survival is a collider: the treatment points into it, and so does each calf's underlying frailty, which also affects carcass weight. Analysing only animals that reached slaughter holds survival fixed, and opens a route from treatment, through frailty, to carcass weight.
The fix starts with the question. Count every animal that started, not just those that finished. Selection hides in ordinary places. Farms that volunteer for a trial. Fields with complete records. Studies that get published, because striking results are easier to publish than dull ones.
Not every selection biases the comparison. The US nitrogen trials ran on volunteer farms, but the strips were compared side by side on the same farms, so the comparison holds. Volunteering limits how far the answer travels, not whether it's fair. The publishing filter applies to us. The twelve case studies on vitagri.org are positive findings, chosen to show what these methods can do.
They show the methods working, not how often a practice works. A fair reading of any evidence needs the studies that found nothing, too. So how do you spot one before it fools you? Ask of every variable you plan to adjust for, or filter on: what causes it?
If the treatment and the outcome, or their causes, both feed into it, it's a collider, and holding it fixed will create a link. Filters count. "Only fields with a lab result" is a filter. "Only animals that reached slaughter" is a filter. The map makes them visible, because each one is a box with arrows coming in from both sides.
Next: the last of the cast. Variables that change how big an effect is, variables that let you measure an effect from outside, and the ones nobody measured.
If the treatment helps weak calves survive, it changes who reaches slaughter. Without it, the weakest calves died and never appeared in the slaughter records; with it, they survived and pulled the treated group’s average down. Survival is a collider, so analysing only the animals that reached slaughter opens a route through each calf’s frailty.
Yes, and the series says so. The twelve case studies on vitagri.org are positive findings, chosen to show what the methods can do. They show the methods working, not how often a practice works. A fair reading of any evidence needs the studies that found nothing, too.
Causal evidence is changing how farming is measured and rewarded. Leave your email and we will tell you when new films, courses and findings are published.
Prefer to read online? Vitagri also writes on Substack. See our privacy policy for how we use your email.