What is propensity score matching?
A propensity score is a unit’s probability of being treated, given everything you measured. In 1983 Paul Rosenbaum and Donald Rubin showed that units with the same score are, on average, balanced on all those characteristics at once. You can match on that one number, or weight by it, so the groups you compare line up on what you measured.
Last time's quick check: families in a cluster are alike, so each extra cluster adds more information than each extra person. When you can't randomise, you can try to build the comparison randomisation would have given you. Matching pairs each treated farm with an untreated farm that looks the same on everything you measured: size, soil, rainfall, enterprise. Compare the pairs, and the measured differences are taken out of the comparison.
With many characteristics, finding an exact twin is hopeless. In 1983, Paul Rosenbaum and Donald Rubin showed that you don't need one. Work out each farm's propensity score: its probability of being treated, given everything you measured. Farms with the same score are, on average, balanced on all of those characteristics at once.
Match on that one number. Or weight each farm by it instead, so that farms which were unlikely to be treated but were count for more. Either way, the groups you compare line up on what you measured. Then you check. Put the treated and comparison groups side by side, characteristic by characteristic. If they still differ, the matching hasn't worked, and the estimate can't be trusted.
Good studies show this balance check, and it's one of the first things to look for when you read one. In India, a study used the national survey of farm households to compare smallholders using seed from formal sources, such as certified seed, with similar farmers using saved or informal seed.
After propensity-score matching, paddy yields were estimated over 20 per cent higher, arhar, or pigeon pea, nearly 50 per cent higher, and paddy net income 23 per cent higher. But the study's own caution applies to every matched study. Matching balances what you measured. Farmers who seek out certified seed may also be better connected, better informed or better financed. If the survey didn't record that, it's still in the answer.
Whether like-with-like on paper is like-with-like in reality depends on your map, and on what you couldn't measure. That's where Module 2's dashed boxes and the prologue's sensitivity analysis come back in. Next: before and after, with a comparison.
It can’t balance what you didn’t measure. In the Indian seed study, farmers who seek out certified seed may also be better connected, better informed or better financed. If the survey didn’t record that, it’s still in the answer. That’s where the dashed boxes on a causal map, and sensitivity analysis, come back in.
After matching or weighting, you put the treated and comparison groups side by side, characteristic by characteristic. If they still differ, the matching hasn’t worked and the estimate can’t be trusted. Good studies show this check, and it’s one of the first things to look for when you read one.
Causal evidence is changing how farming is measured and rewarded. Leave your email and we will tell you when new films, courses and findings are published.
Prefer to read online? Vitagri also writes on Substack. See our privacy policy for how we use your email.