How do randomised trials work on farms?
Treatments are randomised to strips, plots, animals, herds or whole communities and compared side by side. In 113 replicated strip trials in Iowa and New York, a model’s nitrogen advice used 34 per cent less sidedress nitrogen with no significant difference in yield. What you randomise, and the outcome you measure, decide what a trial can tell you.
Module 2's last quick check: before any data are collected, a map can list the rival explanations and say whether the data could answer the question at all. When the answer is "not yet", one of the strongest next steps is a trial. This module is the toolbox of designs.
In Iowa and New York, researchers wanted to know whether a computer model's nitrogen advice beat the farmer's own judgement. They didn't compare farms that used the model with farms that didn't: adopters might differ in all sorts of ways. Instead they ran 113 replicated strip trials. On each farm, strips at the model's rate sat alongside strips at the grower's rate, repeated across the field.
The model's strips used 34 per cent less sidedress nitrogen on average, with no significant difference in yield, and an estimated 65 US dollars a hectare more in partial profit. The farms were volunteers and the prices historical, so the result travels only so far. But inside each farm, the comparison was fair.
Some treatments can't be given to one family and withheld from the next. A nutrition and hygiene programme works through a whole community: shared training, shared markets, neighbours talking. So in the Kenya trial from Module 2, whole groups of farming families were randomised, not individuals. That's a cluster-randomised trial.
The catch is that families in a cluster are alike. A hundred families in ten clusters tell you less than a hundred families randomised one by one. In a cluster trial, the number of clusters matters more than the number of people. On farms it's the same. If a practice is decided for a whole farm, the farm is the unit, not each field or each animal.
With animals, the unit can be the individual. In the seaweed trial from Module 1, each steer was randomised to a diet. But when a whole herd shares one ration, the herd is the unit, and you need many herds, not many cows. Even a perfect randomisation answers only the question its outcome measures.
In the US, 5,005 parents were randomly shown one of seven drink labels online and asked to choose a drink for their child. Warnings plus teaspoons of added sugar cut the sugar in the drinks they chose by 14 per cent. A strong design. But the outcome was a one-off choice on a screen, not what children actually drank over months. A measurement that stands in for the outcome you really care about is called a surrogate.
The seaweed trial's methane per kilogram of feed is a surrogate for a farm's footprint, too. A trial is only as useful as its outcome is close to the real one. Next: when you can't randomise at all, and have to compare like with like.
Some treatments can’t be given to one family and withheld from the next. A nutrition and hygiene programme works through shared training, shared markets and neighbours talking, so in the Kenya trial whole groups of farming families were randomised. Because families in a cluster are alike, a hundred families in ten clusters tell you less than a hundred randomised one by one.
A measurement that stands in for the outcome you really care about. In a US trial, 5,005 parents chose drinks for their children after seeing labels online; warnings with teaspoons of added sugar cut the sugar chosen by 14 per cent. But a one-off choice on a screen isn’t what children actually drank over months.
Causal evidence is changing how farming is measured and rewarded. Leave your email and we will tell you when new films, courses and findings are published.
Prefer to read online? Vitagri also writes on Substack. See our privacy policy for how we use your email.