4.4 Can AI find causes? | Causal Inference

Key points

The PC algorithm, published in 1991, rebuilds maps from patterns of what moves with what.
A family of maps that fit the data equally well is called an equivalence class.
Some of the benchmark success of newer methods came from a quirk in how test data were simulated.
A map drafted by AI still has to be checked by experts and tested: it has to earn its arrows.

Chapters

Key terms

Causal discovery: Methods that try to draw a causal map from data, by reading the patterns chains, forks and colliders leave behind, as the PC algorithm does. From observational data they usually narrow things to a family of maps, and they need strong assumptions. They are best used to draft a map for experts to check and test.
Equivalence class: A family of causal maps that fit the same observational data equally well. A chain and a fork leave the same fingerprint, so data alone can’t tell them apart, and causal discovery usually ends with an equivalence class rather than a single map.
Collider: A variable with two arrows pointing into it, such as whether a grain sample reaches the lab when either good soil or a new treatment can send it there. A collider blocks association between its causes until you hold it fixed, by adjusting for it or selecting on it. Then a link appears from nowhere.
Causal map (DAG): A diagram that draws each thing that varies as a box and each direct effect as an arrow, with no numbers. Also called a directed acyclic graph: arrows point one way and no route returns to its start. Sewall Wright’s path diagrams of 1921 were the first. A map puts assumptions where experts can check them.

Transcript

Last time's quick check: a shuffled treatment has nothing to do with the outcome, so any "effect" it shows is the method finding noise. Wouldn't it be easier if a computer drew the causal map for you? Feed it the data, get back the arrows. That's causal discovery, and it's one of the most active areas of AI research.

Discovery algorithms read fingerprints. Recall Module 2: chains, forks and colliders leave different patterns of what moves with what. A collider is distinctive, because its two causes are unrelated until you hold it fixed. In 1991, Peter Spirtes and Clark Glymour published an algorithm, known as PC after their first names, that reads those patterns to rebuild a map.

But chains and forks leave the same fingerprint. A chain from A through B to C and a fork with B in the middle look identical in the data. So from observational data alone, an algorithm can usually narrow things down to a family of maps that fit equally well. That family is called an equivalence class. It's rarely one map.

Even that needs strong assumptions: that no important common cause is missing from the data, and that real effects never cancel each other out by coincidence. Newer methods turn discovery into an optimisation problem, and look impressive on benchmarks. But in 2021, researchers showed that some of that success came from a quirk in how test data were simulated, not from finding causes.

On real farm data, with missing variables and measurement error, a discovered map can change with small changes to the data. What about large language models? They've read much of what's been written about causes, and they can suggest plausible arrows: rainfall affects yield, not the other way round.

In published tests they do well at naming the direction of familiar cause-and-effect pairs. But given formal causal reasoning problems, such as a set of correlations and a question about what follows, they make many mistakes. One paper called them "causal parrots": they repeat causal knowledge from text without reasoning about it.

So use AI as a drafting partner. A discovery algorithm can suggest arrows you hadn't considered; a language model can gather what's been written. Then put the draft in front of experts, test what it predicts, and choose the design. The map still has to earn its arrows. Next, in Module 5: the chain from soil to plate.

Questions

Why can’t an algorithm tell a chain from a fork?

Because they leave the same fingerprint. In a chain from A through B to C, and in a fork with B in the middle, A and C move together, and holding B fixed separates them. Only colliders are distinctive. So from observational data alone, an algorithm usually narrows things down to an equivalence class of maps.

Are large language models good at causal reasoning?

In published tests they do well at naming the direction of familiar cause-and-effect pairs, such as rainfall affecting yield. But given formal causal reasoning problems, such as a set of correlations and a question about what follows, they make many mistakes. One paper called them “causal parrots”.

Sources

Spirtes, P., Glymour, C. and Scheines, R. (1993). Causation, prediction, and search. Lecture Notes in Statistics. New York: Springer. doi:10.1007/978-1-4612-2748-9
Spirtes, P. and Glymour, C. (1991). An algorithm for fast recovery of sparse causal graphs. Social Science Computer Review, 9(1), 62–72. doi:10.1177/089443939100900106
Reisach, A.G., Seiler, C. and Weichwald, S. (2021). Beware of the simulated DAG! Causal discovery benchmarks may be easy to game. arXiv:2102.13647. doi:10.48550/arXiv.2102.13647
Zečević, M., Willig, M., Dhami, D.S. et al. (2023). Causal parrots: large language models may talk causality but are not causal. arXiv:2308.13067. doi:10.48550/arXiv.2308.13067
Kıcıman, E., Ness, R., Sharma, A. et al. (2023). Causal reasoning and large language models: opening a new frontier for causality. arXiv:2305.00050. doi:10.48550/arXiv.2305.00050
Jin, Z., Liu, J., Lyu, Z. et al. (2023). Can large language models infer causation from correlation? arXiv:2306.05836. doi:10.48550/arXiv.2306.05836

All 30 films · Glossary · The Vitagri Academy