Correlating two habits without fooling yourself

Once you have a year of two habits it is tempting to correlate them, and the result usually looks impressive. Reading completion is 34 percent higher on days you meditated. Steps above 8 000 predict a completed evening routine. These are real patterns in real data and they are worth looking at.

They are also extremely easy to over-read. Two habits that both happen on weekdays will correlate strongly with each other because they both correlate with the weekday, and neither one causes the other. If you have ten habits you have 45 possible pairs, and at conventional thresholds a couple of them will look significant by chance alone. And the direction is undetermined: meditation days might produce reading, or good days might produce both, which is the more likely explanation most of the time.

None of this means stop looking. It means treat a correlation as a hypothesis you can test, because unlike most people staring at charts, you can actually run the experiment. Deliberately meditate on five randomly chosen days over two weeks and see whether reading follows. That is a small, real experiment and it takes a fortnight.

This page covers how to compute the correlation from an export, how to control for weekday, the multiple comparisons problem in plain language, and how to design the two week test.

Outline

Status: outline. The introduction above is finished copy. Everything below is the section plan for the full draft, targeted at about 1 600 words.

1. Computing it. Two habits, one year, and the four lines that produce a number.
2. Controlling for weekday. The confound that explains most impressive results.
3. Forty five pairs. Multiple comparisons, without the jargon.
4. Direction. Three causal stories that fit the same number.
5. The two week test. Randomising your own days, and reading the result.
6. What to do with a real effect. Turning it into a stack, and re-testing later.