Ten years ago science journalists talked about functional MRI (fMRI) scans all of the time. Because if a part of the brain lit up when someone did, said, or read something, it went into a paper. Few asked who was doing the interpreting, how legitimate the scale was, and if it had any scientific relevance. We got media claims that fMRI would predict behavior and the resulting media attention caused scholars to rush to produce even more fMRI papers.

Yet the science is not clear on whether or not fMRI studies can achieve anything. It is contradictory. It has no known relevance to individuals. All fMRI does is show blood flow but some neuroscientists contend that will mean more neurons are working and therefore that Y part of the brain controls X behavior. It does not. The level of activity for any given person won't be the same twice. If a measure changes every time it is collected it can't predict future mental health or behavior any more than surveys of undergraduate students can tell us anything about who will win an election.

A new meta-analysis re-examined 56 studies based on fMRI data to gauge their reliability across 90 experiments. They also examined data from the brain-scanning Human Connectome Project, the closest thing brain science has to a glossary, and looked at test/retest results for 45 individuals. For six out of seven measures of brain function, the correlation between tests taken about four months apart with the same person was weak. The seventh measure studied, language processing, was only a fair correlation. 

Finally they looked at data they collected through the Dunedin Multidisciplinary Health and Development Study in New Zealand, in which 20 individuals were put through task-based fMRI twice, two or three months apart. Again, they found poor correlation from one test to the next in an individual.

The bottom line is that task-based fMRI in its current form can't tell you what an individual's brain activation will look like from one test to the next.

How to fix it?

Medicine does not use symptom-based diagnosis and fMRI, the BMI of neuroscience, may not survive as a valid tool either, but if it wants to survive, scans will need to be longer than a few minutes. Just like lots of people have lower blood pressure after the doctor leaves the room, it may need an hour of scans for fMRI to get a more accurate picture.

Studies claiming to show people are different based on them may just fade away, leaving MRI to the more robust area of brain structure. 

And it may be that connectivity mapping, seeing how areas of the brain are connected to address a task instead of just what areas are active, becomes the norm. 

Citation: Maxwell L. Elliott, Annchen R. Knodt, David Ireland, Meriwether L. Morris, Richie Poulton, Sandhya Ramrakha, Maria L. Sison, Terrie E. Moffitt, Avshalom Caspi, Ahmad R. Hariri, 'What is the Test-Retest Reliability of Common Task-fMRI Measure? New Empirical Evidence and a Meta-Analysis,' Psychological Science, June 3, 2020. DOI: 10.1177/0956797620916786