Pick and choose: Differences in equipment can cause researchers at different sites to find widely inconsistent results.
kool99 / iStock

In brain imaging studies of autism, location may alter results

A new analysis hints at the source of the rampant inconsistency among brain imaging studies in autism: significant differences among study sites.

By Sarah DeWeerdt
27 January 2020 | 3 min read
Editor’s Note

This article was originally published in October 2019 based on data presented at a conference. The results were published in January 2020 in Human Brain Mapping1.

It’s among the most frustrating problems in autism research: Scientists have conducted hundreds of imaging studies, but they are no closer to identifying a brain signature for the condition.

This is largely because studies come to widely different conclusions.

A new analysis provides hints about the source of this inconsistency: significant differences among study sites in terms of equipment, participants or other factors.

The analysis relies on data from four studies that are part of the Autism Brain Imaging Data Exchange, a program for sharing resting-state functional magnetic resonance imaging scans. The studies each took place at a different institution and included a total of 152 autistic people and 152 typical controls.

“It would be impossible for one lab to collect all this data,” says Ye He, a postdoctoral researcher in Dan Kennedy’s lab at Indiana University in Bloomington. He presented the work 22 October at the 2019 Society for Neuroscience annual meeting in Chicago, Illinois.

Before analyzing imaging data, scientists must first remove sources of noise, such as head motion and breathing movements, that could affect the results. This step has several components, with multiple options that each institution and researcher may approach differently.

That’s one possible explanation for the inconsistent results. And to test it, He and her colleagues ran all 304 participants’ imaging data through each of 33 different pipelines of options.

Fortunately, says He, “we have access to a supercomputer.”

The team analyzed functional connectivity in the participants’ brains — looking at the regions that tend to activate and deactivate in synchrony when a person is resting quietly in the scanner.

They averaged the functional connectomes of the autism and control groups at each site. No matter which pipeline they used, the average connectome of each group looked similar.

“So the basic human functional connectome is quite stable,” He says. “That’s promising.”

Group differences:

The team then looked at differences between autism and control groups. Differences at any given site remain consistent regardless of the pipeline. But across sites, the group differences are not reliable.

Differences among sites could include the make and model of scanners at the sites, or the characteristics of the participants the site’s researchers are able to recruit. “We are not sure which factor is responsible,” He says.

Such negative results might seem deflating, but they provide valuable information to scientists on what to improve, says Marion Criaud, a postdoctoral researcher at King’s College London who was not involved in the work. “It’s very nice to see something like this,” Criaud says. “We need to be able to publish things that say we cannot replicate [results].”

He and her colleagues hypothesized that they might be able to detect group differences more reliably if they analyzed only the connections that show the strongest differences between autistic people and controls, rather than entire functional connectomes.

But these connections, too, differ among sites. Some strong links emerged from two or rarely even three studies, but not a single one turned up in all four.

For more reports from the 2019 Society for Neuroscience annual meeting, please click here.

  1. He Y. et al. Hum. Brain Mapp. Epub ahead of print (2020) PubMed