Validating autism subtypes: A crucial but often overlooked step in research

Studies of autism subtypes rarely validate their results, and this has led to a proliferation of autism subtypes of questionable utility. But reliable subtyping can help improve the prognosis for and care of autistic people.

Illustration by Kevin Mercer
Illustration by Kevin Mercer
Listen to this story:

The practice of categorizing autistic people into subtypes based on similarities in their traits and abilities is divisive. Subtypes can have negative connotations, evoking images of stereotyping and marginalization.

For decades, the autism spectrum was, by definition, a collection of subtypes, including Asperger syndrome and pervasive developmental disorder-not otherwise specified. But there was no clear clinical distinction between the subtypes, and they did not fully capture the inherent variation among people on the spectrum. So the fifth and most recent edition of the Diagnostic and Statistical Manual of Mental Disorders, to which clinicians refer to make diagnoses, retired them from use in 2013.

That said, there are often good reasons for subtyping. Identifying subtypes of people who share particular genetic variants may be useful, because these variants may be associated with specific medical issues. Subtyping analysis can also be used to demonstrate the nonexistence of certain subtypes. Or it can help researchers to identify who benefits most from a particular kind of support, without focusing on etiology or ontology.

For these reasons, we should not categorically stop conducting subtyping analyses. But research should focus on the discovery of meaningful subtypes of autism. To seek consensus among scientists on the number and nature of subtypes, we conducted a systematic review of the autism subtyping literature. We limited our search to articles published since 2001 that had used a statistical or machine-learning method to discover subtypes of autistic people. These subtyping methods are data-driven: The researchers did not search for a specific number of subtypes and did not specify in advance what the subtypes would look like; they let the data speak for itself.

We identified 156 articles that met our criteria. Of these, 82 percent found that two to four subtypes described their data well. But these subtypes reflected a highly diverse set of measures, including levels of inflammatory markers, scores on autism trait and sensory sensitivity questionnaires, tests of language skills, hormone levels and patterns of facial features, and this diversity made it difficult to find consensus or draw any firm conclusions. Because the samples included variables that are so heterogeneous across many of these studies, it is impossible to determine whether researchers were looking at the same subdivision from different angles or discovering different subdivisions every time.

Furthermore, we noticed that few studies took additional steps to validate their subtype results and support their claims. Thus, we concluded that a lack of systematic validation has led to a proliferation of autism subtypes of questionable utility. We recommend that researchers systematically validate their results and back them up with additional supporting evidence, especially if a sample is small or the result is otherwise not unambiguous.

Validation strategies:

In our review, published in the July issue of Clinical Psychology Review, we outline seven ways to provide convincing supporting evidence for subtyping analyses. We also provide a “Subtyping Validation Checklist” that researchers can use. These validation strategies call for more analysis, more measurements or more participants but can make any subtyping results all the more interpretable and valuable.

The prototypical form of validation is independent replication, in which the entire recruitment, measurement and analysis procedure is repeated with a second group of participants. Only 9 percent of the articles we reviewed included an independent replication sample, however. Even more rare were studies that looked at the stability of subtypes over time, although there are many calls in the literature for more research in that direction.

Most articles — 88 percent — used a strategy called external validation, which involves comparing subtypes on additional variables not used in the original analysis. For example, external validation may consist of discovering that one subtype has higher quality of life or fewer comorbid diagnoses than other subtypes. But few articles explicitly stated how additional variables validated their results or what would invalidate them. If one subtype is older or has more women than the other subtypes, for instance, it is difficult to interpret the implications of such findings for the validity of the subtypes if the researchers did not specify a hypothesis or rationale for focusing on these demographic variables.

To improve this situation, we have two additional recommendations. First, we ask that researchers describe which validation strategies they have used in their studies and explain their rationale for choosing these approaches. Preferably, they would preregister their validation strategies online and outline what would constitute validation and invalidation of their results. The checklist we created can aid this process.

Second, and more importantly, we ask researchers to explicitly state the goal of their subtyping analysis. There are many reasons to study subtypes. Much of the discord around these kinds of studies seems to arise when subtype labels are reified — in which case these labels may come to be seen as unmodifiable traits that determine somebody’s fate or worth. We would argue that this is rarely — and should not ever be — the goal of subtyping research. Instead, the ultimate goal of this kind of work should be to improve prognosis and care.

Hilde Geurts is professor of clinical neuropsychology at the University of Amsterdam in the Netherlands and senior researcher in the autism clinic of Leo Kannerhuis. Joost Agelink van Rentergem is a postdoctoral researcher at the University of Amsterdam.