The ‘autism epidemic’ and diagnostic substitution

Based on: King & Bearman (2011) American Sociological Review, 76(2), 320-346; 

Data from birth and diagnostic records for all children born in California 1992-2000

Everyone agrees there has been a remarkable increase in autism diagnosis across the world. There is, however, considerable debate about the reasons for this. Three very different kinds of explanation exist.

• Explanation #1 maintains that something in our modern environment has come along to increase the risk of autism. There are numerous candidates, as indicated in this blogpost by Emily Willingham. 
• Explanation #2 sees the risks as largely biological or genetic, with changing patterns of reproduction altering prevalence rates, either because of assortative mating (not much evidence, in my view) or because of an increase in older parents (more plausible). 
• Explanation #3 is very different: it says the increase is not a real increase - it’s just a change in what we count as autism. This has been termed ‘diagnostic substitution’ - the basic idea is that children who would previously have received another diagnosis or no diagnosis are now being identified with autism spectrum disorder (ASD). This could be in part because of new conceptualisations of autism, but may also be fuelled by strategic considerations: resources for children with ASD tend to be much better than those for children with other related conditions, such as language impairment or intellectual handicaps, so this diagnosis may be preferred.

In 2008, my research group published a study that documented one kind of diagnostic substitution. We contacted people who had taken part in our studies of children with specific language impairment years ago. We carried out a standard diagnostic observation procedure for autism with the young adults themselves and, where possible, interviewed their parents about their early history. We found a number of individuals who had been regarded as cases of specific language impairment ten or twenty years ago but who would nowadays be diagnosed with ASD. Although it’s possible that some people develop autistic symptomatology as they get older, in our cases the autistic symptoms appeared to have been present from early childhood - as indicated by the parental interviews. Around half of the sample had been identified as having ‘semantic-pragmatic disorder’ in childhood, but autism had been excluded because at that time, prior to publication of DSM-IV diagnostic guidelines, it was regarded as a very rare condition in which there were severe social and behavioural impairments. How many children would have qualified for ASD diagnoses had they been seen today? Well, it depends. I suspect few people appreciate just how flexible the diagnostic criteria are for autism, even when lengthy standardized diagnostic instruments are used. Although we used the gold standard diagnostic procedures (ADOS-G and ADI-R) we found they seldom gave the same answer. If we diagnosed ASD only when both diagnostic instruments agreed, 21% of cases met criteria. If we included anyone who met criteria for autism or PDDNOS on either ADI-R or ADOS, the rate shot up to 66%.

Last year, a fascinating study by Brugha and colleagues attacked the same question from a different angle. They did an epidemiological survey of a representative sample of adults from the English population, using the ADOS-G, and found that the rates of ASD were similar to those recently reported in children. Within the adult population, rates of ASD did not change with age. Thus, provided we stick to the same diagnostic criteria, then the prevalence of autism is the same for those born several decades ago, as it is for the current generation of children. Importantly, none of these adults with ASD had received a formal diagnosis.

Recently, we conducted a study with another group: children with an additional sex chromosome (i.e. trisomy). We had not intended to study diagnostic substitution: the goal was rather to understand more about the language difficulties that had previously been described in children with sex chromosome trisomies. The effect of an extra sex chromosome is relatively mild: most of these children attend mainstream schools and they do not have any obvious physical abnormalities. Indeed, they can be hard to study because many individuals with trisomies will be unaware of their condition. We gathered information by parental report, and did not do any direct evaluation of the child, but we did ask about whether the child had had any kind of diagnosis by a medical or psychological expert. We confirmed that there was a strong association with language problems in all three kinds of trisomy (girls with XXX, and boys with XYY or XXY), many of whom had had speech-language therapy. But we also found that 2/19 (11%) of boys with XXY and 11/58 (19%) of those with XYY had received an ASD diagnosis.

It is important to emphasise that most children with a sex chromosome trisomy did not have an ASD diagnosis, and many were not giving any cause for concern. Nevertheless, although they are only a minority of cases, the proportion with ASD is much higher than in the general population. We were really surprised at this because before publishing our study we had done a systematic review of the literature on children with sex chromosome trisomies, focusing on studies that avoided ascertainment bias. In these studies, not a single case of autism had been mentioned when discussing outcomes. So was our study a fluke? We are confident this is not the case, because this year two further studies from the USA have been reported (Ross et al and Lee et al, in press), both of which got results very similar to ours, though using different methods.

This research provides further evidence that diagnostic substitution has occurred, suggesting that children who in the past would have been diagnosed with language impairment are now being diagnosed with ASD. The only other way to explain the increased diagnosis rate in children with a known chromosomal abnormality would be if the trisomy acted as a risk factor, making children more sensitive to environmental factors that could cause autism. That’s a possibility, but it seems more likely that cases of ASD were missed in the past because more stringent diagnostic criteria were used, just as was found in our follow-up of children with SLI and in the epidemiological study of adults by Brugha and colleagues.

It is becoming clear that changing diagnostic criteria, increased awareness of ASD, and strategic use of diagnosis to gain access to services, have had a massive effect on the numbers of children with ASD. When I started studies in this area, I thought diagnostic substitution had happened but I did not think it would be sufficient to explain the increase in numbers of ASD diagnoses. But now, on the basis of studies reviewed here, I think it could be the full story.

PS: a slightly extended version of this blogpost was featured on PLOS Blogs on 8th June 2012.

References

Bishop, D., Jacobs, P., Lachlan, K., Wellesley, D., Barnicoat, A., Boyd, P., Fryer, A., Middlemiss, P., Smithson, S., Metcalfe, K., Shears, D., Leggett, V., Nation, K., & Scerif, G. (2010). Autism, language and communication in children with sex chromosome trisomies Archives of Disease in Childhood, 96 (10), 954-959 DOI: 10.1136/adc.2009.179747
 
Bishop, D., Whitehouse, A., Watt, H., & Line, E. (2008). Autism and diagnostic substitution: evidence from a study of adults with a history of developmental language disorder Developmental Medicine & Child Neurology, 50 (5), 341-345 DOI: 10.1111/j.1469-8749.2008.02057.x  

Brugha, T. (2011). Epidemiology of Autism Spectrum Disorders in Adults in the Community in England Archives of General Psychiatry, 68 (5) DOI: 10.1001/archgenpsychiatry.2011.38

Lee, N. R., Wallace, G. L., Adeyemi, E. I., Lopez, K. C., Blumenthal, J. D., Clasen, L. S., & Giedd, J. N. (2012, in press). Dosage effects of X and Y chromosomes on language and social functioning in children with supernumerary sex chromosome aneuploidies: Implications for idiopathic language impairment and autism spectrum disorders. Journal of Child Psychology and Psychiatry. 

Ross, J. L.,et al (2012). Behavioral and social phenotypes in boys with 47, XYY syndrome or 47, XXY Klinefelter syndrome.  Pediatrics, 129(4), 769-778. doi: 10.1542/peds.2011-0719



 

Neuroscientific interventions for dyslexia: red flags

I’m often asked for my views about interventions for dyslexia and related disorders. In recent years there has been a proliferation of interventions offered on the web, many of which claim to treat the brain basis of dyslexia. In theory, this seems a great idea; rather than slogging away at teaching children to read, fix the underlying brain problem. If your child is struggling at school, it can be very tempting to try something that claims to re-organise or stimulate the brain. The problem, though, is sorting the wheat from the chaff. There's no regulation of educational interventions and it can be hard for parents to judge whether it is worth investing time and money in a new approach.
My aim here is to provide some objective criteria that can be used. First, there is scientific evaluation: does the intervention have a plausible basis, and how has it been tested? Where claims are made about changing the brain, are they based on solid neuroscientific research? Second, there are red flags, some of which I listed in a previous post on ‘Pioneering treatment or quackery?” Here I've gathered these together so that there is a ready checklist that can be applied when a new intervention surfaces.

1. Who is behind the treatment and what are their credentials?
What you should look for here are relevant qualifications, particularly a higher degree (preferably doctorate) from an academic institution with a good reputation. Red flags are:

• No information about who is involved ▶#1 
• Intervention developed by someone with no academic credentials ▶#2 
• Citation of spurious credentials; affiliation with organisations that have very lax membership criteria, e.g., Royal Society of Medicine ▶#3 
• A lack of publications in peer-reviewed journals. Publications only in books counts as a red flag, because there is no quality control. ▶#4

It can be hard for a lay person to evaluate point #3, because some people cite qualifications that sound impressive but have no credibility. Academics in the field, however, can quickly identify whether a string of letters is indicative of prestige, or whether they are a smokescreen for lack of formal qualifications.
As far as #4 is concerned, relevant information can be obtained checking an author against a database such as Web of Science. However, access to such databases is largely restricted to academic institutions. Google Scholar is widely available, though its results are not restricted to peer-reviewed literature.

2. Is there a credible scientific basis to the treatment?
This is often difficult for a lay person to evaluate. Google Scholar may be helpful in tracking down articles that discuss the background to the intervention. Ideally, one is looking for a review by someone who is independent of those who developed it and who has good academic credentials. If no relevant journal articles are found on Google Scholar this is a red flag ▶#5. If a journal article is found, try to find whether the journal is a mainstream peer-reviewed publication.

3. Who is the intervention recommended for?
It is implausible that the whole gamut of neurodevelopmental problems has a single underlying cause, and it is unlikely that they will all respond to the same intervention. If an intervention claims to be effective for a host of diverse disorders, then this is a red flag ▶#6.

4. Is there evidence from controlled trials that the intervention is effective?
If there is such evidence, the main website for the intervention should describe it and provide links to the sources. No mention of controlled trials ▶#7, and heavy reliance on testimonials ▶#8 are both red flags. Chldren's progress should be measured on standardized and reliable psychometric tests, i.e. measures that have been developed for this purpose where normal range performance has been established. Failure to provide such information is another red flag ▶#9. It is not uncommon to find reference to trials with no controls, i.e. children’s progress is monitored before and after the intervention, and improvements are described. This is not adequate evidence of efficacy, for reasons I have covered in detail here: essentially, improvement in test scores can arise because of practice on the tests, maturation, statistical variation or expectation effects. If scores from before and after treatment are presented as evidence for efficacy, with no reference to control data, this is another red flag ▶#10, because it indicates that the practitioners do not understand the basic requirements of treatment evaluation.
If the evidence comes solely from children tested by people with a commercial interest, there may even be malpractice, with scores massaged to look better than they are. When there were complaints about an US intervention, Learning RX, ex-employees claimed that they had been encouraged to alter children's test scores to make their progress look better than it was (see comment from 6th Dec 2009). One hopes this is not common, but it is important to be alert to the possibility and to ensure those administering psychological tests are appropriately qualified, and if necessary get an independent assessment.
The strongest evidence for effectiveness comes from randomised controlled trials, which adopt stringent methods that have become the norm in clinical medicine. Where several trials have been conducted, then it is possible to combine the findings in a systematic review, which uses rigorous standards to evaluate evidence to avoid bias that can arise if there is ‘cherrypicking’ of studies. This level of evidence is very rare in behavioural interventions for neurodevelopmental disorders because the studies are expensive and time-consuming to do.

5. What is the attitude of those promoting the intervention to conventional approaches?
The question that an advocate for a new treatment has to answer is, if this is such a good thing, why hasn’t it been picked up by mainstream practitioners?
An answer that implies some kind of conspiracy by the mainstream to suppress a new development is a red flag ▶#11. This kind of argument is widely used by alternative medicine practitioners who maintain that others have vested interests (e.g. payments from pharmaceutical companies). This doesn’t hold water: basically, if a treatment is effective, then it makes no financial sense to reject it, given that people will pay good money for something that works.
Another red flag ▶#12 is if the new intervention is promoted alongside other alternative medicine methods that do not have good supportive evidence. This suggests that the practitioners do not take an evidence-based approach.


6. Are the costs transparent and reasonable?
Lack of information about costs on the website is a red flag ▶#13, especially if you can only get information by phoning (hence allowing the practitioner to adopt a hard sell approach). Is there any provision for a refund if the intervention is ineffective? If someone tells you their treatment has a 90% success rate, then they should be willing to give you your money back if it doesn't work. Another red flag is if you are asked to sign up in advance for a long-term treatment plan ▶#14. For example, in the case of the Dore programme, there were instances of families tied into credit agreements and forced to pay even if they don’t continue with the intervention.  

I’ll illustrate by applying the criteria to Sensory Activation Solutions. This is just one example of neuroscientific interventions on offer on the web. I've singled it out because I was recently asked my opinion after a new SAS Centre opened in Milton Keynes this month.

1. Who is behind the treatment and what are their credentials?

The SAS website states Sensory Activation Solutions (SAS) is the 'brainwave' of Steven Michaëlis and Kaśka Gozdek-Michaëlis and is the culmination of over 30 years of study and work relating to how we learn and how we can be more effective in life. I tried various approaches to search terms but was not able to find any publications by either person on Google Scholar. This is worrying: one would expect 30 years of study to yield some peer-reviewed papers. 

The biography of Steven Michaëlis does not mention any academic qualifications. He has a background in sound processing and computer technologies and has trained as a group counsellor. The website states that: Kaśka Gozdek-Michaëlis is an inter-faith, cross-cultural lecturer, writer, psychotherapist and life-coach with over 25 years experience. She gained a Master Degree in Oriental Studies at the prestigious University of Warsaw, Poland. She is the author of two books in Polish, 'Develop your genius mind' and 'Super-possibilities of your mind'.

Overall, the originators of the treatment are up-front about their background and do not hide behind spurious qualifications. However, neither of them appears to have any training in brain science or neurodevelopmental disorders, and their methods have not been subject to peer review. Two red flags:▶#2 ▶#4 

2. Is there a credible scientific basis to the treatment?

There were no entries in Google Scholar for "Sensory activation solutions", so I read the section on The science behind the SAS programs. This provided a quite complex story, about how playing sounds through headphones "activates the auditory processing centres in the brain... leading to less sensory overload, faster understanding, better verbal expression and improved reading and writing." It is a truism that playing sounds to people will activate auditory centres of the brain: that's what hearing is. The key question is whether the sounds used by SAS do anything special. There are numerous components to the SAS package, including use of vision, touch, taste, smell and proprioception "to reduce sensory overload." Sensory overload is a problem for some children, notably a subset of those with autistic spectrum disorder. But it's not generally viewed as problematic for children with dyslexia. It's also claimed that by presenting different auditory stimuli to the left and right ears, the SAS method can promote right-ear dominance and inter-hemispheric integration. In a video presentation, Michaëlis explains this aspect of the theory further, picking up on some old ideas about cerebral lateralisation, interhemispheric communication and rapid auditory processing. To those who don't know the literature, this will sound convincing, but his account is oversimplistic, and makes leaps from theory to intervention with no evidence. For example, with current methods of imaging it would be possible to test whether SAS stimuli alter children's cerebral lateralisation, but there's no indication of any studies investigating this. Overall, the account of the brain bases of dyslexia is out of line with contemporary neuroscientific research. One red flag: ▶#5

3. Who is the intervention recommended for?

SAS is described as appropriate for attention deficit disorder,  hyper-activity,  dyslexia, dyscalculia, hearing and speech disorders,  stammering,  autism,  Asperger's Syndrome,  Down Syndrome,  global developmental delay, Cerebral Palsy, eating disorders, sleeping disorders. In the video it is also recommended for acquired aphasia. One red flag: ▶#6

4. Is there evidence from controlled trials that the intervention is effective?

The "research" section of the website cites descriptive statistics only, largely based on parent satisfaction indices. There is no evidence that psychometrically sound measures were used to evaluate progress. 

There is a small scientific literature on Auditory Integration training (AIT), which has many features in common with aspects of the SAS package; most  studies focussed on autism, where there is little evidence of efficacy (Sinha et al, 2006). The American Speech-Language-Hearing Association concluded a review of AIT thus: Despite approximately one decade of practice in this country, this method has not met scientific standards for efficacy and safety that would justify its inclusion as a mainstream treatment for these disorders. Four red flags: ▶#7 ▶#8 ▶#9 ▶#10

 5. What is the attitude of those promoting the intervention to conventional approaches?

The 'resources' section of the website contained a wealth of information about other kinds of intervention, both mainstream and alternative. 

 6. Are the costs transparent and reasonable? 

The website was quite complicated to navigate, and I may have missed something, but I could not find any information about costs of treatment, only a phone number. It's not possible therefore to say if costs are reasonable. It seems unlikely that clients would be tied in to long-term contracts, as treatment duration seems quite short, lasting weeks rather than months. One red flag: ▶#13

Overall, you can see that SAS earns nine red flags on my evaluation scale.

I suspect no intervention is perfect, and if you have a child who is struggling at school you may want to go ahead and try an intervention regardless of red flags. My goal here is not to stop people trying new interventions, but to ensure that they do so with their eyes open. If practitioners make claims about changing the brain, then they can expect to have those claims scrutinised by neuroscientists. The list of red flags is intended to help people make informed decisions: it may also serve the purpose of indicating to practitioners what they need to do to win confidence of the scientific community.  

Reference
Sinha, Y., Silove, N., Wheeler, D., & Williams, K. (2006). Auditory integration training and other sound therapies for autism spectrum disorders: a systematic review Archives of Disease in Childhood, 91 (12), 1018-1022 DOI: 10.1136/adc.2006.094649

P.S. 6th March 2013: Here are some additional tips for spotting bad science more generally: