The weird world of US ethics regulation

There has been a lot of interest over the past week in the Burzynski Clinic, a US organisation that offers unorthodox treatment to those with cancer. To get up to speed on the backstory see this blogpost by Josephine Jones.

As someone who spends more of my time than I’d like grappling with research ethics committees, there was one aspect of this story that surprised me. According to this blogpost, the clinic is not allowed to offer medical treatment, but is allowed to recruit patients to take part in clinical trials. But this is expensive for participants. The Observer piece that started all the uproar this week described how a family needed to raise £200,000 so that their very sick little girl could undergo Burzynski’s treatment.

I had assumed that this trial hadn’t undergone ethical scrutiny, because I could not see how any committee could agree that it was ethical to charge someone enormous sums of money to take part in a research project in which there was no guarantee of benefit. I suspect that many people would pay up if they felt they’d exhausted all other options. But this doesn’t mean it’s right.

I was surprised, then, to discover that the Burzynski trial had undergone review by an Institutional Review Board (IRB - the US term for an ethics committee). A letter describing the FDA’s review of the relevant IRB is available on the web. It concludes that “the IRB did not adhere to the applicable statutory requirements and FDA regulations governing the protection of human subjects.”  There’s a detailed exposition of the failings of the Burzynski Institute IRB, but no mention of fees charged to patients. So I followed a few more links and came to a US government site that described regulatory guidelines for ethics committees, which had a specific section on Charging for Investigational Products. It seems the practice of passing on research costs to research participants is allowed in the US system.

There has been considerable debate in academic circles about the opposite situation, where participants are paid to take part in a study. I know of cases where such payments have been prohibited by an ethics committee on the grounds that they provide ‘inducement’, which is generally regarded as a Bad Thing, though there are convincing counterarguments. But I am having difficulty in tracking down any literature at all on the ethics of requiring participants to pay a fee to take part in research. Presumably this is a much rarer circumstance than cases where participants are paid, because in general people need persuading to take part in research. The only people who are likely to pay large sums to be a research participant are those who are in a vulnerable state, feeling they have nothing to lose. But these are the very people who need protection by ethics committees because it’s all too easy for unscrupulous operators to exploit their desperation. Anyone who doesn’t have approval to charge for a medical treatment could just redescribe their activities as a clinical trial and bypass regulatory controls. Surely this cannot be right.

Your Twitter Profile: The Importance of Not Being Earnest

I’m always fascinated by the profiles of people who follow me on Twitter. One of the things I love about Twitter is its ability to link me up with people who I’d never otherwise encounter. It’s great when I find someone from the other side of the world who’s interested in the same things as me. There are, of course, also those who just want to promote their product, and others, like Faringdon Motor Parts and Moaning Myrtle (@toiletmoans) whose interests in my tweets are, frankly, puzzling. But the ones that intrigue me most are the ones with profiles that create an immediate negative impression - or to put it more bluntly, make me just think "Pillock!" (If you need to look that up, you’re not from Essex).

Now language is one of my things - I work on language disorders, and over the years I’ve learned a bit about sociolinguistics - the influence of culture on language use. And that made me realise there were at least two hypotheses that could explain the occasional occurrence of offputting profiles. The first was that I am being followed by genuine pillocks. But the other was that there are cultural differences in what is regarded as an acceptable way of presenting yourself to the world. Maybe a turn of phrase that makes me think "pillock" would make someone else think "cool". And perhaps this is culturally determined.

So what, to my British ear, sets off the pillock detector? The major factor was self-aggrandisement. For instance, someone who describes themselves as "a top intellectual", "highly successful", "award-winning", or "inspirational".

But could this just be a US/UK difference? The British have a total horror of appearing boastful: the basic attitude is that if you are clever/witty/beautiful you should not need to tell people - it should be obvious. Someone who tells you how great they are is transgressing cultural norms. Either they really are great, in which case they are up themselves, as we say in Ilford, or they aren’t, in which case they are a dickhead. When I see a profile that says that someone is "interested in everything, knows nothing", "a lazy pedant", or "procrastinaor extraordinaire", I think of them as a decent sort, and I can be pretty sure they are a Brit. But can this go too far? Many Brits are so anxious to avoid being seen as immodest that they present themselves with a degree of self-deprecation that can be confused by outsiders with false modesty at best, or neurotic depression at worst.

A secondary factor that sets off my negative reactions is syrupy sentiment, as evidenced in phrases such as: "empowering others", "Living my dream", or "I want to share my love". This kind of thing is generally disliked by Brits. I suspect there are two reasons for this. First, in the UK, displays of emotion are usually muted, except in major life-threatening circumstances: so much so that when someone is unabashedly emotional they are treated with suspicion and thought to be insincere. And second, Polyannaish enthusiasm is just uncool. The appropriate take on life’s existential problems is an ironic one.

I was pleased to find my informal impressions backed by by social anthropologist Kate Fox, in her informative and witty book "Watching the English" (Hodder & Stoughton, 2004). Humour, she states, is our "default mode", and most English conversations will involve "banter, teasing, irony, understatement, humorous self-deprecation, mockery or just silliness." (p 61). She goes on to describe the Importance of Not Being Earnest rule: "Seriousness is acceptable, solemnity is prohibited. Sincerity is allowed, earnestness is strictly forbidden. Pomposity and self-importance are outlawed." (p. 62). Fox doesn’t explicitly analyse American discourse in the book, but it is revealing that she states: "the kind of hand-on-heart, gushing earnestness and pompous Bible-thumping solemnity favoured by almost all American politicians would never win a single vote in this country - we watch these speeches on our news programmes with a kind of smugly detached amusement." (p 62).

Anthropologists and linguists have analysed trends such as these in spoken discourse, but I wondered whether they could be revealed in the attenuated context of a Twitter profile. So in an idle moment (well, actually when I was supposed to be doing something else I didn’t want to do) I thought I’d try an informal analysis of my Twitter followers to see if these impressions would be borne out by the data. This is easier said than done, as I could find no simple way to download a list of followers, and so I had to be crafty about using "SaveAs" and "Search and Replace" to actually get a list I could paste into Excel, and when I did that, my triumph was short-lived: I found it’d not saved Location information. At this point, my enthusiasm for the project started to wane - and the task I was supposed to be doing was looking ever more attractive. But, having started, I decided to press on and manually enter location for the first 500 followers. (Fortunately I was able to listen to an episode of the News Quiz while doing this. I started to like all those eggs with no Location recorded). I then hid that column so it would not bias me, and coded the profiles for three features: (a) Gender (male/female/corporate/impossible to tell); (b) Self-promotion: my totally subjective rating of whether the profile triggered the pillock-detector; (c) Syrupy: another subjective judgement of whether the profile contained overly sentimental language. I had intended also to code mentions of cats - I was convinced that there was a British tendency to mention cats in one’s profile, but there were far too few to make analysis feasible. I was a victim of confirmation bias. So were my other intuitions correct? Well, yes and no.

For the analysis I just focused on followers from the US and UK. The first thing to emerge from the analysis was that pillocks were rare in both US and UK - rarer than I would have anticipated. I realised that, like mentions of cats, it’s something I had overestimated, probably because it provoked a reaction in me when it occurred. But, I was pleased to see that nonetheless my instincts were correct: there were 7/97 (7.2%) pillocks in the US sample but only 2/153 (1.3%) in the UK . The sample size is really not adequate, and if I were going to seriously devote myself to sociolinguistics I’d plough on to get a much bigger sample size. But nevertheless, for what it’s worth, this is a statistically significant difference (chi square = 5.97, p = .015 if you really want to know). Syrup followed a similar pattern: again it was rare in both samples, but it was coded for 3/153 of the UK sample compared with 7/97 of the US. I’d coded gender as I had thought this might be a confounding factor, but in fact there were no differences between males and females in either pillocks or syrup. Of course, all these conclusions apply only to my followers, who are bound to be an idiosyncratic subset of people.

My conclusion from all this: we need to be more sensitive to cultural differences in self-expression. Looking over some of the profiles that I categorised as "pillock" I realise that I’m being grossly unfair to their owners.  After all, on a Twitter profile, the only information that people have about you comes from the profile - and your tweets. So it really is preposterous for me to react negatively against someone telling me they are an "award-winning author": that should engender my interest and respect. And, because this is a profile, and not a conversation, if they didn’t tell me, I wouldn’t know. And we really ought to cherish rather than mock those who try to bring a bit of love and kindness into the world. But somehow….

I hope that Americans reading this will get some insight into the tortuous mindset of the Brits: if we come across as dysfunctionally insecure losers it’s not that we really are - it’s that we’d rather you thought that of us than that we were boastful.

Vitamins, genes and language

Thiamine chloride  (source: Wikipedia)

In November 2003, a six-month-old boy was admitted to the emergency department of  a children’s hospital in Tel Aviv. He had been vomiting daily for two months, was apathetic, and had not responsed to anti-emetic drugs. The examining doctor noticed something odd about the child’s eye movements and referred him on to the neuro-ophthalmology department. A brain scan failed to detect any tumour. The doctors remembered a case they had seen 18 months earlier, where a 16-year-old girl had presented with episodic vomiting and abnormal eye movements due to vitamin B1 deficiency.  They injected the child with thiamine and saw improvement after 36 hours. The vomiting stopped, and over the next six weeks the eye movements gradually normalised. When followed up 18 months later he was judged to be completely normal.

This was not, however, an isolated case. Other babies in Israel were turning up in emergency departments with similar symptoms. Where thiamine deficiency was promptly recognised and treated, outcomes were generally good, but two children died and others were left with seizures and neurological impairment. But why were they thiamine deficient? All were being fed the same kosher, non-dairy infant formula, but it contained thiamine. Or did it? Analysis of samples by the Israeli Ministry of Health revealed that levels of thiamine in this product were barely detectable, and there was an immediate product recall. The manufacturer confirmed that human error had led to thiamine being omitted when the formula had been altered.

The cases who had been hospitalised were just the tip of the iceberg. Up to 1000 infants had been fed the formula. Most of these children had shown no signs of neurological problems. But a recent study reported in Brain describes a remarkable link between this early thiamine deprivation and later language development. Fattal and colleagues studied 59 children who had been fed thiamine-deficient formula for at least one month before the age of  13 months, but who were regarded as neurologically asymptomatic. Children who had birth complications or hearing loss were excluded. The authors stress that the children were selected purely on the basis of their exposure to the deficient formula, and not according to their language abilities. All were attending regular schools.  A control group of 35 children was selected from the same health centres, matched on age.

Children were given a range of language tests when they were 5 to 7 years of age. These included measures of sentence comprehension, sentence production, sentence repetition and naming. There were dramatic differences between the two groups of children, with the thiamine-deficient group showing deficits in all these tasks. The authors argued that the profile of performance was identical to that seem in children with a diagnosis of specific language impairment (SLI), with specific problems with certain complex grammatical constructions, and normal performance on a test of conceptual understanding that did not involve any language.

Figure 1 An example of a picture pair used in the comprehension task. 

The child is asked to point to the picture that matches a sentence, 

such as ‘Tar’e li et ha-yalda she-ha-isha mecayeret’ 

(Show me the girl that the woman draws). From Fattal et al, 2011.

I have some methodological quibbles with the paper. The authors excluded three control children who did poorly on the syntactic tests because they were outliers - this seems wrong-headed if the aim is to see whether syntactic problems are more common in children with thiamine-deficiency than in those without. The non-language conceptual tests were too easy, with both groups scoring above 95% correct. To convince me that the children had normal abilities they would need to demonstrate no difference between groups on a sensitive test of nonverbal IQ. My own experience of testing children’s grammatical abilities in English is that ability to do tests such as that shown in Figure 1 can be influenced by attention and memory as well as syntactic ability, and so I think we need to rule out other explanations before accepting the linguistic account offered by the authors. I’d also have liked a bit more information about how the control children were recruited, to be certain they were not a ‘supernormal’ group - often a problem with volunteer samples, and something that could have been addressed if a standarized IQ test had been used. But overall, the effects demonstrated by these authors are important, given that there are so few environmental factors known to selectively affect language skills. These results raise a number of questions about children’s language impairments.

The first question that struck me was whether thiamine deficiency might be implicated in other cases outside this rare instance. I have no expertise in this area, but this paper prompted me to seek out other reports. I learned that thiamine deficiency, also known as infantile beriberi, is extremely rare in the developed world, and when it does occur it is usually because an infant is breastfeeding from a mother who is thiamine deficient. It is therefore important to stress that thiamine deficiency is highly unlikely to be implicated in cases of specific language impairment in Western societies. However, a recent paper reported that it is relatively common in Vientiane, Laos, where there are traditional taboos against eating certain foods in the period after giving birth. The researchers suggested that obvious cases with neurological impairments may be the extreme manifestation of a phenomenon that is widespread in milder form. If so, then the Israeli paper suggests that the problem may be even more serious than originally suggested, because there could be longer-term adverse effects on language development in those who are symptom-free in infancy.

The second question concerns the variation in outcomes of thiamine-deficient infants. Why, when several hundred children had been fed the deficient formula, were only some of them severely affected? An obvious possibility is the extent to which infants were fed foods other than the deficient formula. But there may also be genetic differences between children in how efficiently they process thiamine.

This brings us to the third question: could this observed link between thiamine deficiency and language impairment have relevance for genetic studies of language difficulties? Twin and family studies have indicated that specific language impairment is strongly influenced by genes. However, one seldom finds genes that have a major all-or-none effect. Rather, there are genetic risk variants that have a fairly modest and probabilistic impact on language ability.

Robinson Crusoe Island

A recent study by Villanueva et al illustrates this point. They analysed genetic variation in an isolated population on Robinson Crusoe Island, the only inhabited island in the Juan Fernandez Archipelago, 677 km to the west of Chile. At the time of the study there were 633 inhabitants, most of whom were descended from a small number of founder indviduals. This population is of particular interest to geneticists as there is an unusually high rate of specific language impairment.  A genome-wide analysis failed to identify any single major gene that distinguished affected from unaffected individuals. However, there was a small region of chromosome 7 where there genetic structure was statistically different between affected and unaffected cases, and which contained genetic variants that had previously been found linked to language impairments in other samples. One of these, TPK1 is involved in the catalysis of the conversion of thiamine to thiamine pyrophosphate. It must be stressed that the genetic association between a thiamine-related genetic variant and  language impairment is probabilistic and weak, and far more research will be needed to establish whether it is generalises beyond the rare population studied by Villanueva and colleagues. But this observation points the way to a potential mechanism by which a genetic variant could influence language development.

To sum up: the importance of the study by Fattal and colleagues is two-fold. First, it emphasises the extent to which there can be adverse longer-term consequences of thiamine deficiency in children who may not have obvious symptoms, an observation which may assume importance in cultures where there is inadequate nutrition in breast-feeding mothers. Second, it highlights a role of thiamine in early neurodevelopment, which may prove an important clue to neuroscientists and geneticists investigating risks for language impairment.

References

Fattal I, Friedmann N, & Fattal-Valevski A (2011). The crucial role of thiamine in the development of syntax and lexical retrieval: a study of infantile thiamine deficiency. Brain : a journal of neurology, 134 (Pt 6), 1720-39 PMID: 21558277  

Villanueva P, Newbury DF, Jara L, De Barbieri Z, Mirza G, Palomino HM, Fernández MA, Cazier JB, Monaco AP, & Palomino H (2011). Genome-wide analysis of genetic susceptibility to language impairment in an isolated Chilean population. European journal of human genetics : EJHG, 19 (6), 687-95 PMID: 21248734

A message to the world

from a teenager with language difficulties

Accentuate the negative

Suppose you run a study to compare two groups of children: say a dyslexic group and a control group. Your favourite theory predicts a difference in auditory perception, but you find no difference between the groups. What to do? You may feel a further study is needed: perhaps there were floor or ceiling effects that masked true differences. Maybe you need more participants to detect a small effect. But what if you can’t find flaws in the study and decide to publish the result? You’re likely to hit problems. Quite simply, null results are much harder to publish than positive findings. In effect, you are telling the world “Here’s an interesting theory that could explain dyslexia, but it’s wrong.” It’s not exactly an inspirational message, unless the theory is so prominent and well-accepted that the null finding is surprising. And if that is the case, then it’s unlikely that your single study is going to be convincing enough to topple the status quo. It has been recognised for years that this “file drawer problem” leads to distortion of the research literature, creating an impression that positive results are far more robust than they really are (Rosenthal, 1979).

The medical profession has become aware of the issue and it’s now becoming common practice for clinical trials to be registered before a study commences, and for journals to undertake to publish the results of methodologically strong studies regardless of outcome. In the past couple of years, two early-intervention studies with null results have been published, on autism (Green et al, 2010) and late talkers (Wake et al, 2011). Neither study creates a feel-good sensation: it’s disappointing that so much effort and good intentions failed to make a difference. But it’s important to know that, to avoid raising false hopes and wasting scarce resources on things that aren’t effective. Yet it’s unlikely that either study would have found space in a high-impact journal in the days before trial registration.

Registration can also exert an important influence in cases where conflict of interest or other factors make researchers reluctant to publish null results. For instance, in 2007, Cylharova et al published a study relating membrane fatty acid levels to dyslexia in adults. This research group has a particular interest in fatty acids and neurodevelopmental disabilities, and the senior author has written a book on this topic. The researchers argued that the balance of omega 3 and omega 6 fatty acids differed between dyslexics and non-dyslexics, and concluded: “To gain a more precise understanding of the effects of omega-3 HUFA treatment, the results of this study need to be confirmed by blood biochemical analysis before and after supplementation”. They further stated that a randomised controlled trial was underway. Yet four years later, no results have been published and requests for information about the findings are met with silence. If the trial had been registered, the authors would have been required to report the results, or explain why they could not do so.

Advance registration of research is not a feasible option for most areas of psychology, so what steps can we take to reduce publication bias? Many years ago a wise journal editor told me that publication decisions should be based on evaluation of just the Introduction and Methods sections of a paper: if an interesting hypothesis had been identified, and the methods were appropriate to test it, then the paper should be published, regardless of the results.

People often respond to this idea saying that it would just mean the literature would be full of boring stuff. But remember, I'm not suggesting that any old rubbish should get published: there has to be a good case for doing the study made in the Introduction, and the Methods have to be strong. Also, some kinds of boring results are important: miminally, publication of a null result may save some hapless graduate student from spending three years trying to demonstrate an effect that’s not there. Estimates of effect sizes in meta-analyses are compromised if only positive findings get reported. More seriously, if we are talking about research with clinical implications, then over-estimation of effects can lead to inappropriate interventions being adopted.

Things are slowly changing and it’s getting easier to publish null results. The advent of electronic journals has made a big difference because there is no longer such pressure on page space. The electronic journal PLOS One adopts a publication policy that is pretty close to that proposed by the wise editor: they state they will publish all papers that are technically sound. So my advice to those of you who have null data from well-designed experiments languishing in that file drawer: get your findings out there in the public domain.

References

Cyhlarova, E., Bell, J., Dick, J., MacKinlay, E., Stein, J., & Richardson, A. (2007). Membrane fatty acids, reading and spelling in dyslexic and non-dyslexic adults European Neuropsychopharmacology, 17 (2), 116-121 DOI: 10.1016/j.euroneuro.2006.07.003

Green, J., Charman, T., McConachie, H., Aldred, C., Slonims, V., Howlin, P., Le Couteur, A., Leadbitter, K., Hudry, K., Byford, S., Barrett, B., Temple, K., Macdonald, W., & Pickles, A. (2010). Parent-mediated communication-focused treatment in children with autism (PACT): a randomised controlled trial The Lancet, 375 (9732), 2152-2160 DOI: 10.1016/S0140-6736(10)60587-9 

Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86 (3), 638-641 DOI: 10.1037/0033-2909.86.3.638 

Wake M, Tobin S, Girolametto L, Ukoumunne OC, Gold L, Levickis P, Sheehan J, Goldfeld S, & Reilly S (2011). Outcomes of population based language promotion for slow to talk toddlers at ages 2 and 3 years: Let's Learn Language cluster randomised controlled trial. BMJ (Clinical research ed.), 343 PMID: 21852344

G

Lies, damned lies, and spin

©www.cartoonstock.com

The Department for Education (DfE) issued a press report this week entitled “England's 15-year-olds' reading is more than a year behind the best”. The conclusions were taken from analysis of data from the PISA 2009 study, an OECD survey of 15-year-olds in the principal industrialised countries.

The DfE report paints a dire picture: “GCSE pupils' reading is more than a year behind the standard of their peers in Shanghai, Korea and Finland….Fifteen-year-olds in England are also at least six months behind those in Hong Kong, Singapore, Canada, New Zealand, Japan and Australia, according to the Department for Education's (DfE) analysis of the OECD's 2009 Programme for International Student Assessment (PISA) study.” The report goes on to talk of England slipping behind other nations in reading.

Schools Minister Nick Gibb is quoted as saying: “The gulf between our 15-year-olds' reading abilities and those from other countries is stark – a gap that starts to open in the very first few years of a child's education.”

I started to smell a rat when I looked at a chart in the report, entitled “Attainment gap between England and the countries performing significantly better than England” (my emphasis). This seemed an odd kind of chart to provide if one wanted to evaluate how England is doing compared to other countries. So I turned to the report provided by the people who did the survey.

Here are some salient points taken verbatim from their summary on reading:

• Twelve countries had mean scores for reading which were significantly higher than that of England. In 14 countries the difference in mean scores from that in England was not statistically significant. Thirty-eight countries had mean scores that were significantly lower than England.
• The mean score for reading in England was slightly above the OECD average but this difference was not statistically significant.
• England’s performance in 2009 does not differ greatly from that in the last PISA survey in 2006.

There is, of course, no problem with aiming high and wanting our children to be among the top achievers in the world. But that’s no excuse for the DfE's mendacious manipulation of information.

Reference

Bradshaw, J., Ager, R., Burge, B. and Wheater, R. (2010). PISA 2009: Achievement of 15-Year-Olds in England. Slough: NFER.

The joys of inventing data

Have I gone over to the dark side? Cracked under pressure from the REF to resort to fabrication of results to secure that elusive Nature paper? Or had my brain addled by so many requests for information from ethics committees that I’ve just decided that its easier to be unethical? Well readers will be reassured to hear that none of these things is true. What I have to say concerns the benefits of made-up data for helping understand how to analyse real data.

In my field of experimental psychology, students get a thorough grounding in statistics and learn how to apply various methods for testing whether groups differ from one another, whether variables are associated and so on. But what they typically don’t get is any instruction in how to simulate datasets. This may be a historical hangover. When I first started out in the field, people didn’t have their own computers, and if you wanted to do an analysis you either laboriously assembled a set of instructions in Fortran which were punched onto cards and run on a mainframe computer (overnight if you were lucky), or you did the sums on a pocket calculator. Data simulation was just unfeasible for most people. Over the years, the landscape has changed beyond recognition and there are now windows-based applications that allow one to do complex multivariate statistics at the press of a button. There is a danger, however, which is that people do analyses without understanding them. And one of the biggest problems of all is a tendency to apply statistical analyses post hoc. You can tell people over and over that this is a Bad Thing (see Gould and Hardin, 2003) but they just don’t get it. A little simulation exercise can be worth a thousand words.

So here’s an illustration. Suppose we’ve got two groups each of 10 people, let’s say left-handers and right-handers. And we’ve given them a battery of 20 cognitive tests. When we scrutinise the results, we find that they don’t differ on most of the measures, but there’s a test of mathematical skill on which the left-handers outperform the right-handers. We do a t-test and are delighted to find that on this measure, the difference between groups is significant at the .05 level, so we write up a paper entitled "Left-handed advantage for mathematical skills" and submit it to a learned journal, not mentioning the other 19 tests. After all, they weren’t very interesting. Sounds OK? Well, it isn’t. We have fallen into the trap of using statistical methods that are valid for testing a hypothesis that is specified a priori in a situation where the hypothesis only emerged after scrutinising the data.

Let’s generate some data. Most people have access to Microsoft Excel, which is perfect for simple simulations. In row 1 we put our column labels, which are group, var1, var2, …. var 20.

In column A, we then have ten zeroes followed by ten ones, indicating group identity. We then use random numbers to complete the table. The simplest way to do this is to just type in each cell:

   =RAND()

This generates a random number between 0 and 1.

A more sophisticated option is to generate a random z-score. This creates random numbers that meet the assumption of many statistical tests that data are normally distributed. You do this by typing:

   =NORMSINV(RAND())

At the foot of each column you can compute the mean and standard deviation for each group, and Excel automatically computes a p-value based on the t-test for comparing the groups with a command such as:

=TTEST(B2:B11,B12:B22,2,2)

See this site if you need an explanation of this formula.

So the formulae in the first three columns look like this (rows 4-20 are hidden): 

Copy this formula across all columns. I added conditional formatting to row 27 so that ‘significant’ p-values are highlighted in yellow (and it just so happens with this example that the generated data gave a p-value less than .05 for column C).

Every time you type anything at all on the sheet, all the random numbers are updated: I’ve just added a row called ‘thisrun’ and typing any number in cell B29 will re-run the simulation.  This provides a simple way of generating a series of simulations and seeing when p-values fall below .05. On some runs, all the t-tests are nonsignificant, but you’ll quickly see that on many runs one or more p-values are below .05. In fact, on average, across numerous runs, the average number of significant values is going to be one because we have twenty columns, and 1/20 = .05. That’s what p < .05 means! If this doesn’t convince you of the importance of specifying your hypothesis in advance, rather than selecting data for analysis post hoc, nothing will.

This is a very simple example, but you can extend the approach to much more complicated analytic methods. It gets challenging in Excel if you want to generate correlated variables, though if you type a correlation coefficient in cell A1, and have a random number in column B, and copy this formula down from cell C2, then columns B and C will be correlated by the value in cell A1:

=B2*A$1+NORMSINV(RAND())*SQRT(1-A$1^2)

NB, you won’t get the exact correlation on each run: the precision will increase with the number of rows you simulate.

Other applications, such as Matlab or R, allow you to generate correlated data more easily. There are examples of simulating multivariate normal datasets in R in my blog on twin methods.

Simulation can be used not just for exploring a whole host of issues around statistical methods. For instance, you can simulate data to see how sample size affects results, or how results change if you fail to meet assumptions of a method. But overall, my message is that data simulation is a simple and informative approach to gaining understanding of statistical analysis. It should be used much more widely in training students.

Reference

Good, P. I., & Hardin, J. W. (2003). Common errors in statistics (and how to avoid them). Hoboken, NJ: Wiley.

How to become a celebrity scientific expert

Maybe you’re tired of grotting away at the lab bench. Or finding it hard to get a tenured job. Perhaps your last paper was rejected and you haven’t the spirit to fight back. Do not despair. There is an alternative. The media are always on the look-out for a scientist who will fearlessly speak out and generate newsworthy stories. You can gain kudos as an expert, even if if you haven't got much of a track record in the subject, by following a few simple rules.

Rule #1. Establish your credentials. You need to have lots of letters after your name. It doesn’t really matter what they mean, so long as they sound impressive. It’s also good to be a fellow of some kind of Royal Society. Some of these are rather snooty and appoint fellows by an exclusive election process, but it’s a little known fact that others require little more than a minimal indication of academic standing and will admit you to the fellowship provided you fill in a form and agree to pay an annual subscription. So sign up as a Fellow of the Royal Society of Medicine, and keep good company with a range of naturopaths, homeopaths and chiropracters who have discovered this easy route to eminence. The really nice thing is that even academics can be hoodwinked by this one.

Rule #2. Find a controversial topic. This is key. You have to be willing to take a definite position on something that people have strong views about. A scare story is good - we’ve all been doing X for years but it could damage us. Finding someone to blame is also good - people who do Y are feckless. And the buzz word of the decade is neuroscience, so if you can work that in, success is guaranteed. If you're short of ideas, the list of the right might help inspire you. A recent article in the Biologist hits the spot with “The biological effects of day care”, managing to get us worried about an everyday activity, blame working mothers, and get in a neuro message all at once. It's even spiced up with a bit of conspiracy theory: experts know that day care is bad for children’s brains but nobody is allowed to speak out because it is too politically sensitive. This presses so many buttons that few journalists could resist the story.

Rule #3. Specify a causal chain. As we shall see when it comes to assembling evidence, it is particularly useful to have a causal chain with several steps. For instance:

The point here is that if you can usually find at least some studies that provide evidence for bits of the causal chain. Although it may be inconvenient if, as in this case, studies looking for a link between A and D fail to come up with clear evidence (Lucas-Thompson et al, 2010), you can rely on two things: first, few readers will be familiar with the research literature, so they will only know as much as you tell them. And second, step C, brain abnormality, is highly salient and once you start talking about that, it will distract attention from the other levels of description.

Rule #4. Avoid rigorous peer review. You don’t want to have your views critiqued by someone who knows the literature, or checks your sources. Writing books is a safe bet for avoiding pre-publication scientific critique. As far as journals go, the Biologist is ideal. This publication for the members of the Society of Biology claims to be peer-reviewed, but, as we shall see, the review process is far from rigorous.

Rule #5. Assemble supportive evidence. Note, it is important not to present all relevant studies, just those with findings that can be fitted into the causal chain.

The author of the paper in question, Dr Aric Sigman, presents us with so much positive evidence that he manages to give the impression that the whole casual chain has been validated. He starts by mentioning studies that investigated the link between A and B and find that salivary cortisol is increased in the afternoons in children attending day care. This is a product of the hypothalamo-pituitary-adrenal (HPA) axis, which is elevated in response to stress. This result is well-established, both from studies comparing groups of children who do and don’t attend day care, and from comparing the same children on days when they stay at home or go to day care. This is a potentially concerning finding, if it can be shown that there are consequences for children’s learning and behaviour. As Sigman notes “Of central concern is that the routine stress experienced at day care could cause permanent changes in the child’s neuroendocrine networks, with long-term consequences for their mental and physical health as adults.” (p. 30). The article that he cites does discuss this issue, but notes the complexity of causal relationships and cautions against assuming that the cortisol elevation is harmful. In fact the authors draw attention to an animal study suggesting a very different conclusion:

“As in the work on cortisol responses to fullday child care, these separations in squirrel monkey infants produced marked and repeated activations of the HPA axis. However, followed into the late juvenile and early adult age, animals exposed to this form of early life stress were found to be less fearful, to produce lower rather than higher cortisol responses to stressors, and to show more optimal development of prefrontal regulatory brain circuits; consistent with these findings, they also performed better on tests of executive functioning. Thus, at least for this animal model, repeated separation stress early in life fostered a form of resilience.” Gunnar et al (2010) (my emphasis)

Sigman avoids mentioning any of this and turns instead to look at the links between cortisol levels and ill health, starting with studies that show a link between cortisol and cardiovascular disease. He does not explain that these were done on people aged over 65 years, but rather implants in people’s minds the notion that this is relevant for his arguments about daycare in toddlers.  Next comes the serious stuff: research linking elevated cortisol to the brain. We are told that “Cortisol is considered neurotoxic and has a global impact on cerebral size (e.g. McEwen 2007; Sheline 2003).” (p. 29). Again, we are left with the distinct impression that children who attend day care will have small brains, but to find out what actually is meant here we need to read the cited articles. When we do, we find that McEwen (2007) is a thorough research review of the physiology and neurobiology of stress and adaptation that nowhere mentions the terms ‘neurotoxic’, ‘global’ or ‘cerebral size’. Rather, in this article McEwen develops a complex theory that considers both positive and negative impact of stress. It actually has a section subtitled “Protection and damage: the two sides of the response to stressors” which discusses animal studies demonstrating how elevated cortisol can either improve or interfere with brain function, depending on context. Neurotoxicity does feature in the other article, by Sheline (2003), but this is concerned with mood disorders in adults, and discusses effects of hypercortisolemia, a condition where there is chronic elevation of cortisol, rather than a temporary increase at specific times of day. Subsequent citations are to papers that considered the role of cortisol in psychiatric disorders such as anxiety and depression: note that now the evidence is focused on a link between cortisol and adult psychiatric disorders in the opposite direction (from D to B), yet it is presented in the context of discussing consequences of high cortisol in children who attended day care. Sigman further states: “a higher cortisol awakening curve may be a biological marker for an underlying disposition towards developing depressive and anxiety disorders” (p. 30), even though the studies of toddlers attending day care show a cortisol response that develops through the day, rather than a chronically raised level: see Vermeer and van IJzendoorn (2006) for a well-balanced discussion of such evidence.

It would be tedious to wade through every cited article, but it’s worth considering just one more example. We are told “In human grey matter, the quality of a mother’s care in early childhood is thought to alter the size of the hippocampus (Buss et al, 2007)” (p. 30). Erm no. Buss et al clearly stated there were no differences in left or right hippocampal volume between those categorised as having high or low maternal care. What they did find was a complicated interaction between birth weight, gender and maternal care, such that birth weight predicted hippocampal volume only in female subjects reporting low maternal care. But even this limited result doesn’t support Sigman’s case. We are talking about ‘low maternal care’, not ‘attendance at daycare’. And guess what? When we look at ‘low maternal care’ we find it measured from a self-report questionnaire, where low care is partly identified in terms of maternal overprotection. Those categorised this way were more likely to have endorsed items describing their mother in such terms as:

• Tended to baby me
• Tried to make me feel dependent on her
• Felt I could not look after myself unless she was around
• Was overprotective of me

I’m not an expert in the neurobiology of stress. I can track down articles that appear to have been cited by Sigman (the reference list is behind a paywall) and see where he's given a misleading account, but what I don’t know is how much relevant literature has been omitted. This, of course, is what the celebrity scientist can rely on: there's only a handful of people who both have the expertise in the area, and are obsessive enough to trawl through your writing (if they can access it) and challenge any misleading statements.

Rule #6. Anticipate criticism but don't let it worry you. People who actually do research in the area you are reviewing may get irritated, but most scientists in the field wouldn’t bother on the grounds that they don't know who you are, and aren’t interested in pursuing academic debates outside the domain of mainstream journals. The worst you may get is a few nerdy bloggers such as Gimpy or Mind Hacks criticising you for lack of scholarship, sensationalism and cherrypicking of evidence. Or, if you're really unlucky, you might be up against Ben Goldacre on Newsnight. But meanwhile, your purpose as celebrity scientist has been achieved: your views are all over the media.

References

Buss, C., Lord, C., Wadiwalla, M., Hellhammer, D. H., Lupien, S. J., Meaney, M. J., et al. (2007). Maternal care modulates the relationship between prenatal risk and hippocampal volume in women but not in men. Journal of Neuroscience, 27(10), 2592-2595. dx.doi.org/10.1523/JNEUROSCI.3252-06.2007

Gunnar MR, Kryzer E, Van Ryzin MJ, & Phillips DA (2010). The rise in cortisol in family day care: associations with aspects of care quality, child behavior, and child sex. Child Development, 81 (3), 851-69. PMID: 20573109

Lucas-Thompson, R. G., Goldberg, W. A., & Prause, J. A. (2010). Maternal work early in the lives of children and its distal associations with achievement and behavior problems: A meta-analysis. Psychological Bulletin, 136(6), 915-942. DOI: 10.1037/a0020875
McEwen, B. S. (2007). Physiology and neurobiology of stress and adaptation: Central role of the brain. Physiological Reviews, 87(3), 873-904.doi: 10.​1152/​physrev.​00041.​2006 (Open Access)

Sigman, A. (2011). Mother superior? The biological effects of day care. The Biologist, 5 (3), 29-32.


Vermeer, H. J. & van IJzendoorn, M. H. (2006). Children's elevated cortisol levels at daycare: A review and meta-analysis. Early Childhood Research Quarterly, 21(3), 390-401.doi 10.1016/j.ecresq.2006.07.004

Early intervention: What's not to like?

If a child has language problems, when would be the best age to intervene? At 18 months of age, when they’re just at the outset of learning language, or at five years, when they’re in school? Most people would say this is a no-brainer, with early intervention being preferred on two counts:

• There are all kinds of secondary consequences of language difficulties: effects on self-esteem, educational outcomes and social interactions. Potentially, early intervention can avoid these.
• It is easier to influence the course of development while the brain is still plastic. An analogy can be made with vision, where it is well-recognised that amblyopia (or "lazy eye") needs to be corrected early in life, because otherwise visual pathways in the brain do not develop normally, and the potential for good vision in the lazy eye is lost.

Currently, interest by policy-makers in early intervention has focussed mainly on children’s social and emotional outcomes, with a report by MP Graham Allen emphasising the benefits, not just for children’s outcomes, but also in economic terms. The argument is that by preventing problems from developing, we have the potential to save millions of pounds that would otherwise be spent in dealing with problems that manifest later in childhood.

The Allen report does not say much about children’s language development, but similar arguments are often made, and in some areas of the country, speech and language therapy services put most of their resources into intervention with preschoolers.

There is, however, a problem with early intervention that is easily overlooked, but which is well-documented in the case of children’s language problems. This is the phenomenon of the "late bloomer". Quite simply, the earlier you identify children’s language difficulties, the higher the proportion of cases will prove to be "false positives" who spontaneously move into the normal range without any intervention. We’ve known about this phenomenon for many years: For instance, a study conducted by Fischel et al in 1989 followed 26 two-year-olds recruited because their parents reported that they understood complete sentences but could say only a few words. Five months after initial assessment, one third still had problems, one third had made some improvement, and one third were in the normal range. Another study by Thal et al in 1991 followed ten children who scored in the bottom 10% for expressive vocabulary at the age of 18 to 29 months. One year after initial assessment, six had caught up, whereas the remaining four still had delayed language. These early small-scale studies have since been confirmed by much larger population-based studies in the Netherlands and Australia.

The late-bloomer phenomenon was neatly demonstrated in a study just published in the British Medical Journal by an Australian team headed by paediatrician Prof Melissa Wake and speech pathologist Prof Sheena Reilly. They recruited children from a large population-based study, where parents were asked to complete a Sure Start vocabulary screening measure when their child was 18 months of age, as well as a child behaviour checklist. Around 20 per cent of children were reported as having no or very limited spoken words. 301 of these children were randomly allocated to intervention or control groups. The intervention, "Let's Learn Language", was based on a widely-used approach where parents are trained to adopt strategies to enhance communicative interactions with their child. The children were then given a detailed assessment at two years of age, and again at three years. Results were striking: there were no hints of any difference between children in the intervention group and control group on any language or behavioural measures, either at 2 years or at 3 years.

The study authors noted various strengths and weaknesses of their study. Among these they discussed the possibility that the intensity of the intervention (six weekly sessions, each lasting 2 hours) may not have been enough. But they went on to note that “the normal mean language and vocabulary scores achieved by both intervention and control children by age 3 years suggest that natural resolution, rather than our intervention’s intensity being too low, explains the null findings.”

They then point out the sobering conclusion to be drawn: quite simply, if you intervene with children who are likely to improve spontaneously, there will considerable waste of government’s and families’ resources.

Does this mean we should give up on early intervention? No. But it does mean that we need to target such intervention much more carefully. At present, one of the big questions for those of us investigating late talkers is to find characteristics that will allow us to identify those children who won’t make spontaneous progress. This has proved to be surprisingly difficult.

Another important message applies to intervention studies more generally. If you provide an intervention for a condition that spontaneously improves, it is easy to become convinced that you’ve been effective. Parents were very positive about the intervention program. There was remarkably good attendance, and when asked to rate specific features of the program and its effects, around three quarters of the parents gave positive responses. This may explain why both parents and professionals find it hard to believe such interventions have no impact: they do see improvement. Only if you do a properly controlled trial will the lack of effect become apparent, not because treated children don’t improve, but rather because the control group gets better as well.

Reference: (Open Access) :-)
Wake M, Tobin S, Girolametto L, Ukoumunne OC, Gold L, Levickis P, Sheehan J, Goldfeld S, & Reilly S (2011). Outcomes of population based language promotion for slow to talk toddlers at ages 2 and 3 years: Let's Learn Language cluster randomised controlled trial. BMJ (Clinical research ed.), 343 PMID: 21852344

So you want to be a research assistant? Advice for psychologists

©CartoonStock.com

The dire state of the academic jobs market was brought home to me recently. I’d advertised for someone to act as a graduate research assistant/co-ordinator. This kind of post is a good choice for a junior person who wants to gain experience before applying for clinical or educational psychology training, or while considering whether to do a doctorate.  Normally I get around 30-40 applicants for this kind of job. This time it was 123.  This, apparently, is nothing. These days, for psychology assistant jobs, which act as a gateway to oversubscribed clinical psychology doctorate programmes,  the number of applicants can run into the hundreds.

One thing that strikes me is how little insight many applicants have into what happens to their job application. I hope that this post, explaining the process from the employer's perspective, might help aspiring job-seekers improve their chances of getting to interview.

With over 120 applications to process, if I allowed only two minutes for each application, it’d take me four hours to shortlist. Of course, that’s not how it works. There has to be an initial triage procedure where the selection panel views the applications looking for reasons not to shortlist. We were able to exclude around ¾ of the applications on the basis of a fairly brief scan. But we then had to select a shortlist of five from the remainder. This is done on the basis of a careful re-reading of those applications that survive triage.

So how do you get past this double hurdle and avoid initial triage, and then make it to the shortlist? Well, here are some tips. They seem very obvious and simple, but worth stating, as many of the applications we received didn’t seem aware of them.

• Follow the instructions for job applicants, and read the further particulars. I gather that there are some careers advisors who recommend candidates should send their application direct to the principal investigator, rather than via administration, because it will get noticed. It will indeed, but it will create the impression that you are incapable of reading instructions.
• Specify how you meet the selection criteria. Our university bends over backwards to operate a fair and transparent recruitment policy. We need to be able to demonstrate that our decisions are based on the selection criteria in the job advert, and not on some idiosyncratic prejudice. The ideal applicant lists the selection criteria in the same order that they appear in the job description and briefly explains how they meet them. It makes the job of the selection panel much, much easier, and they will give you credit for being both intelligent and considerate.
• Don’t apply if you don’t meet the essential selection criteria. So, if the job requires you to drive, then don’t apply if you don’t have a driving licence (or a chauffeur).  When I was young and naïve, I assumed people wouldn’t apply for a job if they didn’t meet the criteria, and ended up appointing a non-driver to a job that involved travelling to remote locations with heavy equipment. It is not a mistake I’ll make again.
• Don’t assume anything is obvious. To continue with the example above, if the job involves driving and you don’t mention that you can drive, the person evaluating your application won’t know whether you’ve forgotten to tell them, or if you are avoiding mentioning this because you can’t drive. Either way, it’s bad news for your application, and in the current market, it’ll go on the ‘no’ pile.
• Don’t send a standard application that is appropriate for any job. It’s key to include a cover letter or personal statement that indicates that you have read the further particulars for this specific post. Use Google to find out more about the post/employer. On the other hand, the employer really doesn’t want or need to be told about the subject matter of the research - once I had the equivalent of a short undergraduate essay, complete with references, included in an application, and though it demonstrated keeness, it was complete overkill.
• Read through your application before you submit it. I’ve had applicants who describe how enthusiastic they are about the prospect of working, not in my institution, but in another university. I’ve had applications where entire paragraphs were duplicated. A melange of fonts changing mid-paragraph, or even mid-sentence, creates a poor impression.
• Run the cover letter/personal statement through a spell checker, and check the English. Anyone working for me will be sending letters and information sheets out to the general public on my behalf. It creates a bad impression if there are errors, and so you’ve a very high chance of getting on the ‘no’ pile if you make mistakes on an important document like a job application.
• Be honest. If there’s something unusual about your application, explain it. I have, for instance, shortlisted a person who’d had a prolonged period of sick leave, but who gave a clear and honest explanation of the situation and was able to offer reassurance about ability to do the job.
• Be concise, but not too concise. The cover letter/personal statement should cover all the selection criteria, but avoid wordiness. One to two single-spaced pages is about right.

And if you get to interview? Well, this blog post has some useful hints:

10 Psychological Techniques to Help You Get a New Job

But what if you follow all my advice and still fail to get to interview? Alas, given the massive mismatch between the number of bright, talented people and the number of jobs on offer, many good candidates are bound to miss out. It certainly doesn’t mean you are unemployable. But try this exercise: look at the selection criteria and your application, and pretend you are the employer, not the candidate: An employer with a huge stack of applications and limited time. What do you think looks good, and what are the weaker points? Can you gain further experience so that the weaker points can be remedied in future job applications? Or maybe the weaknesses include something like a poor degree class, which can’t be fixed. Perhaps your specific set of talents and interests just aren’t a good fit to this kind of job, in which case you need to consider other options.  

If all else fails, you may want to cheer yourself up by reflecting on how people who don’t go along with the system can nevertheless have interesting and influential lives, by reading  Hunter S. Thompson's 1958 job application to the Vancouver Sun  

Susan Greenfield and autistic spectrum disorder: was she misrepresented?

I have had many emails in response to my open letter to Baroness Greenfield. All but one have been approving. The one exception is an eminent Professor who has chided me for misrepresenting her views. I am reproducing here our unedited email correspondence. I have anonymised the name of the correspondent, as he has not given permission for it to be used, though I will happily break the anonymity if he wishes me to do so, so he can take credit for his arguments.

As a non-celebrity scientist, I would like to get on with my day job and do some data analysis, and so have decided to reproduce the debate here, so that others can pursue it. Please feel free to comment, though please note, I will delete any comments that are off-topic, i.e. those not pertaining to issues around the validity of Greenfield’s claims, and the extent to which they have been misrepresented.

From: xxx@xxx.ac.uk

Sent: 10 August 2011 13:27 

To: Dorothy Bishop

Re: Misrepresentation of Greenfield’s article

 Dear Professor Bishop,

In your blog of 28 September 2010 you flattered yourself with the aspiration of being a “Paragon”. However, your blog of 4 August 2011 betrays that aspiration and violates the principles of scientific debate. You are misrepresenting Greenfield’s article in New Scientist. To claim that she is blaming what you call “internet use” for the grievous condition of autism is a travesty. The word autism does not appear in that article; Greenfield specifically refers to “autistic spectrum disorders”. Nevertheless, you implore her to “stop talking about autism” and unpleasantly characterise her comments as “illogical garbage”. For clarity I shall repeat myself: autism is not the subject of that article.

It is imperative that scientists engage with all sectors of society and do so accurately, honourably and without intemperate, personal comments. Publishing an assertion which misrepresents the evidence is unacceptable. Furthermore, your blog ignores Greenfield’s explicit references to peer-reviewed papers which provide data consistent with aspects of her general hypothesis (which is not about autism). Perhaps I should remind you of one of the key sentences in Greenfield’s article: “it is not the technologies themselves that I'm criticising, but how they are used and the extent to which they are used”.

Your failure to live up to the aspiration you expressed in your blog of 28 September 2010 saddens me and many other members of our community. In that blog you stated: “Paragons write personal letters to authors”. However, given the public pronouncements which you have made, a public retraction of your misrepresentation is now required. Your earlier experiences as an journal editor will no doubt confirm this requirement.

-------------------------------------------------------------

From: Dorothy Bishop

Sent: 10 August 2011 16:31 

To: xxx@xxx.ac.uk

Re: Misrepresentation of Greenfield’s article

I have no intention of withdrawing what I have said. I am happy to defend it.

You seem to think there is a clear distinction between autism and 'autistic spectrum disorders'.

There is not; many people treat them as synonyms, and those who interpret them differently regard ASD as a milder form of the same condition. There is no justification for linking either the severe or the broader category with internet use. The argument I made about a cause needing to precede it effect applies just as much to ASD, broadly defined, as to core autism. ASD does not suddenly appear in middle childhood - the symptoms are evident from around 2 years of age, and so are not plausibly caused by internet use.

If the article is not 'about' ASD/autism, then why does Greenfield mention it at all? This really does upset parents of affected children.

And isn't she aware of the large literature debating reasons for the increasing prevalence of ASD/autism diagnosis? - if she is going to cite this to support her argument, then it behoves her to do her homework.

It is really not acceptable to use innuendo to imply associations, but then back off if challenged to produce evidence.

See: http://www.guardian.co.uk/science/the-lay-scientist/2011/aug/08/1

There is a more fundamental problem here. Susan Greenfield is listened to because she is a scientist. But unlike other scientists engaged in public communcation, she does not confine herself to explaining science to a broader audience. She uses the media to promote her own new theories. What she conspicuously does not do is to publish these ideas in the peer-reviewed scientific literature. This is a shame because it means she has become disconnected from the rest of the scientific community. I would have been happy to voice my criticism by the more conventional means of peer review, which would have been private, or as commentary on a scientific paper, but I am denied that opportunity because Susan Greenfield does not publish these ideas in the scientific literature. Since her views are widely distributed through magazines and newspapers, those of us who find them flawed have no alternative but to challenge them in the public domain. I am aware that a great many people have made 'intemperate personal comments' about Susan Greenfield, but I do not accept that I have done so; I criticised the ideas rather than the person.

I might add that yours is the first critical comment I've had. I have had numerous supportive emails and comments from scientists who have not only written to say they agree, but have thanked me for raising this.

----------------------------------------------------------

From: xxx@xxx.ac.uk

Sent: 11 August 2011 09:49 

To: Dorothy Bishop

Re: Misrepresentation of Greenfield’s article

Dear Professor Bishop,

Thanks for your response.

You present yourself as sanguine about conflating Autism and Autistic Spectrum Disorders. I find this surprising and alarming.

Your case now rests on your conviction that all of the adolescents or adults who are currently being diagnosed with any Autistic Spectrum Disorder (at an increasing incidence) could have been diagnosed as such “from around 2 years of age”. Please direct me towards peer-reviewed prospective studies which support this claim.

---------------------------------------------------------------

From: Dorothy Bishop

Sent: 11 August 2011 10:49 

To: xxx@xxx.ac.uk

Re: Misrepresentation of Greenfield’s article

I will send you some peer-reviewed papers when I have some free time, but meanwhile, please see Criterion C in the DSM5 proposed revision, as well as the rationale section, which explains the terminology.

You might also find it useful to talk to Professor Sir Michael Rutter, who is the world's leading expert on autism.

--------------------------------------------------------------

From: xxx@xxx.ac.uk

Sent: 11 August 2011 12:00 

To: Dorothy Bishop

Re: Misrepresentation of Greenfield’s article

Criterion C in the link you have provided does not address the matter in question: namely, whether there is well-controlled evidence which supports your conviction that all of the adolescents or adults who are currently being diagnosed with any Autistic Spectrum Disorder (at an increasing incidence) could have been diagnosed as such “from around 2 years of age”.

Criterion C merely raises a circular argument, which would be susceptible to unreliable retrospection.

I will indeed raise these matters with Michael Rutter.

But, more importantly, I look forward to receiving from you peer-reviewed papers which substantiate your specific claims.

Sincerely

-----------------------------------------------------------------

Dorothy Bishop

Sent: 11 August 2011 15:51 

To: xxx@xxx.ac.uk

Re: Greenfield’s article

Your initial complaint was that I had misrepresented Greenfield because I had failed to distinguish ASD and autism. I trust the DSM5 document has clarified the point for you and you now accept this was not misrepresentation.

You are now demanding that I provide peer reviewed evidence for my supposed "conviction" that "all of the adolescents or adults who are currently being diagnosed with any Autistic Spectrum Disorder (at an increasing incidence) could have been diagnosed as such “from around 2 years of age”.

I have sent you information pointing out that it is is part of the diagnostic criteria for ASD to have onset in early childhood.

This is not a circular argument. It is merely pointing out that ASD, as defined by gold standard diagnostic criteria, could not be caused by environmental influences that only start in later childhood.  I reiterate the last sentence from the DSM 5 rationale section: "Autism spectrum disorder is a neurodevelopmental disorder and must be present from infancy or early childhood, but may not be detected until later because of minimal social demands and support from parents or caregivers in early years."

Note that this does not mean that all children with ASD will be diagnosed in childhood, but it does mean that they have evidence of autism in early childhood.  This is typically identified by an interview instrument such as the Autism Diagnostic Interview.

To clarify my argument.

1. When asked for evidence that the internet is changing people's brains, Greenfield stated, among other things, "There is an increase in people with autistic spectrum disorders."

To most people this would imply that she is saying the internet is a causal factor in the increase in autistic spectrum disorders.

2. There has been an increase in autistic spectrum diagnoses over the years.

However, this evidence comes from epidemiological studies that do use standard diagnostic criteria including the onset criteria (see attached articles).

3. Since internet use cannot plausibly cause a disorder starting in a toddler, this is not a valid argument.

You now demand that I prove that "all of the adolescents or adults who are currently being diagnosed with any Autistic Spectrum Disorder (at an increasing incidence) could have been diagnosed as such “from around 2 years of age”. "

This is an attempt to move the goalposts. Of course diagnosis is not perfect. There may be misdiagnosed cases. The fact that you demand this evidence suggests that Greenfield's argument (as filtered by you) is now :

a) there are children who don't have autism in early childhood but who develop some kind of quasi-autism in middle childhood

b) this is caused by internet use

c) such cases account for the increase in ASD diagnoses, even though they don't meet DSM criteria for ASD

Do you have any evidence for any of these postulates ?

If that is not what you are saying, what exactly is the claim?

You have also not responded to the point I made about the appropriate place for a scientist to publish new scientific theories. Do you think it is appropriate to make statements about aetiology of a major neurodevelopmental disorder in a non peer-reviewed journal such as New Scientist, when there is no peer-reviewed work to back them up, even if the causal claims are by innuendo rather than direct statement?

If you would like your point of view have broader recognition, I would be happy to publish this correspondence on my blog, so that Greenfield's position and the supposed limitations of my arguments could be given wider publicity.

 pdfs of the following papers were attached:

Baird G, Simonoff E, Pickles A, Chandler S, Loucas T, Meldrum D, Charman T: Prevalence of disorders of the autism spectrum in a population cohort of children in South Thames: the Special Needs and Autism Project (SNAP). Lancet 2006, 368 (9531):210-215.

Baron-Cohen S, Scott FJ, Allison C, Williams J, Bolton P, Matthews FE, Brayne C: Prevalence of autism-spectrum conditions: UK school-based population study. British Journal of Psychiatry 2009, 194:500-509.

Brugha, T. S., McManus, S., Bankart, J., Scott, F., Purdon, S., Smith, J., et al. (2011). Epidemiology of Autism Spectrum Disorders in Adults in the Community in England. Arch Gen Psychiatry, 68(5), 459-465.

Fombonne, E. (2005). The changing epidemiology of autism. Journal of Applied Research in Intellectual Disabilities, 18, 281-294.

Kim, Y. S., Leventhal, B. L., Koh, Y.-J., Fombonne, E., Laska, E., Lim, E.-C., et al. (2011). Prevalence of autism spectrum disorders in a total population sample. American Journal of Psychiatry.

Rutter, M. (2005). Incidence of autism spectrum disorders: Changes over time and their meaning. Acta Paediatrica, 94, 2-15.

Taylor, B. (2006). Vaccines and the changing epidemiology of autism. Child: care, health and development, 32(5), 511-519.

Williams, J. G., Higgins, J. P. T., & Brayne, C. E. G. (2006). Systematic review of prevalence studies of autism spectrum disorders. Archives of Disease in Childhood, 91, 8-15.

Wing, L., & Potter, D. (2002). The epidemiology of autistic spectrum disorders: is the prevalence rising? Ment Retard Dev Disabil Res Rev, 8, 151-161.

P.S. 13.52 on 12th August 2011
A further response from xxx

Dear Professor Bishop,

I am astonished by your peremptory decision to publish our correspondence without permission. I ask you to add the response below, without any editing, as a matter of urgency.

Dear Professor Bishop,

In your first email you stated: “ASD does not suddenly appear in middle childhood - the symptoms are evident from around 2 years of age”. This non-ambiguous statement means that all people who are diagnosed with an Autistic Spectrum Disorder after early childhood will have been displaying its symptoms from around 2 years of age.

You now point out: “it is part of the diagnostic criteria for ASD to have onset in early childhood”. The difference from your initial statement is salient. Thus, it is the case that that unless those symptoms are present in early childhood, an Autistic Spectrum Disorder may not, by definition, be diagnosed.

In this context, you draw attention to “Criterion C in the DSM5 proposed revision”. As I am sure you realise, DSM5 will not supersede DSM-IV until 2013. The criteria you describe as “gold standard diagnostic criteria” are part of a proposed revision.

I shall consider just one matter arising:

Autistic Disorder and Asperger’s Disorder are addressed separately under DSM-IV. The current diagnostic criteria for Asperger’s Disorder (DSM-IV) include the following: “There is no clinically significant general delay in language (e.g. single words used by age 2 years, communicative phrases used by age 3 years). There is no clinically significant delay in cognitive development or in the development of age-appropriate self-help skills, adaptive behaviour (other than in social interaction), and curiosity about the environment in childhood”. Indeed, a delay in social interaction is the only age-related point mentioned; no critical age is given for its onset.

I recognise that the revisions for DSM5 under current consideration are being guided by the following:

”Asperger’s Disorder. The work group is proposing that this disorder be subsumed into an existing disorder:  Autistic Disorder (Autism Spectrum Disorder)”.

If this were to be enacted, diagnosis of Asperger’s Disorder would be precluded, unless its symptoms were present in early childhood (as specified by Criterion C). Again, I feel it is appropriate to ask for evidence which supports your original statement: “the symptoms are evident from around 2 years of age”. According to your gold standard DSM5, this must apply to Asperger’s Disorder. It is reasonable for me to ask whether this has been substantiated by prospective studies which are free from potentially unreliable parental retrospection. I may be in error, but I have found no such study among the papers you kindly sent me. I sincerely apologise if I have overlooked something relevant.

The immensely complex matters of aetiology and diagnosis are not given due consideration if proposed revisions (which are still subject to consultation) are presented as “gold standard”.

In my preceding email I wrote: “I look forward to receiving from you peer-reviewed papers which substantiate your specific claims”. I am saddened to note that you have chosen to misrepresent this polite request as “demanding”. It seems that our discourse will not be fruitful and that it should be closed.