BishopBlog

2011 Orwellian Prize for Journalistic Misrepresentation

2012-01-29T08:21:00.002Z

So the time has come round for the announcement of the 2011 Orwellian Prize. The prize is given for an article in an English-language national newspaper that achieves an unusually high level of inaccuracy. Only articles that describe a piece of published scientific research are eligible. Points are given for every statement in the article that does not match the original source, as follows:

Factual error in the headline: 3 points
Factual error in a subtitle: 2 points
Factual error in the body of the article: 1 point

Last year, I got two nominations, but, as described here, neither adequately met the criteria. This year, I’ve had just one nomination, from Neurobonkers, but it’s set a standard for inaccuracy that will be hard to beat. The article that I first selected in 2010 to illustrate the scoring system scored 16 points. This one achieves a startling 23 points. The source article by Kucewicz et al (2011) can be found here. Here is a screenshot of the account in the Daily Mail, with errors marked in red (3 points), orange (2 points) and blue (1 point).

There is a detailed analysis of errors in this blogpost by Neurobonkers, which I urge you to look at. Suffice it to say, the academic paper is not about cannabis, smoking or schizophrenia. Rather it is about an artificial compound that is not present in cannabis, which was injected into rats, and which led to changes in their brain waves.

There were some complaints to the Press Complaints Commission, and presumably in response to this, the article was modified. The headline, which originally read Just ONE cannabis joint 'can bring on schizophrenia' as well as damaging memory was altered to Just ONE cannabis joint 'can cause psychiatric episodes similar to schizophrenia' as well as damaging memory. Perhaps even the Daily Mail found the notion of a schizophrenic rat implausible. But the rest of the article remains, as a scare story about cannabis. And here is what makes this article such a prime candidate for the Orwellian Award: this is not about a hyped press release by a university, or misunderstanding of complex science. It's not even about sensationalising a scientific finding to draw readers in. No, this is about using a scientific paper as a prop in the Daily Mail's anti-cannabis campaign. A ploy that the newspaper has previously used in another ideological battle, on climate change. When reporting research, no respect is given to the truth: scientists are simply used to bolster a preconceived opinion, and if they don't do that, their findings are distorted.

Twelve of this article's 23 points came from the headline. Journalists don’t write the headlines. They therefore dislike my scoring system because it penalises errors in headlines more than errors in the body of the text. My view is that it’s the headlines that count for most: far more people will read the headline than the text, and for many readers it's the only part of the article they will process. It’s important that it's accurate. Although few would defend frank lying, some editors seem to think it doesn’t matter if a headline is hyperbolic, provided it sells the paper or gets someone to read further. This very issue has been a topic of debate in the recent Leveson Inquiry into culture, practice and ethics in the UK media. I feel strongly that it's a cop-out to just wash one's hands of it and blame anonymous sub-editors for misleading headlines. I shall therefore continue to award points in proportion to the prominence of the material. But I appreciate it’s not then fair to make the award to the journalist. Indeed, given the Mail's agenda on cannabis, the journalist in this case may well have been under duress to write a scare story. I will accordingly be making the award to Paul Dacre, Editor of the Daily Mail. I will be happy also to send the token of appreciation for the nomination to Neurobonkers if he/she is willing to email me to tell me where to send it.

I'm pleased not to have had more nominations this year: it suggests that, despite all the grumblings about science journalism, the field is in rude health. I've certainly read a lot of good science reportage in our national newspapers, and where articles have made me angry, it's often because of hype by a press office or scientist, rather than distortion by the press. There are, however, still a few topics, among them drugs policy, where the political stakes are high and scientific reporting is cynically exploited to support an otherwise weak argument.

Coming up with an award certificate and item turns out to be an excellent way of avoiding doing serious work.....

Reference
Kucewicz, M., Tricklebank, M., Bogacz, R., & Jones, M. (2011). Dysfunctional Prefrontal Cortical Network Activity and Interactions following Cannabinoid Receptor Activation Journal of Neuroscience, 31 (43), 15560-15568 DOI: 10.1523/JNEUROSCI.2970-11.2011

Psychoanalytic treatment for autism: Interviews with French analysts

2012-01-23T22:50:00.000Z

Last week a French documentary called “Le Mur” attracted attention in the New York Times. Plans to screen the film later this week in Philadelphia are being contested by three of the interviewees. They are all psychoanalysts who treat children with autism, and they complain that the film has been unfairly edited to make them look ridiculous. They are suing the film-maker, Sophie Robert in her native France. All the signs, though, suggest that their legal action is causing a Streisand effect, drawing attention to them and generating considerably more criticism of their activities than would otherwise have been the case. A website called “Support the Wall” has links to the documentary, with English subtitles, and to other sites highly critical of France’s idiosyncratic approach to autism intervention.

At nearly an hour long, the film is quite demanding to sit through, so I thought it might be helpful to post a transcript of the interviews with psychoanalysts, so people can judge for themselves whether the analysts have been ridiculed.

This text should, however, be read bearing the following points in mind:

The translation was not done by a native English speaker, and I simply copied the text from the subtitles, rather than attempting to improve on the translation. It is important to realise that some apparently incoherent utterances could be due to poor translation.
The subtitling is very amateurish and uses multiple colours in a way that can make it difficult to read at times. Unintelligible material is denoted as (xxx)
The speakers are indicated in brackets, with (I) denoting the interviewer (presumably Sophie Robert). All other speakers are psychoanalysts.
My ability to keep track of all the analysts is imperfect, and so I may sometimes have misattributed a statement - please refer to the original movie if you want to be certain of who said what.

(Update, 26th Jan 2012: World Service Health Check podcast on this topic)

The Wall

Introduction

For more than thirty years, the international scientific community has acknowledged that autism is a neurologic disorder that is the cause of a handicap in social interaction. All autists have the same anomaly in one area of the brain, the upper temporal line¹ identified in 2000 by Dr Monica Zilbovicius. In France, psychiatry, being very largely dominated by psychoanalysis, ignores these discoveries. To psychoanalysis, autism is a psychosis. In other words, a major psychic disorder resulting from a bad maternal relationship.

¹ Comment by DVB: I assume this refers to the superior temporal sulcus. I regard the statement about anomalies in all autists as an over-simplification

------------------------------------------------------------------

The Interviews

(Dr Alexandre Stevens) I think that autism is a mode of reaction of the subject that is obviously very early in his logical history.

It is that (xxx) response to what comes as invasion of the world and of the other.

He clams up.

He clams up, he gets into a bubble and refuses to enter the mechanisms of speech.

But (xxx) some autists speak, don't they

So it is more than speech, in the subjective mechanisms

I mean, speaking, yes, but without being really involved.

(interviewer) Do you make a structural distinction between psychosis and autism?

(AS) No, I don't.

(I) Are autists psychotic?

(AS) Yes. That is to say, autism is an extreme situation of something that is in the scope of the set of psychoses.

*****

(Professeur Pierre Delion) The structural distinction that one can make between autism on the one hand and child psychosis on the other hand

that is (xxx) the aspect of the description of the underlying structure does not stand up to analysis in the continuity of this first structure and the second one.

It seems to me that there are paths between the autistic structure, the psychotic structure, and the dysharmonic structure.

Now, when we say that, we upset many associations of parents of autistic children who think that autists have nothing to do with psychotics

To me, this is a mistake; there are many things in common.

*****

(Dr Genevieve Loison) It is the crocodile! (gets out toy crocodile)

So the crocodile tells us right away what it is all about.

They play with it and when they put the hand or an object in, I am worried.

(demonstrates with hand in mouth of croc)

When they get on top of it and hit it, I feel reassured. They are fighting!

(I) Why? What does it mean, the crocodile?

(GL) The crocodile is the mother's belly, the mother's teeth

(I) Is that what Lacan used to call the mother: a crocodile?

(GL)Yes, so the goal of our work is to forbid her to eat.

(I) To eat the child?

(GL) We put a stick in him

When the child starts getting out this, sometimes he puts his hand

(demo hand in croc mouth). Sometimes he puts a figurine in.

(I) And this pen, what does it stand for?

(GL) (Puts pen across croc jaws) So this means "You can't any more" It is the bar.

(I) Is it the father's phallus?

(GL) That's it. You can't do that any more.

(I) It is the father's law, that bars the child from his mother and that forbids the mother to destroy the child.

(GL) Right, and to devour him.

(I) Does this also relate to autistic children?

(GL) Of course, yes. For autists it is a matter of stage, you see.

We take the children at some stage.

The autistic children, they often put the child in there (has what looks like a tortoise in shape of a bag?)

They go in there. Then, we get worried, right?

(Interlude with family with autistic children)

(05:15)(I) Regarding both psychosis and autism, there is an explanation that is traditionally used is that a maternal depression during pregnancy or the first months of the baby’s life would have altered the mother-child relationship, and could be responsible for severe disorders.

(Prof Daniel Widlocher) So, this is what serious colleagues tell us, so consequently I listen to them.

*****

(AS) It can be the case, when the child comes in such conditions where the other, his first “other”, the mother is very depressed. I mean she will be absent from him, will be in another look at him, that it could sometimes make it so that the child chooses to withdraw.

Sometimes, when the mother is depressed, when she is pregnant or at birth, sometimes the child can be autistic.

*****

(GL)Woah, well, we have seen a few of them. I remember, it was when I was in a psychiatric clinic that I have seen them, mostly.

I have seen massive things, massive voids, disorders.

I have seen abandoned mothers! For each autist there was, I would say, a different cause.

I have seen many mothers abandoned at the end of their pregnancy.

And the child… depressive mothers too, mothers in the baby blues afterwards, abandoned at the end of pregnancy, who have a baby blues after delivery, who are themselves in the emptiness and who put the child in the relational emptiness.

*****

(DW) For example, deep deficiency

For instance, a mother who, for reasons of depression does not take any care of her child, stays mute in front of him, clotted, while the child is there, starting to hang on to her, looking for contact.

It can, if it lasts long during childhood have consequences on the young adult who will keep being very cold, inhibited from an affective perspective, because, when he was a child, he unfortunately had a very cold mother.

*****

(I) Is there a higher incidence of the cases of psychosis or autism in countries where there is a war?

In the favelas of Rio? In all circumstances where mothers have good reasons to be depressed?

(GL) I am not aware of that, I cannot answer.

I can only answer about what happens in my practice, you see.

(07:55) (interlude)

(08:41)(I) How can one explain that a maternal depression could cause a child to be autistic?

To start with in utero?

(09.04)(AS) So, first of all, in utero some things already happen

I am not a specialist of in utero, but there are things happening.

The child moves, mothers speak

Mothers talk to him and react to a number of stimuli

In fact, what do children know about this link to the other, in utero?

It does not completely go through language.

But it goes through the whole environment of corporal sounds

Finally, a depressed mother, when the child is there in utero, does the child feel anything?

I am totally unable to tell you, but I do not feel like it is odd to think so.

*****

(Dr Aldo Naouri) Most of the time, the symptom of the child is no more no less than the symptom that allotted to have by the maternal unconscious mainly

Because children are in a relationship that is very very permeable in the communication with their mother

Gestation literally conditions a child and forms him.

And really gives him something entirely produced by the body, that comes from his mother's body.

This weakens the field.

It puts the child in a state where his resistance is altered by the conditions that come from his environment

*****

(10:22)(Prof Bernard Golse) For the baby, there is one half of his genes, of his chromosomes are from his mother the other half from his father

So, there is one part that is like the mother, this one does not pose any problem

but there is one part that is like the father and that immediately poses a problem

As soon as the baby is conceived, the motherly organism will immediately secrete a very strong wave of antibodies to expel the baby that is half stranger to the mother's body

It is a little sad to say, if I can say, the first thing that biologically, the mother cannot stand in her baby is the part that comes from the father

So what is anthropological in there is the double "no"

No, I do not recognise this baby; I want to eject him, and at once a no to the no.

Which we will find later in the language

There is a whole interesting question

and in language one will find such double negations.

This comes at the biology level.

(further interlude with family)

(I) Why don't severe autistics speak?

(Prof Daniel Widlocher) I don't know

My first idea is that they do not speak because they are not stimulated for speech.

Like children that are very little surrounded by a stream of language often have language difficulties even though they do not have autism

Consequently there is language stimulation, a parental stimulation of the environment that is very important

*****

(AS) We are not neurologically wired for language

Psychoanalysis does not belong here, if you want

We are not wired for language.

Of course it takes the capacity to speak but the essential thing is that precisely language is outside of the organism

*****

(GL) Why would they communicate, they are in fusion!

They communicate without words, there is no need to acquire speech

Speech implies access to the symbolic, access to the father.

I am not only with my mother

There exists a father who created me in the origin, who was the founder, with his paternal might, who has created a child.

Only progressively with the child discover this.

(I) In what does the father embody the symbolic?

(GL) Ha, this is difficult

He is access to abstraction, access to the distance

Access one has to speak when one is no longer glued

He comes to interpose between the mother and the child

*****

(Prof Bernard Golse) Speaking with the other means that one is no longer merged with the other.

As long as one is no longer separate, as long as one is combined in the other

which is the problem for autistic children, precisely for whom the other does not exist

but when the other starts emerging they are still in such a pathologic distance that they can't speak with the other

*****

(I) About autistic children it is said that their mother cannot catch their eye?

(GL)Yes, they are not at all in the relationship

they are staying in an egg

(I) They flee the mother because they are merged with the mother?

(GL) They have not taken off at all.

They have stayed in the egg, in the uterus

Why do you want them to look or speak?

*****

(14:22)(BG) You can not think of the other one, you can not speak with the other one unless you have some free range.

As long as you are still included, interpenetrated, merged with the other one, you can neither think to the other one, nor speak with him

this is almost a truism

If language touches us so much all our life long, and psychoanalysis goes through language it is that language is only about separation.

That is a vision.... very very well.... it is a vision of development that is extremely specific to the psychoanalytic vision of things

(interlude with family)

(I) The Psyche does not evolve independently of the brain. It does not wander alone in the void.

If we have one side a normal child with a well-functioning brain and on the other side an autistic child with a malfunctioning brain, doesn't it make a fundamental difference in its capacity to communicate with the outer world?

(Esthela Solano) This way of conceiving the causality of autism is very reductive.

What we can notice when we take care of autistic children is precisely that autistic children are sick of language

That autism is a way to defend themselves from language

(interlude with family)

(voiceover) The Austrian-American psychoanalyst Bruno Bettelheim was the precursor of the psychoanalytic treatment of autism.

Bettelheim compared autistic children with concentration camp prisoners that were swinging back and forth, staggered by terror, waiting for an imminent death.

Reasoning by analogy, he was convinced that autistic children were the victims of torturer parents, freezing mothers who had desired the death of their child.

Bettelheim's work has been largely rejected in the USA since more than thirty years.

But how is it in France?

*****

(PD) Bruno Bettelheim is a victim of injustice of contemporary history.

I think that he did his work at a time when no one cared for autistic children, an absolutely exemplary pioneering work.

He got interested in autistic children because he was coming out of an artificial deficiency experience that he had gone through in concentration camps.

He arrived on the autistic planet thinking that finally here are children that maybe suffered from deficiencies in a manner similar to what I have lived myself.

I will try to treat them from this hypothesis, meaning by separating them from their parents.

*****

(GL) There was absolutely no tie.

Mothers who have them so fear them, even during pregnancy

And after, there is absolutely no tie.

And the child is abandoned and can even wonder how he has been conceived.

But the fact is that they were conceived.

He is totally empty, pregnancy is only organic, the child does not exist as a person.

He is totally in the relational emptiness.

And this gives what Bruno Bettelheim, yes, of course

*****

(Yann Bogopolsky) No no, Bruno Bettelheim has said much more than that.

This is the caricature.

I do not say this for you but for those who well

It is not only about mothers, it is also about the father's place in the mother's desire.

And it is about a certain amount of data, not only those with which the child is born, and the link that has been created, or the desires of death that such mother could have had for this new born child.

But all the mothers who have desires of death for their child, all these desires will not necessarily make their child psychotic!

(interlude)

(voiceover) At the end of the 60s, psychoanalysis is starting to decline everywhere in the world.

But it takes a phenomenal success in France, with the impulsion of an ambitious and charismatic psychiatrist.

For Jacques Lacan, psychotic and autistic children are victims of the alienation by a psychogenic mother.

A woman who refuses to stop her pregnancy because she is unable to split from a child who is a substitute for a penis that she has not received when she was born.

*****

(YB) At the beginning, the child thinks that he is his mother's phallus.

Namely that he is this object that would give everything, fill his mother with joy, make her have orgasm.

*****

(I) Therefore autism is maternal fusion?

(GL) Wait, there are a lot of theories, right?

To me, I have either fusion or total abandon and emptiness.

It's one or the other. The two poles. It's not the same.

But when they are totally in emptiness

but what I deal with here at the practice, is rather fusion

For total abandons, I do not see them, they go to institutions.

(I) Bruno Bettelheim says that autistic children are victims of mothers that are too cold, and on the other side we have Jacques Lacan who theorizes psychosis and autism as maternal fusion, through some sort of a relationship that is almost incestuous between the mother and the child?

(GL)Yes, yes, absolutely

(I) So on the other side, we have a mother who is too warm?

(GL) Yes, too warm or too cold is not good.

It takes a mother, as Winnicott says, good enough.

Neither too good too warm, neither totally empty of course.

If there is nothing, no relationship, it will not give anything good.

(I) How can you explain that two attitudes, two totally opposed behaviours, could yield the same result?

(GL) There are different mechanisms; it is our job to put things back in the middle.

*****

(voiceover) The English analyst paediatrician Donald Winnicott is the author of the concept of "good enough mother", according to which a mother who wants to do too well is toxic for her child.

He claims that motherhood would take mothers and infants through a transitional madness stage, prototype of forthcoming psychic disorders.

*****

(GL) (approaching with scary toy spider) Unconsciously, this is what you are doing to me, in your legs.

*****

(voiceover) With Winnicott, we have come full circle

Motherhood is psychogenic by nature.

*****

(I) Do you agree with the idea of maternal madness?

That every woman, every mother goes through the first months after delivery a stage of madness that can later be problematic for the infant if it lasts?

(AS) Yes, yes ,but this name maternal madness does not disturb me at all.

(I) You are a psychiatrist, madness is definitely something with a negative meaning isn't it?

(AS) Not for me

*****

(Laurent Danon-Boileau) There is this idea that is usually called the maternal madness of the first times

There is a moment when the mother is so glad to have her brat that she is the one with the brat and there is no way anyone could intervene

(I) Why call this madness, by the way?

(LD) Because of this idea of some sort of wholeness that gives some kind of powerful might.

There is this idea that, starting when you have done such a thing, you are almighty, nothing can reach you, etc etc.

*****

(GL) You know, madness, it's the first month of life

But the thing is we have all been mad, or almost

(I) Why call this thing, this capacity for the mother to decrypt and understand her child's behaviour without language, why call this madness?

(GL) Because if we stopped there, the child would go into madness

Madness is precisely what you were asking about earlier about autism; they stopped there

*****

(LD) Maternal madness, is maternal madness a major roadblock to the emergence of language?

Said like that, of course, yes!

*****

(25:40)(AN) The mother is one the side of nature.

and from this standpoint, she is animal, if you want.

whereas the father was the one who founded culture

This is what made Claude Levi Strauss say this superb definition when he speaks about the couple.

He says that it is the dramatic union of nature and culture.

You see, when law supports culture, claiming that there is something fatherly, it means we are in the culture, return to nature shall not happen.

(interlude)

(voiceover) In the course of my interviews, several psychoanalysts had asserted that autism was the consequence of a maternal incest

*****

(AS) The maternal function consists of intervening in two ways

On the one hand saying no to the fusion between mother and the child

The father is the one forbids the mother

(I) He forbids sexually? Right?

(AS) Who forbids the pleasure of the mother

Namely, without ambiguity, who forbids that the child takes pleasure exclusively with his mother

as well as the mother takes pleasure exclusively with the child

*****

(Jacqueline Schaeffer) At the child's birth there is a honeymoon, right?

Sometimes, it is not so much of a honeymoon, it can be dramatic

But there is a honeymoon, there is a fusion, there is an extraordinary

Well, at the same time, there is a huge pleasure together

The baby very quickly, well, there is what we can say

There is not sex difference but there is a big erotic pleasure taken together

*****

(I) The fact that a mother takes care of her child like a human being, this is not sexual

(YB) Well, yes it is!

Sexuality, in a first place, is not in the sense that all take it, as genital sexuality

It is everything that, to Freud, because he is the one who theorized this, will be the body's parts that will give pleasure to the child

So the child and mother bodies are there in a tight unison and even the infant child does not know that he is not his mother's body.

He thinks they are one.

Well, he thinks - we attribute t his to him - he thinks he is one with his mother.

(29:36)So, it is a work of separating the bodies.

And the pleasure that the child will get in his relationship with the other, who is first his mother, will happen, one, in a clinch

And two, the pleasure will be related to the different orifices of the body.

*****

(LD) There are psychoanalysts who, a long time ago, have talked about something they called the lover's censorship.

This means something very simple.

It means that the mother is changing the infant and that all of a sudden; she takes too much pleasure by touching the infant

And she feels, oh la, this is strange

It is not logical; I am not treating him like

(I) An incestuous thing?

(LD) Something a little incestuous, what's going on?

Well, she will think of the man with whom she conceived him, in other words her lover.

And therefore, this will create a distance between herself and the infant.

This is what we call the lover censorship.

(I) She has to have a man on her mind for her not to be in an incestuous relationship with her child?

(LD) It's not "she has to" - one cannot avoid it!

*****

(JS) During maternal care, the mother can very well arouse the penis of a little boy.

I mean, maternal care, and the little boys do not avoid this!

We see many little boys who, when they get changed, when they get washed by the mother, the little penis reacts

*****

(YB) The father is there to forbid and at the same time protect the child.

That is, to protect the child from the incestuous desire of the mother.

(I) Do you think that all mothers have incestuous desires towards their child?

(YB) Oh yes, whether they are aware of it or not! Yes!

(I) Why are you so sure?

(YB) Well, first from psychoanalytic writings.

I mean, desire is not forbidden, its fulfilment is.

(I) But, that the love of a mother for her child is an incestuous desire, this is-

(YB) Well yes, because she already has some trouble to separate from her child.

There is a unity that gets done only if the father of the child, or the one in charge of this who is not necessarily the genitor, does not come and tell the mother that enjoyment has to happen between them.

Does not remind his desire to the mother, and the pleasure that she will be able to have through the desire that ties them together.

The mother will have pleasure by caressing her child

By having him on her body all day, what else, she will have pleasure

And, back to your previous question, this is how the body will present and that sexuality, insofar as it is related to body pleasure, and that is reused in language, is naturally present in the questions of psychoanalysts.

(I) Then, why are most of the incests perpetrated by men?

*****

(AN) First we have to notice that father-daughter incest is definitely more frequent than mother-son incest

And why are there not, or no more, mother-son incests?

For a very simple reason, which is that the motherly attitude to children, whether they are boys or girls, is spontaneously an attitude of incestuous essence.

Every mother's dream is that her child does not lack anything

The motherly tendency towards the child is an incestuous tendency

And there is no need to act. In her attitude, there is already enough, she does not act.

*****

(JS) Paternal incest does not cause so much damage. It makes the girls a little moronic.

But maternal incest creates psychosis. That is, madness.

There cannot be a maternal incest between a boy and his mother without an enormous mental disorder. It is not possible.

Precisely because of this barrier.

But the daughter with the father, she did not come out of the father, right she has not been in the father's belly

So there is something, you see. I'd say it is a secondary incest.

Whereas the primary incest, the real one, it is the mother

It is to penetrate the mother.

And boys who penetrate their mother are psychotic.

Whereas the daughter is not.

The girls can, we have many experiences with daughters with a paternal incest, they can manage somehow.

Now, I can't say that it works very very well

Some of them are very very bad, some a little less so

And there are those who manage by some other way

What I was saying, a little moronic. But it is very different.

(interlude)

(voiceover) To psychoanalysts, the father is guilty too.

Guilty of being absent or transparent, guilty of being submissive to his wife.

Guilty of having been unable to intervene between the child and the maternal ogre.

*****

(GL) He failed.

Often, do you know how many times we see fathers who would have liked to?

and this is what we call forclusion

Forclusion of the father’s name

(I) What does it mean?

(GL) Good question!

It means that the father, in simple terms has made the child but that he (the father) does not exist.

It is denied, his existence is denied.

(I) Does the mother deny his existence?

(GL) Denies his function, his existence. There is only her and the child who count

The father does not exist.

Maybe he is there to bring some money in, he is here like an extra

He does not have the function of the husband, loved, considered in his speech

When the mother considers the father’s speech, the child discovers speech.

(I) If the child does not speak, it is because the mother discredits the father’s word?

(GL) Well, there are not so many here; it is mostly in institutions; in psychiatric clinics, where I had severe autists.

*****

(ES) Fundamentally, the father’s function is symbolic.

And sometimes the real father does not carry this symbolic function.

He can be adorable and nice, but nevertheless the child faces a symbolic deficiency on the side of the paternal function.

*****

(36:26)(GL) Either he does not have room or he only exists through violence, or he is violent himself.

We sometimes realise that he is violent to exist.

And that he pries doors open in a way, when no one opens them for him!

*****

(ES) When on speaks about the paternal function it is about something that drives you in your life.

It is like a highway, a compass.

(I) Why wouldn’t the mother, as a woman, provide the child this basis? Even on her own?

(ES) There are some mothers who can transmit a paternal function.

(I) But why isn’t it a maternal function? Why isn’t it a maternal symbolic? Why give it a gender?

(ES) The law of the mother is the law of whim.

*****

(AN) In 1984, a biologist established this extraordinary proof that placenta is from exclusive paternal origin.

So that it is under the control of genes brought by the spermatozoon.

In other words, the placenta is what prevents the mother from destroying her child, and a child from killing his mother.

(I) What enables the mother to feed her child?

(AN) Yes, that allows, that transfers and filters all the food the mother brings him.

In other words, it is a regulating element between them, this placenta it is an interposition.

I mean it feels like the father’s attitude within the decisions he takes, this patriarchy that he sets up, this male domination, etc., has always been the empirical search of the function that the placenta holds.

And that lets every child come into the world without being destroyed.

(interlude)

(voiceover) Since Bruno Bettelheim, the major psychoanalytic principle for the treatment of autistic children consists in separate children from their parents.

Parents are sometimes subject to pressure to undergo therapy because they are considered the source of the problem.

Today still in France and Belgium a number of psychiatric institutions are places that are not accessible to families, who are not informed of what goes on.

What does the psychoanalytic treatment of autistic children consist in?

*****

(ES) Let’s say that when we receive an autistic child we practice a psychoanalysis that is pure invention.

We are facing a subject who, most of the time, cannot speak.

*****

(LD) I am rather like in an observation attitude.

I mean, with an autistic child, I do very little.

What does very little mean?

That I sit my butt down close to him, and I wait for something to happen.

And I forget, I try to forget everything.

I forget time, I forget that we are pressed by time for him to acquire language, I forget everything.

Because I tell myself that, since I am in this kind of weightlessness, there could well happen something that I can not foresee.

*****

(41:20)(AS) You have authors like Tustin or Magaret Mahler who explain how it is first about winning over the child.

Here we are in a practice, but it is the same in an institution.

Winning over the child. I do not remember which of them describes this.

It is about stepping a little back, not in his line of sight.

Not speaking too loud, rather following what he says than anticipating it.

*****

(ES) No willingness of control, no educative willingness

No compelling of anything.

*****

(BG) We finally try by all means to make the autistic child feel that another one exists who is not threatening.

*****

(ES) It is precisely about taking into account the most insignificant details and figure out that this insignificant detail can be interpreted with some meaning.

And bit by bit we can proceed by supposing that there is a sign and we take it as something like the intent of a speech.

*****

(LD) Personally, if the kid does not do anything during the session, if I drowse beside him, I don’t care.

I am used to that in my work as a psychoanalyst.

But this implies a few things.

One, it implies not getting bored when we think with our own ideas.

Two, it implies rowing against this whole social thing that pushes you to be pressed by time it is all very nice, but if nevertheless if the child is still like that in ten years, it’s not on you to … etc.

All this is true; I stay in the position of a psychoanalyst, which means having no memory, no expectation.

And starting from then, something happens.

And that is an attitude, I believe, that is a deeply psychoanalytic attitude.

*****

(BG) When… Perhaps… Well..

I am sensitive to what you tell me that the audience should feel that we accept to fully endorse our convictions.

*****

(LD) If you sing a little song, and that the autistic child doesn’t feel bad with it there is something happening.

So you’re going to tell me, there is no need of a psychoanalyst for that.

Yes, if you are not using it as an educative method.

Yes, if you that that after all I don’t care about what the child will do with it.

It seems like he seems interested

We do that and we’ll see what happens then. Or we won’t see then.

The fundamental point in my attitude as a psychoanalyst with respect to these children it is to abdicate the idea of a progress.

And this is not easy, you can believe me.

My analytic ideal demands to abandon this dimension, but it turns out that I belong to a society in which I am paid to give care.

Consequently, I am in a conflict.

But this, a situation of conflict an analyst must be able to endure it or he takes another job!

Because this is the basis of our analytic practice.

*****

(I) What is the impact of psychoanalysis on autistic children?

What can an autistic child reasonably expect in terms of result?

(PD) But I can’t answer this, this is not a matter for a psychoanalyst!

*****

(I) What can an autistic child reasonably expect from an analytic work?

In terms of results?

(45:20)(LD) (very long pause) the pleasure of taking interest in a soap bubble.

I can’t answer anything else.

(interlude)

(voiceover) Nevertheless, there are solutions.

they are called PECS, TEACCH, and ABA.

These educative and behavioral methods have been set up in the USA over thirty years ago to enable people with autism to communicate, open up to the world.

Thanks to these tools that are appropriate to their handicap, young autistic children make, in a few months, significant progress.

Unfortunately, psychoanalysts fiercely stand in the way of their establishment in France.

*****

(AS) In the French speaking world the invasion of cognitive behavior techniques is a new invasion.

Recent, but very present today.

Psychoanalysis fights against this invasion

A number of colleagues, in particular Jacques Alain Miller have taken the lead of this struggle, this fight.

Others too in others organisations.

It is a very important fight to keep alive the dimension of subjectivity or the singularities of each subject with respect to the behavioral idea of managing by squares.

*****

(Eric Laurent) You know, it is for psychoanalysis to be this device of disenchantment.

there are hopes that come from biology, it could be marvellous to believe in them!

If one can believe that tomorrow, tomorrow we will have the solutions!

Well, psychoanalysis, as a speech that illuminates all beliefs tries to enable humanity to live without believing too big whims, it’s part of our effort.

So, the dialogue with neurosciences, is not only to inform ourselves of the results and let know that it does not alter our fundamental practice, the orientation of our practice, it is also about trying to make humanity live without having too big hopes in the various good news that are published every day that are intended to keep a rate of good news in an environment that has so little.

*****

(voiceover) In the 80’s, 100% of the French psychiatrists and psychologists were trained in psychoanalysis.

Since the 90s this trend is declining, but they are still near 80% today.

This situation, unique in the world, except Argentina, has tremendous consequences in the care of the handicapped.

(updated 24th January 2012)

Novelty, interest and replicability

2012-01-19T08:53:00.000Z

So at last, your paper is written. It represents the culmination of many years’ work. You think is an important advance for the field. You write it up. You carefully format it for your favoured journal. You grapple with the journal’s portal, tracking down details of recommended reviewers and then sit back. You anticipate a delay of a few weeks before you get reviewer comments. But, no. What’s this? A decision letter within a week: “Unfortunately we receive many more papers than we can publish or indeed review and must make difficult decisions on the basis of novelty and general interest as well as technical correctness.” It’s the publishing equivalent of the grim reaper: a reject without review.

It happens increasingly often, especially if you send work to journals with high impact factors. I’ve been an editor and I know there are difficult decisions to make. It can be kinder to an author to reject immediately if you sense that the paper isn’t going to make it through the review process. One thing you learn as an author is that there’s no point protesting or moaning. You just try again with another journal. I’m confident our paper is important and will get published, and there’s no reason for me to single this journal out for complaint. But this experience has made me reflect more generally on factors affecting publication, and I do think there are things about the system that are problematic.

So, using this blog as my soapbox, there are two points I’d like to make: A little one and a big one. Let’s get the little one out of the way first. It’s simply this: if a journal commonly rejects papers without review, then it shouldn’t be fussy about the format in which a paper is submitted. It’s just silly for busy people to spend time getting the references correctly punctuated, or converting their figures to a specific format, if there’s a strong probability that their paper will be bounced. Let the formatting issues be addressed after the first round of review.

The second point concerns the criteria of “novelty and general interest”. My guess is that our paper was triaged on the novelty criterion because it involved replication. We reported a study that involved measuring electrical brain responses to sounds. We compared these responses in children with developmental language impairments and typically-developing children. The rationale is explained in a blogpost I wrote for the Wellcome Trust.

We’re not the first people to do this kind of research. There have been a few previous studies, but it’s a fair summary to say the literature is messy. I reviewed part of it a few years back and I was shocked at how bad things were. It was virtually impossible to draw any general conclusions from 26 studies. Now these studies are really hard to do. Just recruiting people is difficult and it can take months if not years to get an adequate sample. Then there is the data analysis which is not for the innumerate or faint-hearted. So a huge amount of time and money had gone into these studies, but we didn’t seem to be progressing very far. The reason was simple: you couldn’t generalise because nobody ever attempted to replicate previous research. The studies were focussed on the same big questions, but they differed in important ways. So if they got different results, you couldn’t tell why.

In response to this, part of my research strategy has been to take those studies that look the strongest and attempt to replicate them. So when we found strikingly similar results to a study by Shafer et al (2010) I was excited. The fact that two independent labs on different sides of the world had obtained virtually the same result gave me confidence in the findings. I was able to build on this result to do some novel analyses that helped establish direction of causal influences, and felt we at last we were getting somewhere. But my excitement was clearly not shared by the journal editor, who no doubt felt our findings were not sufficiently novel. I wasn’t particularly surprised by this decision, as this is the way things work. But is the focus on novelty good for science?

The problem is that unless novel findings are replicated, we don’t know which results are solid and reliable. We ought to know: we apply statistical methods with the sole goal of establishing this. But in practice, statistics are seldom used appropriately. People generate complex datasets and then explore different ways of analysing data to find statistically significant results. In electrophysiological studies, there are numerous alternative ways in which data can be analysed, by examining different peaks in a waveform, different methods of identifying peaks, different electrodes, different time windows, and so on. If you do this, it is all too easy for “false positives” to be mistaken as genuine effects (Simmons, Nelson, & Simonsohn, 2011). And the problem is compounded by the “file drawer problem” whereby people don’t publish null results. Such considerations led Ioannidis (2005) to conclude that most published research findings are false.

This is well-recognised in the field of genetics, where it became apparent that most early studies linking genetic variants to phenotypes were spurious (see Flint et al). The reaction, reflected in a recent editorial in Behavior Genetics has been to insist that authors replicate findings of associations between genes and behaviour. So if you want to say something novel, you have to demonstrate the effect in two independent samples.

This is all well and good, but requiring that authors replicate their results is unrealistic in a field where a study takes several years to complete, or involves a rare disorder. You can, however, create an expectation that researchers include a replication of prior work when designing a study, and/or use existing research to generate a priori predictions about expected effects.

It wouldn’t be good for science if journals only published boring replications of things we already knew. Once a finding is established as reliable, then there’s no point in repeating the study. But something that has been demonstrated at least twice in independent samples (replicable) is far more important to science than something that has never been shown before (novel), because the latter is likely to be spurious. I see this as a massive challenge for psychology and neuroscience.

In short, my view is that top journals should reverse their priorities and treat replicability as more important than novelty.

Unfortunately, most scientists don’t bother to attempt replications because they know the work will be hard to publish. We will only reverse that perception if journal editors begin to put emphasis on replicability.

A few individuals are speaking out on this topic. I recommend a blogpost by Brian Knutson who argued, “Replication should be celebrated rather than denigrated.” He suggested that we need a replicability index to complement the H-index. If scientists were rewarded for doing studies that others can replicate, we might see a very different rank ordering of research stars.

I leave the last word to Kent Anderson: “Perhaps we’re measuring the wrong things … Perhaps we should measure how many results have been replicated. Without that, we are pursuing a cacophony of claims, not cultivating a world of harmonious truths.”

Simmons, J., Nelson, L., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant Psychological Science, 22 (11), 1359-1366 DOI: 10.1177/0956797611417632

Time for academics to withdraw free labour

2012-01-07T12:03:00.000Z

Jack is a sheep farmer. He gets some government subsidies, and also works long hours to keep his sheep happy and healthy. When his beasts are ready for slaughter, he offers them to an abattoir. The abattoir is very choosy and may reject Jack’s sheep, which is a disaster for him, as there is no other route to the market. If he is lucky the abattoir will accept the animals, slaughter them and sell them, at a large profit, to the supermarket. Jack does not see any of this money. The populace struggle to afford the price of meat, but the government has no control over this. When Jack feels like a nice piece of lamb, he buys it from the supermarket. Meanwhile, Jack provides his services for free as an inspector of other farmers’ animals.

Crazy story, right? But that’s the model that academic publishing follows. Academics work their butts off to get research funding, often from government. They then do the research and write up and submit it for publication. They run the gauntlet of picky reviewers and editors to get the work accepted for publication. Once it is published, it appears in a journal which is sold on to academic institutions for large profits. Post publication, the academic often has to pay a cost equivalent to several hardback books to get a formatted electronic copy of the article. Meanwhile, the journals justify this by arguing they have extensive costs. But in fact, it is the academic community that does the bulk of the work for free, acting as editors and peer reviewers. Increasingly, they are expected also to do copy editing and graphic design, tasks that were previously undertaken by professional journal staff.

It has taken many years for the torpid academic community to wake up to this ludicrous situation, but things are slowly starting to change. In some fields, academics are starting to take things into their own hands and cut commercial publishers out of the loop, but this still the exception rather than the rule. A more widely adopted innovation has been Open Access publishing. On the one hand, electronic publishing has made it possible for journal papers to be posted online and made freely accessible. On the other, major funders, notably NIH in the USA and the Wellcome Trust in the UK, have insisted that researchers whom they fund must make their published work Open Access. Obviously, something has to give: the publishers are not going to do their work for nothing. But the system does work, with a combination of new journals that are Open Access from the start, and older ones agreeing to make selected articles Open Access, in both cases for a fee. In general, the funders agree to pay the charge.

This week, however, a story broke suggesting that the traditional publishers are trying to fight back and force NIH to backtrack on its Open Access policy. Things hotted up with this post from Michael Eisen who noted that one major publisher, Elsevier, has been lobbying a NY Congresswoman, Carolyn Maloney, to persuade her to support a bill that would limit Open Access publishing. Harvard University gave a detailed response to the bill, which can be found here.

I want my response to this story to go beyond just tut-tutting and shaking my head. Academics do have some power here. We provide the articles for Elsevier journals, and we do a lot of unpaid work reviewing and editing for them. None of us wants to restrict our opportunities for publishing, but these days there are a lot of outlets available. When deciding where to submit a paper, I suspect that most academics, like me, take little notice of who the publisher of a journal is. I focus more on whether the journal has a good editor, my prior experience of publication lags, and whether Open Access is available. But as from now, I shall include publisher in the criteria I adopt, and avoid Elsevier as far as I can. Also, if asked to review for a journal, I’ll check if it is in the Elsevier stable, using this handy website, and if so, I’ll explain why I’m not prepared to review. I suggest that if you are as annoyed as I am by this story, you do likewise, and refuse to engage with Elsevier journals.

Addendum, 10th January 2012

Some people on Twitter have asked if people should be paid for the work they do as author/editor/reviewer. Definitely not. It would just make matters worse, because publishers would factor in these costs and charge even more for journals.

No, I just want a change in the model whereby publishers make enormous and undeserved profits from academics. There are various ways this could be done.

1. The publishers could charge less: currently if you try and download a single journal article, you are charged around £20, even though the production costs are minimal.

2. Retain the current model but remove commercial publishers from the loop, with publication of research limited to learned societies, universities, funders.

3. Retain the current model but make all journals Open Access, with the funder or university paying a one-off publication fee.

4. More radically, move to a system such as arxiv, which I discussed here.

On the whole, academics are an interesting bunch. We’re not all that interested in money, but we are skilled and can produce things of commercial value. It’s a golden opportunity for someone who does want to make money to step in a make a profit. Publishers like Elsevier would have been fine if they hadn’t been so greedy and had charged modest sums for their product. Instead, they pushed costs as high as the market could bear, making huge profits, while at the same time giving authors less and less. (Copy-editors have become an endangered species). Instead of facilitating scientific communication, they have put obstacles in the way. But part of the blame lies with the academic community, who have been far too passive. We should have tackled this years ago before it got out of hand.

Will I still be tweeting in 2013?

2012-01-02T11:06:00.000Z

A new phenomenon has occurred in my Twitter timeline of late. These are tweets from people I don’t follow that are labelled ‘Promoted Tweets’. When I see one, I block the sender. At present, they are rare enough for this to be only mildly irritating. But, as someone who remembers email in the days before spam (well, actually, I remember the days before email, and even the days before personal computers...), I worry that things could change fast.

When discussing Twitter with fellow academics, one thing they always ask is whether it isn’t just another tedious thing that you have to wade through, like email. Email is currently the curse of academics everywhere: in her New Year’s blogpost, Athene Donald noted her resolution to delete spam emails unread first thing in the day, and commentators on her blog clearly resonate to this, as I do. I have been cheerfully telling people that the wonderful thing about Twitter is that you only get messages from people you choose to follow, and it's not at all like email. Other people can’t get at you. Well, they can, a bit, in that they can get into your ‘mentions’ list by mentioning you, but your timeline has always been totally under your control. But the folks at Twitter have other plans, as explained here.

Needless to say, Twitter is a business. It’s not my God-given right to have a free Twitter account. If I engage with the system, I need to play by its rules. But I really am not that addicted. To me, having to fend off people who want my time, money or attention is extremely tedious. Twitter has been a delight precisely because it has been virtually free of such irritants. Make Twitter more like email, and I will just leave. Really.

Publishers, psychological tests and greed

2011-12-30T09:27:00.000Z

There was an intriguing piece in the New England Journal of Medicine this week about a commonly used screening test that indicates if someone is likely to have dementia. The Mini Mental State Examination (MMSE) is widely used throughout the world because it is quick and easy to administer. The test is very simple: you need no equipment, and the eleven items, involving questions to test orientation (e.g. “Where are we?”) and language (e.g. “What is this?” while showing the patient a wristwatch) are reproduced at the end of the original article about the MMSE, which was published in 1975.

The problem is that now the authors have taken steps to license the test, so that it has to be purchased from Psychological Assessment Resources. The cost is modest, $1.23 per test, but nevertheless more than the cost of photocopying one side of paper, which is what people have been doing for years. And of course, if people have to use only officially purchased copies of MMSE there are the additional costs of raising purchase orders, postage, storing packs of forms, and so on.

I’ve got a particular interest in this story, as I have published psychological tests, both off my own bat, and through a test publishing company. I started out in the late 1970s, when I developed a test of children’s comprehension called the Test for Reception of Grammar (TROG). This was more complicated than MMSE in two important respects. It involved lots of brightly coloured pictures as well as a record form, and in order to decide if a child had comprehension problems, I needed to establish how well typical children performed at different ages. The latter process, known as test standardisation, is not a trivial task, because you have to test lots of children to get a good estimate of the range of scores as well as the average score at different ages. This early work was done as part of a study funded by the Medical Research Council (MRC), but I assumed that, if the project worked out, we’d need a test publisher, and so I contacted one. The project involved two big costs. First there was the cost of my time and effort in devising the test, finding reliable people to test hundreds of children nationwide, analyse the results and write the manual. The other cost was printing colour test booklets. I had assumed that the test publisher would be willing to cover this, but they weren’t. They suggested that the MRC should find another several thousand pounds to cover printing. Now this made me cross. The publisher would get for free a fully standardised test that they could sell, no doubt at vast profit, but they wanted someone else to foot the bill for production costs. MRC were actually making quite positive noises about finding the money, but I was irritated enough to explore other options. I found a local printer and learned about the arcane world of different colour separation processes, and came away with a reasonable quote. I also discovered something quite interesting. The costs were all in the initial process of creating plates: the actual printing costs were trivial. This meant that it cost no more to print 1,000 picture books than the 100 copies I needed. And the costs of printing record forms were trivial. I returned to MRC and suggested we left the publisher out of the equation, and they agreed. All proceeded very smoothly, but once the standardisation was completed, I had a problem. There were 900 unused copies of the picture book. I discussed with MRC what we should do. They suggested I could give them away, but this would mean the test would become obsolete as soon as all the copies were used up. In the end, we reached an agreement that I could sell the test in a kind of cottage industry, and share any profits with MRC. And so I did for about the next 15 years. I didn’t bother to copyright the test because it was cheaper to buy it from me than to photocopy it. Nevertheless, I made a nice profit, and took considerable pleasure in telling the publisher to piss off some years later when they approached me expressing interest in TROG.

My next foray into test publishing was with a four-page questionnaire, the Children’s Communication Checklist (CCC). As with TROG, I hadn’t set out to devise an assessment: it came about because there wasn’t anything out there that did what I wanted, so I had to make my own instrument. I published a paper on the CCC in 1998, and listed all the items in an Appendix. I had a problem, though. I was getting busier all the time. For some years I had been paying graduate students to look after TROG sales: the weekly trip to the post office with heavy parcels had become too much of a chore. And every time I moved house, there was the question of what to do with the stock: boxes of picture books and record forms. I also realised that TROG was getting out of date - it’s well recognised that tests need restandardising every ten years or so. I also wanted to develop a test of narrative language. And the CCC was far from perfect and needed revamping and standardising. So I took the big step: I contacted a test publisher. A different one from before. To cut a long story short, they put money into the standardisation, covered production costs, and offered highly professional editorial support. There are now three of my tests in their catalogue.

The upside for me? The tests are actually marketed, so sales are massive compared with my cottage industry activities. And I no longer have to keep a cellar full of cardboard boxes of stock, or concern myself with organising printing and despatching tests, or dealing with complaints from someone whose finger was cut by an injudiciously placed staple. There is a downside, though. The tests are far more expensive. Having done the publishing myself, I know a little secret of the test publishing business: they don’t make their profits from actual test materials such as coloured picture books or IQ test kit. The profits are all in the record forms. These cost peanuts to produce and are sold at a mind-boggling mark-up.

I went into the deal with the publisher with my eyes open. They are a business and I knew they’d make profit from my academic work - just as journal publishers do. I reckon they’ve done more to deserve that profit than most journal publishers, as they put money into test development. That involved taking a gamble that the tests would sell. I have benefited from having a large professional organisation promoting my work, and I do get royalties on the tests. I recycle these back to a relevant charity, and there’s something pleasing about profits from testing children’s language being ploughed back into helping children with language problems.

But my publisher’s situation is very very different from the situation with MMSE. The only people who could plausibly argue they deserve to make money from the test are its authors: the publisher has put no money into development of the test and taken no risks. The authors appear to be claiming that the test items are their intellectual property, and that anyone who attempts to develop a similar test is infringing their copyright. But where did the MMSE items come from? A quick read of the introduction to the 1975 paper gives an answer. Most of them are based a longer assessment described in a 1971 article by Withers and Hinton. It would seem that the main contribution of Folstein et al was to shorten an existing test. I wonder if the British Journal of Psychiatry should go after them for copyright infringement?

Newman, J., & Feldman, R. (2011). Copyright and Open Access at the Bedside New England Journal of Medicine, 365 (26), 2447-2449 DOI: 10.1056/NEJMp1110652

P.S. Another post that includes some information on how MMSE was developed.

You can read more by scrolling down to "The Mini Exam with Maximal Staying Power" on this site from 2007.

NHS research ethics procedures: a modern-day Circumlocution Office

2011-12-18T13:04:00.000Z

In Little Dorritt, Charles Dickens rails against the stifling effects of bureaucracy:

No public business of any kind could possibly be done at any time without the acquiescence of the Circumlocution Office.… the Circumlocution Office was down upon any ill-advised public servant who was going to do it, or who appeared to be by any surprising accident in remote danger of doing it, with a minute, and a memorandum, and a letter of instructions that extinguished him.

Substitute “NHS research ethics procedures” for Circumlocution Office, and “researcher” for public servant, and you have a perfect description of a contemporary problem.

December 2010

My programme grant has been running now for over a year, and it’s time to gird up my loins to tackle NHS ethics. I’ve had plenty of other research to keep me busy, but I’m aware that I’ve been putting off this task after earlier aversive experiences. “Come on,” I tell myself, “you deal with unpleasant and bureaucratic tasks regularly - reviewing grants, responding to reviewer comments, completing your tax return. You really just have to treat this in the same way.”

It starts well enough. I track down a website for the Integrated Research Application System IIRAS). I start to have misgivings when it tells me that it’ll take approximately an hour to work through its e-learning training module. To my mind, any web-based form that requires training in its use needs redesigning. But I bite the bullet and work through the training. Not too bad, I think. I can handle this. I start to complete the form. I’m particularly happy to find little buttons associated with each question that explain what they want you to say. A definite improvement, as in the earlier versions you spent a lot of time trying to work out what the questions were getting at. It also cleverly adapts so that it excludes questions that aren’t relevant to your application. This turns out to be a two-edged sword, as I discover some weeks later. But at present I am progressing and in a cheerful mood.

The process is interrupted by need to travel from Australia to UK, Christmas, snow, massive revision to do to address reviewer comments on a paper, etc.

January 2011

Input more information, design information sheets, consent forms, etc, etc. Still feeling buoyant. The form is virtually complete, except for some information from collaborators and bits that need to be completed by Oxford R&D. I realise we want an information video for kids who can’t read, but it’ll need to be approved, but we don’t want to go to all the trouble and expense of making it before getting approval. Discuss with helpful person from Oxford R&D, who suggests I write a script for approval. I also book in the film crew, shortlist and interview candidates for research assistant posts on the project, send draft to all collaborators for approval, and ask geneticist collaborator for help with some details. Am finding that progress is slower and slower, because navigating the form is so difficult: it displays one page at a time and does not scroll. You can specify a question to go to, but it’s not easy to remember which questions correspond to which numbered item, and so you end up repeatedly printing out the whole form and shuffling through a mountain of paper to find the relevant question. Keeping things consistent is a big headache.

February 2011

Two weeks’ holiday, then enter final details that were sent to me by collaborators and send the whole lot off to R&D.

The dynamic form starts to reveal its diabolic properties when I enter a new collaborator from Cardiff, only to find that the form now pops up with a new question, along the lines of “How will you meet the requirements of the Welsh Language Act 1993?”. I won’t. We’re studying language, and all our tests are in English, so only English speakers will be recruited. Explain that, and hope it works out.

But now it gets seriously worse. I’ve entered lots of clinical colleagues as “NHS Sites”, but it turns out they aren’t sites. They are Patient Identification Centres. I have to delete them all from the form. Well, I think, at least that makes life simpler. But it doesn’t. Because now they aren’t sites any more, new questions pop up. Who will do the patient recruitment, and how will we pay for it? This one is a Catch 22. Previously our research assistants have been supervised by a consultant to go through records to find relevant cases. Some places required that you get honorary NHS status, and that could necessitate fulfilling other requirements. I actually had to get vaccinated for tetanus as part of getting an NHS contract some years ago. They said it was in case I got bitten by a child, something that has not happened to me in 35 years of researching. But I digress. Now, it seems, even a fully vaccinated, child-proofed, police-checked researcher is not allowed to go through medical records to identify cases unless patients have given prior consent. Which, of course, they won’t have, since they don’t know about the study.

“Help!” I say to my lovely clinical colleagues. “What do we do now?”. Well, they have a suggestion. If I can register with something called CLRN, then they can help with patient recruitment. I’m given contact details for a research nurse affiliated with CLRN who soothes my brow and encourages me to go the CLRN route. I have to fill in something called a NIHR CSP Application Form which apparently goes to a body called the “portfolio adoption team” who can decide whether to adopt me and my project. All of these forms want a project start date and duration. I did have early April as notional start date, but that’s beginning to look optimistic.

Late February: comments back from R&D. Have been through application with a fine toothcomb and picked up various things they anticipate won’t be liked by the ethics committee. Impressed with the thoroughness and promptness of the response, and found the people at R&D very helpful over the phone, but my goodness, there is a lot to cope with here:

First, it seems I am still in a muddle about the definition of NHS sites, so have filled in bits wrongly that need to be entered elsewhere. Am also confused about the distinction between an “outcome” and an “outcome measure”.

Then there is the question of whether I need “Site specific forms”. The word “site” is starting to cause autonomic reactions in me. Here’s what I’m told: “Please supply an NHS SSI form for each research site; Please note for Patient Identification Centres (PICs) R&D approval is required but you do not need an SSI form for these provided no research activity takes place on that site – taking consent to take part in the project is a research activity, giving out information on the study/advertising the study is not considered a research activity.”

I also baulk at the suggestion that I should add to the information sheet: “The University has arrangements in place to provide for harm arising from participation in the study for which the University is the Research Sponsor. NHS indemnity operates in respect of the clinical treatment with which you are provided.” Since I don’t understand what this means, I doubt my participants will, and the participants aren’t receiving any clinical treatment. Out of curiosity, I paste these two sentences into a readability index website. It gives the passage a Flesch-Kincaid Grade Level of 22, with readability score of 4 (on a scale of 0 to 100, where 100 is easy). I try to keep my information sheets at maximum 8th grade level, so reword the bits I do understand and delete the bits that seem irrelevant or incomprehensible.

I reluctantly went along with the idea that I should devise an “Assent form” for children. This is like a kiddie consent form, but with easier language, to be signed by both child and researcher. They seem to be a blanket requirement these days, regardless of the level of risk posed by research procedures. I dislike the Assent form because I am not sure what purpose it serves, other than to make children nervous about what they are getting themselves into. It has no legal status, and we can’t gather psychological test data from unco-operative children. Others share my view that this requirement is incoherent and wrong. But I want to do this study, so feel I have no choice. I had a look on the web and NHS guidance sites to look at suggested wordings, and did not like them, so did a modified and simplified version I hoped would be approved. It would be interesting to do some research on Assent forms to see how they are perceived by children.

March 2011

Hooray! By the start of March, I’m ready to submit my forms. Since IRAS is all electronic, I had assumed I would do it with a button press, but that would be too simple. Multiple copies must be sent by snail mail within a specific time frame. There has been serious research on the environmental impact of this. But first there is the question of booking an appointment with an ethics committee. There’s a whole centre devoted to this task, and they have standard questions that they ask you about the nature of the research. I was doing well with these until we got to the question about children. Yes, I was going to do research with children. Ah, well then I couldn’t go to any old ethics committee, I had to go to one with a paediatrician. And, unfortunately, there weren’t any slots on committees in Oxfordshire with paediatricians. But, said the helpful girl on the phone, I could try calling the Oxfordshire people directly and they might be able to book me in. At 12.05 I call the number I’ve been given, only to get an automated message saying the office is only open from 10 to 12. Since the following morning I’m busy (I am trying to do my regular job through all this), despair starts to set in. But I break out of a meeting to call them the next morning. The phone rings. And rings. Back to my meeting. Break out again, repeat experience. Eventually I get through. Person at end of phone takes me through the same list of questions about type of research, and finds a convenient slot with an Oxfordshire committee, which I can make if I move an appointment. Move the appointment. Get called back to say that committee can’t unfortunately take me, because they don’t do proposals with children. Am offered another slot on a day when I have arranged to examine a PhD in London. Next one in Oxford is a month later, well after the proposed start date for the research. Best they can do is to offer me a slot with a Berkshire committee, who do have a paediatrician and are just one hour’s drive away, and which is later than the original slot, but sooner than the Oxford one. I decide to go for it. I then receive a remarkable document with a lot of multicoloured writing, which gives me a booking confirmation number, and a lot of instructions.

This triggers a frantic process because you then have seven days to get all the material delivered to the ethics committee. This may not seem difficult, except that all the information sheets and consent forms need to have little header put on them with the booking number and date, and they also want copies of things like a CV, copies of test forms and suchlike, and worse still, there have to be signatures not just from me but also from R&D, who are in a hospital a couple of miles away up a hill. Unfortunately coincides with a period when my PA is absent, and so I rush around like a demented cockroach getting this all together. I’d not budgeted much time for this bit, as I’d assumed submission would involve pressing a button on my computer and uploading some attachments and my diary was full. Somehow I had to find a couple of hours for fiddling with forms, a trip up the hill for a signature the next day, and a journey to the post office to ensure it would all get delivered on time.

I also needed to get the documents to CLRN. This could be done by email, but that soon bounced back. Once more the critical distinction between sites and centres eluded me, and I was told that I had to submit corrected documents because:

“In Part C, if the only research site is the University of Oxford and the other organisations listed are Participant Identification Centres (PICs), there should be listed under the heading Participant Identification Centre(PIC)Collaborator/Contact immediately below the University of Oxford entry, and not separately.”

So back to the form again to alter this bit. At last it is accepted. But this now triggers new emails, including one from London saying:

“We have been notified that you may be participating in the above study. If the Chief Investigator or Study Coordinator confirms this, Central and East London CLRN will be supporting you locally through the NIHR CSP process and we look forward to working with you on this project.

If this is confirmed, please email all relevant documents to me when you submit your SSI Form through IRAS. The documents you need to submit are listed on the Checklist tab within your SSI Form in IRAS…..etc etc”

The SSI form was one I thought I didn’t have to complete, so I phoned the number given on the email, who said they couldn’t comment and I should ask Oxford, so I asked Oxford, who agreed I didn’t need to do anything.

Meanwhile, there’s yet another form that has popped up that wants to know what training in ethics the researchers have had. Since I haven’t had formal training, I’m told I can either go on a half-day course, or take an on-line course in five modules, each lasting around 45 minutes. I try the online course, but find most of the material is not relevant to me. It starts with pictures of concentration camp victims to emphasise why people need to be protected from reseachers, then goes on to give information focussed on clinical trials. I’m not doing a clinical trial. The quizzes at the end of each module don’t seem designed to check whether you have mastered the subtleties of ethical reasoning, so much as whether you know your way around the bureaucratic maze that is involved in ethical approval, and in particular whether you understand all the acronyms.

April 2011

The six weeks from early March to mid April were joyfully free from communications with ethics people, and normal life resumed. My new staff took up their posts and we made a start on filming for an information DVD for the project, and decided that we would delay the editing stage until after the Berkshire meeting. The day of the committee meeting dawned sunny and bright and I drove off to Berkshire, where I had a perfectly reasonable chat with the ethics committee about the project for about 15 minutes. The Paediatrician was absent. I explained I wanted to assemble the information video, but was told I had to wait until I received a letter documenting changes they’d want me to make. When this arrived, about a week later, they wanted some minor rewording of one sentence. This would be trivial for a written information sheet, but entailed some refilming and careful editing. In addition, the committee raised a point that had not been discussed when I met with them, namely that they were concerned at a statement we had made saying we would give feedback to parents about their children’s language assessment if we found difficulties that had not previously been detected. This, I was told, was an incentive, and I should “soften” the language. This was seriously baffling, as you either tell someone you’ll give them feedback or you don’t. I could not see how to reword it, and I also felt the concern about incentives was just silly. I sent them a copy of a paper on this topic for good measure.

May 2011

Oh frabjous day! At last I receive a letter giving consent for the study to go ahead. I think my troubles are over, and we swing into action with those parts of the project that don’t involve NHS recruitment. But joy is short-lived. I am only just beginning to understand the multifarious ways in which it is possible to Get Things Wrong when dealing with the Circumlocutions Office. I now start to have communications with the CLRN, who want copies of all documentation (including protocol, consent forms, the information video, etc etc - a total of 15 documents) and then tell me:

“The R&D Signature pages uploaded to the doc store on 27th June 2011 do not marry up with the R&D Form uploaded on 15th March 2011”

Requests for new form-filling also come in from the CCRN Portfolio. I’m getting seriously confused about who all these people are, but complete the form anyway.

And, worst still, in August I get a request from TVCLRN for a copy of the letter I sent to Berkshire in which I responded to their initial comments. I had written it at a time when my computer was malfunctioning so it’s not with other correspondence. I spend some time looking on other computers for an electronic copy. It seems that without a copy of this letter, they will not be satisfied. Anyhow, I think this will be simple to sort out, and phone the Berkshire ethics committee to ask if they could please send me a copy of the letter that I had written to them. Amazingly, I’m told that “due to GCP guidelines” the Berkshire ethics committee cannot give me a copy. Stalemate. I can’t actually remember how we dealt with this in the end, as my brain started to succumb to Circumlocution Overload.

The last 6 months

We have a meeting with the clinical geneticists with whom we’re collaborating, and I find that most of them are as confused as I am by the whole process. We discuss the Catch 22 situation whereby we aren’t allowed to help go through files to identify suitable patients because of ethical concerns, which means they have to take time out of their busy schedules to do so. This is where the CLRN is supposed to help, by providing research nurses who can assist, but only if we complete loads more paperwork. And having done this, after months of to-ing and fro-ing with requests for documentation or clarification, one of the CLRN centres has just written this week to say they can’t help us at all because they are a Patient Identification Centre and they need to be a PI, whatever that is. I’m currently trying to unravel what this means, and I think it means that they have to become an NHS Site - which was what I had originally assumed when I started filling in the forms. But in order for them to do so, there are yet more forms to complete.

Meanwhile, in October, I had a request from the UKCRN saying I needed to upload monthly data on patient recruitment in a specific format, and sending me a 35 page manual explaining how to do this. Fortunately, after several exchanges on email, I was able to establish that we did not need to do this, as the hospitals we were dealing with were Patient Identification Centres rather than Sites. But now we have a PIC that wants to become a Site, who knows what new demands will appear?

And then, this week, a new complication. The geneticists who are referring to our study need to check with a child’s GP that it is appropriate to send them the recruitment materials. But an eagle-eyed administrator spotted that this letter “was not an ethically approved form”. I was surprised at this. This is not a letter to a patient; it is a standard communication between NHS professionals. Nevertheless, my R&D contact confirmed that this letter would need approval, and that I’d have to fill in a form for a “substantial amendment”, which would then need to be approved by all the R&D sites as well as the Berkshire ethics committee.

When I expressed my despair about the process on Twitter, I had some comments from ethicists, one of whom said “If you're doing research on ppl then someone has to look after them, no?” Of course, the answer is “yes”, and in fact the project I’m working on does raise important ethical issues. As another commentator pointed out, the problem is not usually with the ethics procedures, and it is true that the IRAS form is much better than its predecessor and guides you through issues that you need to think about and offers good advice. But the whole process has got tangled up in bureaucratic legal issues and most of my problems don’t have anything to do with protecting patients and have everything to do with protecting institutions against remote possibilities of litigation.

Concluding thoughts

1. In the summer, I was contacted by a member of the public who was concerned about the way in which a medical project done at Oxford University was being used to promote unproven diagnostic tests and treatment for a serious medical condition. I recommended that my contact should write to the relevant person dealing with ethics in the University. I was sanguine that this would be taken seriously: here was an allegation of serious infringement of ethical standards and all my dealings with our R&D department indicated they were sticklers for correct procedures. A month or so passed; they didn’t reply to the complainant. I was embarrassed by this and so wrote to point out that a serious complaint had gone uninvestigated. After a further delay we both got a bland reply that did not answer the specific questions that had been raised and just reassured us the matter was being investigated. This just confirms my cynicism about the role of our systems in protecting patients. As Thomas Sowell pointed out: “You will never understand bureaucracies until you understand that for bureaucrats procedure is everything and outcomes are nothing.”

2. The current system is deterring people from doing research. The problem is not with the individuals running the system: they’ve mostly been highly professional, helpful and competent, but they are running a modern Circumlocution Office. I’ve interacted with at least 27 people about my proposal, and that’s not counting the Research Ethics Committee members. I’m a few years off retirement and I’ve already decided that I won’t tangle with NHS Ethics again. I’m in the fortunate position that I can do research studies that don’t involve NHS patients, and I want to spend the time remaining to me engaged in the activity I like, rather than chasing pieces of paper so that someone somewhere can file them, or waiting for someone to agree that an innocuous letter from a Consultant to a GP is ethically acceptable.

3. To end on a positive note: I think there is another way. The default assumption seems to be that all researchers are unscrupulous rogues who’ll go off the rails unless continuously monitored. The system should be revamped as a mechanism for training researchers to be aware of ethical issues and helping them deal with difficult issues. For research procedures that are in common use, one can develop standard protocols that document how things should be done to ensure best practice. On this model, a researcher would indicate that their research would follow protocol X and be trusted to do the research in an ethical fashion. The training would also ensure that researchers would recognise when a study involved ethically complex or controversial aspects that fell outside a protocol, and would be expected to seek advice from the Research Ethics Committee. The training would not revolve around learning acronyms, but would rather challenge people with case studies of ethical dilemmas to ensure that issues such as confidentiality, consent and risk were at the forefront of the researcher’s mind. This is the kind of model we use for people engaged in other activities that could pose risks to others - e.g., medical staff, teachers, car-drivers. Life would come to a standstill if every activity they undertook had to be scrutinised and approved. Instead, we train people to perform to a high standard, and then trust them to get on with it. We need to adopt the same approach to researchers if we are not to stifle research activity with human participants.

Further reading

Kielmann T, Tierney A, Porteous R, Huby G, Sheikh A, & Pinnock H (2007). The Department of Health's research governance framework remains an impediment to multi-centre studies: findings from a national descriptive study. Journal of the Royal Society of Medicine, 100 (5), 234-8 PMID: 17470931

Knowles, R. L., Bull, C., Wren, C., & Dezateux, C. (2011). Ethics, governance and consent in the UK: implications for research into the longer-term outcomes of congenital heart defects. Archives of Disease in Childhood, 96(1), 14-20.

Robinson, L., Drewery, S., Ellershaw, J., Smith, J., Whittle, S., & Murdoch-Eaton, D. (2007). Research governance: impeding both research and teaching? A survey of impact on undergraduate research opportunities. Medical Education, 41(8), 729-736.

Warlow, C. (2005). Over-regulation of clinical research: a threat to public health. Clinical Medicine, 5(1), 33-38.

Wilkinson, M., & Moore, A. (1997). Inducement in research. Bioethics, 11, 374-389.

Pioneering treatment or quackery? How to decide

2011-12-04T08:20:00.001Z

My mother was only slightly older than I am now when she died of emphysema (chronic obstructive pulmonary disease). It’s a progressive condition for which there is no cure, though it can be managed by use of inhalers and oxygen. I am still angry at the discomfort she endured in her last years, as she turned from one alternative practitioner to another. It started with a zealous nutritionist who was a pupil of hers. He had a complicated list of foods she should avoid: I don’t remember much about the details, except that when she was in hospital I protested at the awful meal she’d been given - unadorned pasta and peas - only to be told that this was at her request. Meat, sauces, fats, cheese were all off the menu. My mother was a great cook who enjoyed good food, but she was seriously underweight and the unappetising meals were not helping. In that last year she also tried acupuncture, which she did not enjoy: she told me how it involved lying freezing on a couch having needles prodded into her stick-like body. Homeopathy was another source of hope, and the various remedies stacked up in the kitchen. Strangely enough, spiritual healing was resisted, even though my Uncle Syd was a practitioner. That seemed too implausible for my atheistic mother, whose view was: “If there is a God, why did he make us intelligent enough to question his existence?”

From time to time, friends and relatives of mine have asked my advice about other treatments that are out there. There is, for instance, the Stem Cell Institute in Panama, offering treatment for multiple sclerosis, spinal cord injury, osteoarthritis, rheumatoid arthritis, other autoimmune diseases, autism, and cerebral palsy. Or nutritional therapist Lucille Leader, who has a special interest in supporting patients with Parkinson's Disease, Multiple Sclerosis and Inflammatory Bowel Disease. My mother would surely have been interest in AirEnergy, a “compact machine that creates 'energised air' that feeds every cell in your body with oxygen that it can absorb and use more efficiently”.

Another source of queries are parents of the children with neurodevelopmental disorders who are the focus of my research. If you Google for treatments for dyslexia you are confronted by a plethora of options. There is the Dyslexia Treatment Centre, which offers Neurolinguistic Programming and hypnotherapy to help children with dyslexia, dyspraxia or ADHD. Meanwhile the Dore Programme markets a set of “daily physical exercises that aim to improve balance, co-ordination, concentration and social skills” to help those with dyslexia, dyspraxia, ADHD or Asperger’s syndrome. The Dawson Program offers vibrational kinesiology to correct imbalances in the body’s energy fields. I could go on, and on, and on….

So how on earth can we decide which treatments to trust and which are useless or even fraudulent? There are published lists of warning signs (e.g. ehow Health, Quackwatch), but I wonder how useful they are to the average consumer. For instance, the cartoon by scienceblogs will make skeptics laugh, but I doubt it will be much help for anyone with no science background who is looking for advice. So here’s my twopennyworth. First, a list of things you need to ignore when evaluating a treatment.

1. The sincerity of the practitioner. It’s a mistake to assume all purveyors of ineffective treatments are evil bastards out to make money of the desperate. Many, probably most, believe honestly in what they are doing. The nutritionist who advised my mother was a charming man who did not charge her a penny - but still did her harm by ensuring her last months were spent on an inadequate and boring diet. The problem is if practitioners don’t adopt scientific methods of evalulating treatments they will convince themselves they are doing good, because some people get better anyway, and they’ll attribute the improvement to their method.

2. The professionalism of the website. Some dodgy treatments have very slick marketing. The Dore Treatment, which I regard as of dubious efficacy, had huge success when it first appeared. Its founder, Wyford Dore was a businessman who had no background in neurodevelopmental disorders but knew a great deal about marketing. He ensured that if you typed ‘dyslexia treatment’ into Google his impressive website was the first thing you’d hit.

3. Fancy-looking credentials. These can be misleading if you aren’t an expert - and sometimes even if you are. My bugbear is ‘Fellow the Royal Society of Medicine’, which sounds very impressive - similar to Fellow the Royal Society (which really is impressive). In fact, the threshold for fellowship is pretty low, so much so that fellows are told by the RSM that they should not use FRSM on a curriculum vitae. So when you see this on someone’s list of credentials, it means the opposite of what you think: they are likely to be a charlatan. It’s also worth realising that it’s pretty easy to set up your own organisation and offer your own qualifications. I could set up the Society of Skeptical Quackbusters and offer Fellowship to anyone I choose. The letters FSSQ might look good, but carry no guarantee of anything.

4. Testimonials. There is evidence (reviewed here) that humans trust testimonials far more than facts and figures. It’s a tendency that’s hard to overcome, despite scientific training. I still find myself getting swayed if I hear someone tell me of their positive experience with some new nutritional supplement, and thinking, maybe there’s something in it. Advertisers know this: it’s one thing to say that 9 out of 10 cats prefer KittyMunch, but to make it really effective you need a cute cat going ecstatic over the food bowl. If you are deciding whether to go for a treatment you must force yourself to ignore testimonials. For a start, you don’t even know if they are genuine: anyone who regards sick and desperate people as a business opportunity is quite capable of employing actors to pose as satisfied customers. Second, you are given no information about how typical they are. You might be less impressed by the person telling you their dyslexia was cured if you knew that there were a hundred others who paid for the treatment and got no benefit. And the cancer patients who die after a miracle cure are the ones you won’t hear about.

5. Research articles. Practitioners of alternative treatments are finding that the public is getting better educated, and they may be asked about research evidence. So it’s becoming more common to find a link to ‘research’ on websites advertising treatments. The problem is that all too often this is not what it seems. This was recently illustrated by an analysis of research publications from the Burzynski clinic, which offers the opportunity to participate in expensive trials of cancer treatment. I was interested also to see the research listed on the website of FastForword, a company that markets a computerized intervention for children’s language and literacy problems. Under a long list of Foundational Research articles, they list one of my papers that fails to support their theory that phonological and auditory difficulties have common origins. More generally, the reference list contains articles that are relevant to the theory behind the intervention, but don’t necessarily support it. Few people other than me would know that. And a recent meta-analysis of randomized controlled trials of FastForword is a notable omission from the list of references provided. Overall, this website seems to exemplify a strategy that has previously been adopted in other areas such as climate change, impact of tobacco or sex differences, where you create an impression of a huge mass of scientific evidence, which can only be counteracted if painstakingly unpicked by an expert who knows the literature well enough to evaluate what’s been missed out, as well as what’s in there. It’s similar to what Ben Goldacre has termed ‘referenciness’, or the ‘Gish gallop’ technique of creationists. It’s most dangerous when employed by those who know enough about science to make it look believable. The theory behind FastForword is not unreasonable, but the evidence for it is far less compelling than the website would suggest.

So those are the things that can lull you into a false sense of acceptance. What about the red flags, warning signs that suggest you are dealing with a dodgy enterprise? None of these on its own is foolproof, but where several are present together, beware.

Is there any theory behind the intervention, and if so is it deemed plausible by mainstream scientists? Don’t be impressed by sciency-sounding theories - these are often designed to mislead. Neuroscience terms are often incorporated to give superficial plausibility: I parodied this in my latest novel, with the invention of Neuropositive Nutrition, which is based on links between nutrients, the thalamus and the immune system. I suspect if I set up a website promoting it, I’d soon have customers. Unfortunately, it can be hard to sort the wheat from the chaff, but NHSChoices is good for objective, evidence-based information. Most universities have a communications office that may be able to point you to someone who could indicate whether an intervention has any scientific credibility.
How specific is the treatment? A common feature of dodgy treatments is that they claim to work for a wide variety of conditions. Most effective treatments are rather specific in their mode of action.
Does the practitioner reject conventional treatments? That’s usually a bad sign, especially if there are effective mainstream approaches.
Does the practitioner embrace more than one kind of alternative treatment? I was intriguted when doing my brief research on Fellows of the Royal Society of Medicine to see how alternative interventions tend to cluster together. The same person who is offering chiropractic is often also recommended hypnotherapy, nutritional supplements and homeopathy. Since modern medical advances have all depended on adopting a scientific stance, anyone who adopts a range of methods that don’t have scientific support is likely to be a bad bet.
Are those developing the intervention cautious, and interested in doing proper trials? Do they know what a randomised controlled trial is? If they aren’t doing them, why not? See this book for an accessible explanation of why this is important.
Does it look as though those promoting the intervention are deliberately exploiting people’s gullibility by relying heavily on testimonials? Use of celebrities to promote a product is a technique used by the advertising industry to manipulate people’s judgement. It’s a red flag.
Are costs reasonable? Does the website give you any idea of how much they are, or do you have to phone up for information? (bad sign!). Are people tied in to long-term treatment/payment plans? Are you being asked to pay to take part in a clinical trial? (Very unusual and ethically dubious). Do you get a refund if it doesn’t work? If yes, read the terms and condition very carefully so you understand exactly the circumstances under which you get your money back. For instance, I’ve seen a document from the Dore organisation that promised a money-back guarantee on condition there was ‘no physiological change’. That was interpreted as change on tests of balance and eye movements. These change with age and practice, and don’t necessarily mean a treatment has worked. Failing to improve in reading did not qualify you for the refund.
Can the practitioner answer the question of why mainstream medicine/education has not adopted their methods? If the answer refers to others having competing interests, be very, very suspicious. Remember, mainstream practitioners want to make people better, and anyone who can offer effective treatments is going to be more successful than someone who can’t.

The weird world of US ethics regulation

2011-11-25T08:52:00.001Z

There has been a lot of interest over the past week in the Burzynski Clinic, a US organisation that offers unorthodox treatment to those with cancer. To get up to speed on the backstory see this blogpost by Josephine Jones.

As someone who spends more of my time than I’d like grappling with research ethics committees, there was one aspect of this story that surprised me. According to this blogpost, the clinic is not allowed to offer medical treatment, but is allowed to recruit patients to take part in clinical trials. But this is expensive for participants. The Observer piece that started all the uproar this week described how a family needed to raise £200,000 so that their very sick little girl could undergo Burzynski’s treatment.

I had assumed that this trial hadn’t undergone ethical scrutiny, because I could not see how any committee could agree that it was ethical to charge someone enormous sums of money to take part in a research project in which there was no guarantee of benefit. I suspect that many people would pay up if they felt they’d exhausted all other options. But this doesn’t mean it’s right.

I was surprised, then, to discover that the Burzynski trial had undergone review by an Institutional Review Board (IRB - the US term for an ethics committee). A letter describing the FDA’s review of the relevant IRB is available on the web. It concludes that “the IRB did not adhere to the applicable statutory requirements and FDA regulations governing the protection of human subjects.” There’s a detailed exposition of the failings of the Burzynski Institute IRB, but no mention of fees charged to patients. So I followed a few more links and came to a US government site that described regulatory guidelines for ethics committees, which had a specific section on Charging for Investigational Products. It seems the practice of passing on research costs to research participants is allowed in the US system.

There has been considerable debate in academic circles about the opposite situation, where participants are paid to take part in a study. I know of cases where such payments have been prohibited by an ethics committee on the grounds that they provide ‘inducement’, which is generally regarded as a Bad Thing, though there are convincing counterarguments. But I am having difficulty in tracking down any literature at all on the ethics of requiring participants to pay a fee to take part in research. Presumably this is a much rarer circumstance than cases where participants are paid, because in general people need persuading to take part in research. The only people who are likely to pay large sums to be a research participant are those who are in a vulnerable state, feeling they have nothing to lose. But these are the very people who need protection by ethics committees because it’s all too easy for unscrupulous operators to exploit their desperation. Anyone who doesn’t have approval to charge for a medical treatment could just redescribe their activities as a clinical trial and bypass regulatory controls. Surely this cannot be right.

Your Twitter Profile: The Importance of Not Being Earnest

2011-11-19T17:15:00.001Z

I’m always fascinated by the profiles of people who follow me on Twitter. One of the things I love about Twitter is its ability to link me up with people who I’d never otherwise encounter. It’s great when I find someone from the other side of the world who’s interested in the same things as me. There are, of course, also those who just want to promote their product, and others, like Faringdon Motor Parts and Moaning Myrtle (@toiletmoans) whose interests in my tweets are, frankly, puzzling. But the ones that intrigue me most are the ones with profiles that create an immediate negative impression - or to put it more bluntly, make me just think "Pillock!" (If you need to look that up, you’re not from Essex).

Now language is one of my things - I work on language disorders, and over the years I’ve learned a bit about sociolinguistics - the influence of culture on language use. And that made me realise there were at least two hypotheses that could explain the occasional occurrence of offputting profiles. The first was that I am being followed by genuine pillocks. But the other was that there are cultural differences in what is regarded as an acceptable way of presenting yourself to the world. Maybe a turn of phrase that makes me think "pillock" would make someone else think "cool". And perhaps this is culturally determined.

So what, to my British ear, sets off the pillock detector? The major factor was self-aggrandisement. For instance, someone who describes themselves as "a top intellectual", "highly successful", "award-winning", or "inspirational".

But could this just be a US/UK difference? The British have a total horror of appearing boastful: the basic attitude is that if you are clever/witty/beautiful you should not need to tell people - it should be obvious. Someone who tells you how great they are is transgressing cultural norms. Either they really are great, in which case they are up themselves, as we say in Ilford, or they aren’t, in which case they are a dickhead. When I see a profile that says that someone is "interested in everything, knows nothing", "a lazy pedant", or "procrastinaor extraordinaire", I think of them as a decent sort, and I can be pretty sure they are a Brit. But can this go too far? Many Brits are so anxious to avoid being seen as immodest that they present themselves with a degree of self-deprecation that can be confused by outsiders with false modesty at best, or neurotic depression at worst.

A secondary factor that sets off my negative reactions is syrupy sentiment, as evidenced in phrases such as: "empowering others", "Living my dream", or "I want to share my love". This kind of thing is generally disliked by Brits. I suspect there are two reasons for this. First, in the UK, displays of emotion are usually muted, except in major life-threatening circumstances: so much so that when someone is unabashedly emotional they are treated with suspicion and thought to be insincere. And second, Polyannaish enthusiasm is just uncool. The appropriate take on life’s existential problems is an ironic one.

I was pleased to find my informal impressions backed by by social anthropologist Kate Fox, in her informative and witty book "Watching the English" (Hodder & Stoughton, 2004). Humour, she states, is our "default mode", and most English conversations will involve "banter, teasing, irony, understatement, humorous self-deprecation, mockery or just silliness." (p 61). She goes on to describe the Importance of Not Being Earnest rule: "Seriousness is acceptable, solemnity is prohibited. Sincerity is allowed, earnestness is strictly forbidden. Pomposity and self-importance are outlawed." (p. 62). Fox doesn’t explicitly analyse American discourse in the book, but it is revealing that she states: "the kind of hand-on-heart, gushing earnestness and pompous Bible-thumping solemnity favoured by almost all American politicians would never win a single vote in this country - we watch these speeches on our news programmes with a kind of smugly detached amusement." (p 62).

Anthropologists and linguists have analysed trends such as these in spoken discourse, but I wondered whether they could be revealed in the attenuated context of a Twitter profile. So in an idle moment (well, actually when I was supposed to be doing something else I didn’t want to do) I thought I’d try an informal analysis of my Twitter followers to see if these impressions would be borne out by the data. This is easier said than done, as I could find no simple way to download a list of followers, and so I had to be crafty about using "SaveAs" and "Search and Replace" to actually get a list I could paste into Excel, and when I did that, my triumph was short-lived: I found it’d not saved Location information. At this point, my enthusiasm for the project started to wane - and the task I was supposed to be doing was looking ever more attractive. But, having started, I decided to press on and manually enter location for the first 500 followers. (Fortunately I was able to listen to an episode of the News Quiz while doing this. I started to like all those eggs with no Location recorded). I then hid that column so it would not bias me, and coded the profiles for three features: (a) Gender (male/female/corporate/impossible to tell); (b) Self-promotion: my totally subjective rating of whether the profile triggered the pillock-detector; (c) Syrupy: another subjective judgement of whether the profile contained overly sentimental language. I had intended also to code mentions of cats - I was convinced that there was a British tendency to mention cats in one’s profile, but there were far too few to make analysis feasible. I was a victim of confirmation bias. So were my other intuitions correct? Well, yes and no.

For the analysis I just focused on followers from the US and UK. The first thing to emerge from the analysis was that pillocks were rare in both US and UK - rarer than I would have anticipated. I realised that, like mentions of cats, it’s something I had overestimated, probably because it provoked a reaction in me when it occurred. But, I was pleased to see that nonetheless my instincts were correct: there were 7/97 (7.2%) pillocks in the US sample but only 2/153 (1.3%) in the UK . The sample size is really not adequate, and if I were going to seriously devote myself to sociolinguistics I’d plough on to get a much bigger sample size. But nevertheless, for what it’s worth, this is a statistically significant difference (chi square = 5.97, p = .015 if you really want to know). Syrup followed a similar pattern: again it was rare in both samples, but it was coded for 3/153 of the UK sample compared with 7/97 of the US. I’d coded gender as I had thought this might be a confounding factor, but in fact there were no differences between males and females in either pillocks or syrup. Of course, all these conclusions apply only to my followers, who are bound to be an idiosyncratic subset of people.

My conclusion from all this: we need to be more sensitive to cultural differences in self-expression. Looking over some of the profiles that I categorised as "pillock" I realise that I’m being grossly unfair to their owners. After all, on a Twitter profile, the only information that people have about you comes from the profile - and your tweets. So it really is preposterous for me to react negatively against someone telling me they are an "award-winning author": that should engender my interest and respect. And, because this is a profile, and not a conversation, if they didn’t tell me, I wouldn’t know. And we really ought to cherish rather than mock those who try to bring a bit of love and kindness into the world. But somehow….

I hope that Americans reading this will get some insight into the tortuous mindset of the Brits: if we come across as dysfunctionally insecure losers it’s not that we really are - it’s that we’d rather you thought that of us than that we were boastful.

Vitamins, genes and language

2011-11-13T10:26:00.001Z


Thiamine chloride (source: Wikipedia)

In November 2003, a six-month-old boy was admitted to the emergency department of a children’s hospital in Tel Aviv. He had been vomiting daily for two months, was apathetic, and had not responsed to anti-emetic drugs. The examining doctor noticed something odd about the child’s eye movements and referred him on to the neuro-ophthalmology department. A brain scan failed to detect any tumour. The doctors remembered a case they had seen 18 months earlier, where a 16-year-old girl had presented with episodic vomiting and abnormal eye movements due to vitamin B1 deficiency. They injected the child with thiamine and saw improvement after 36 hours. The vomiting stopped, and over the next six weeks the eye movements gradually normalised. When followed up 18 months later he was judged to be completely normal.

This was not, however, an isolated case. Other babies in Israel were turning up in emergency departments with similar symptoms. Where thiamine deficiency was promptly recognised and treated, outcomes were generally good, but two children died and others were left with seizures and neurological impairment. But why were they thiamine deficient? All were being fed the same kosher, non-dairy infant formula, but it contained thiamine. Or did it? Analysis of samples by the Israeli Ministry of Health revealed that levels of thiamine in this product were barely detectable, and there was an immediate product recall. The manufacturer confirmed that human error had led to thiamine being omitted when the formula had been altered.

The cases who had been hospitalised were just the tip of the iceberg. Up to 1000 infants had been fed the formula. Most of these children had shown no signs of neurological problems. But a recent study reported in Brain describes a remarkable link between this early thiamine deprivation and later language development. Fattal and colleagues studied 59 children who had been fed thiamine-deficient formula for at least one month before the age of 13 months, but who were regarded as neurologically asymptomatic. Children who had birth complications or hearing loss were excluded. The authors stress that the children were selected purely on the basis of their exposure to the deficient formula, and not according to their language abilities. All were attending regular schools. A control group of 35 children was selected from the same health centres, matched on age.

Children were given a range of language tests when they were 5 to 7 years of age. These included measures of sentence comprehension, sentence production, sentence repetition and naming. There were dramatic differences between the two groups of children, with the thiamine-deficient group showing deficits in all these tasks. The authors argued that the profile of performance was identical to that seem in children with a diagnosis of specific language impairment (SLI), with specific problems with certain complex grammatical constructions, and normal performance on a test of conceptual understanding that did not involve any language.

Figure 1 An example of a picture pair used in the comprehension task.

The child is asked to point to the picture that matches a sentence,

such as ‘Tar’e li et ha-yalda she-ha-isha mecayeret’

(Show me the girl that the woman draws). From Fattal et al, 2011.

I have some methodological quibbles with the paper. The authors excluded three control children who did poorly on the syntactic tests because they were outliers - this seems wrong-headed if the aim is to see whether syntactic problems are more common in children with thiamine-deficiency than in those without. The non-language conceptual tests were too easy, with both groups scoring above 95% correct. To convince me that the children had normal abilities they would need to demonstrate no difference between groups on a sensitive test of nonverbal IQ. My own experience of testing children’s grammatical abilities in English is that ability to do tests such as that shown in Figure 1 can be influenced by attention and memory as well as syntactic ability, and so I think we need to rule out other explanations before accepting the linguistic account offered by the authors. I’d also have liked a bit more information about how the control children were recruited, to be certain they were not a ‘supernormal’ group - often a problem with volunteer samples, and something that could have been addressed if a standarized IQ test had been used. But overall, the effects demonstrated by these authors are important, given that there are so few environmental factors known to selectively affect language skills. These results raise a number of questions about children’s language impairments.

The first question that struck me was whether thiamine deficiency might be implicated in other cases outside this rare instance. I have no expertise in this area, but this paper prompted me to seek out other reports. I learned that thiamine deficiency, also known as infantile beriberi, is extremely rare in the developed world, and when it does occur it is usually because an infant is breastfeeding from a mother who is thiamine deficient. It is therefore important to stress that thiamine deficiency is highly unlikely to be implicated in cases of specific language impairment in Western societies. However, a recent paper reported that it is relatively common in Vientiane, Laos, where there are traditional taboos against eating certain foods in the period after giving birth. The researchers suggested that obvious cases with neurological impairments may be the extreme manifestation of a phenomenon that is widespread in milder form. If so, then the Israeli paper suggests that the problem may be even more serious than originally suggested, because there could be longer-term adverse effects on language development in those who are symptom-free in infancy.

The second question concerns the variation in outcomes of thiamine-deficient infants. Why, when several hundred children had been fed the deficient formula, were only some of them severely affected? An obvious possibility is the extent to which infants were fed foods other than the deficient formula. But there may also be genetic differences between children in how efficiently they process thiamine.

This brings us to the third question: could this observed link between thiamine deficiency and language impairment have relevance for genetic studies of language difficulties? Twin and family studies have indicated that specific language impairment is strongly influenced by genes. However, one seldom finds genes that have a major all-or-none effect. Rather, there are genetic risk variants that have a fairly modest and probabilistic impact on language ability.

Robinson Crusoe Island

A recent study by Villanueva et al illustrates this point. They analysed genetic variation in an isolated population on Robinson Crusoe Island, the only inhabited island in the Juan Fernandez Archipelago, 677 km to the west of Chile. At the time of the study there were 633 inhabitants, most of whom were descended from a small number of founder indviduals. This population is of particular interest to geneticists as there is an unusually high rate of specific language impairment. A genome-wide analysis failed to identify any single major gene that distinguished affected from unaffected individuals. However, there was a small region of chromosome 7 where there genetic structure was statistically different between affected and unaffected cases, and which contained genetic variants that had previously been found linked to language impairments in other samples. One of these, TPK1 is involved in the catalysis of the conversion of thiamine to thiamine pyrophosphate. It must be stressed that the genetic association between a thiamine-related genetic variant and language impairment is probabilistic and weak, and far more research will be needed to establish whether it is generalises beyond the rare population studied by Villanueva and colleagues. But this observation points the way to a potential mechanism by which a genetic variant could influence language development.

To sum up: the importance of the study by Fattal and colleagues is two-fold. First, it emphasises the extent to which there can be adverse longer-term consequences of thiamine deficiency in children who may not have obvious symptoms, an observation which may assume importance in cultures where there is inadequate nutrition in breast-feeding mothers. Second, it highlights a role of thiamine in early neurodevelopment, which may prove an important clue to neuroscientists and geneticists investigating risks for language impairment.

References

Fattal I, Friedmann N, & Fattal-Valevski A (2011). The crucial role of thiamine in the development of syntax and lexical retrieval: a study of infantile thiamine deficiency. Brain : a journal of neurology, 134 (Pt 6), 1720-39 PMID: 21558277

Villanueva P, Newbury DF, Jara L, De Barbieri Z, Mirza G, Palomino HM, Fernández MA, Cazier JB, Monaco AP, & Palomino H (2011). Genome-wide analysis of genetic susceptibility to language impairment in an isolated Chilean population. European journal of human genetics : EJHG, 19 (6), 687-95 PMID: 21248734

A message to the world

2011-10-31T14:54:00.000Z

from a teenager with language difficulties

Accentuate the negative

2011-10-26T21:11:00.000+01:00

Suppose you run a study to compare two groups of children: say a dyslexic group and a control group. Your favourite theory predicts a difference in auditory perception, but you find no difference between the groups. What to do? You may feel a further study is needed: perhaps there were floor or ceiling effects that masked true differences. Maybe you need more participants to detect a small effect. But what if you can’t find flaws in the study and decide to publish the result? You’re likely to hit problems. Quite simply, null results are much harder to publish than positive findings. In effect, you are telling the world “Here’s an interesting theory that could explain dyslexia, but it’s wrong.” It’s not exactly an inspirational message, unless the theory is so prominent and well-accepted that the null finding is surprising. And if that is the case, then it’s unlikely that your single study is going to be convincing enough to topple the status quo. It has been recognised for years that this “file drawer problem” leads to distortion of the research literature, creating an impression that positive results are far more robust than they really are (Rosenthal, 1979).

The medical profession has become aware of the issue and it’s now becoming common practice for clinical trials to be registered before a study commences, and for journals to undertake to publish the results of methodologically strong studies regardless of outcome. In the past couple of years, two early-intervention studies with null results have been published, on autism (Green et al, 2010) and late talkers (Wake et al, 2011). Neither study creates a feel-good sensation: it’s disappointing that so much effort and good intentions failed to make a difference. But it’s important to know that, to avoid raising false hopes and wasting scarce resources on things that aren’t effective. Yet it’s unlikely that either study would have found space in a high-impact journal in the days before trial registration.

Registration can also exert an important influence in cases where conflict of interest or other factors make researchers reluctant to publish null results. For instance, in 2007, Cylharova et al published a study relating membrane fatty acid levels to dyslexia in adults. This research group has a particular interest in fatty acids and neurodevelopmental disabilities, and the senior author has written a book on this topic. The researchers argued that the balance of omega 3 and omega 6 fatty acids differed between dyslexics and non-dyslexics, and concluded: “To gain a more precise understanding of the effects of omega-3 HUFA treatment, the results of this study need to be confirmed by blood biochemical analysis before and after supplementation”. They further stated that a randomised controlled trial was underway. Yet four years later, no results have been published and requests for information about the findings are met with silence. If the trial had been registered, the authors would have been required to report the results, or explain why they could not do so.

Advance registration of research is not a feasible option for most areas of psychology, so what steps can we take to reduce publication bias? Many years ago a wise journal editor told me that publication decisions should be based on evaluation of just the Introduction and Methods sections of a paper: if an interesting hypothesis had been identified, and the methods were appropriate to test it, then the paper should be published, regardless of the results.

People often respond to this idea saying that it would just mean the literature would be full of boring stuff. But remember, I'm not suggesting that any old rubbish should get published: there has to be a good case for doing the study made in the Introduction, and the Methods have to be strong. Also, some kinds of boring results are important: miminally, publication of a null result may save some hapless graduate student from spending three years trying to demonstrate an effect that’s not there. Estimates of effect sizes in meta-analyses are compromised if only positive findings get reported. More seriously, if we are talking about research with clinical implications, then over-estimation of effects can lead to inappropriate interventions being adopted.

Things are slowly changing and it’s getting easier to publish null results. The advent of electronic journals has made a big difference because there is no longer such pressure on page space. The electronic journal PLOS One adopts a publication policy that is pretty close to that proposed by the wise editor: they state they will publish all papers that are technically sound. So my advice to those of you who have null data from well-designed experiments languishing in that file drawer: get your findings out there in the public domain.

References

Cyhlarova, E., Bell, J., Dick, J., MacKinlay, E., Stein, J., & Richardson, A. (2007). Membrane fatty acids, reading and spelling in dyslexic and non-dyslexic adults European Neuropsychopharmacology, 17 (2), 116-121 DOI: 10.1016/j.euroneuro.2006.07.003

Green, J., Charman, T., McConachie, H., Aldred, C., Slonims, V., Howlin, P., Le Couteur, A., Leadbitter, K., Hudry, K., Byford, S., Barrett, B., Temple, K., Macdonald, W., & Pickles, A. (2010). Parent-mediated communication-focused treatment in children with autism (PACT): a randomised controlled trial The Lancet, 375 (9732), 2152-2160 DOI: 10.1016/S0140-6736(10)60587-9

Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86 (3), 638-641 DOI: 10.1037/0033-2909.86.3.638

Wake M, Tobin S, Girolametto L, Ukoumunne OC, Gold L, Levickis P, Sheehan J, Goldfeld S, & Reilly S (2011). Outcomes of population based language promotion for slow to talk toddlers at ages 2 and 3 years: Let's Learn Language cluster randomised controlled trial. BMJ (Clinical research ed.), 343 PMID: 21852344

Lies, damned lies, and spin

2011-10-15T08:06:00.003+01:00

©www.cartoonstock.com

The Department for Education (DfE) issued a press report this week entitled “England's 15-year-olds' reading is more than a year behind the best”. The conclusions were taken from analysis of data from the PISA 2009 study, an OECD survey of 15-year-olds in the principal industrialised countries.

The DfE report paints a dire picture: “GCSE pupils' reading is more than a year behind the standard of their peers in Shanghai, Korea and Finland….Fifteen-year-olds in England are also at least six months behind those in Hong Kong, Singapore, Canada, New Zealand, Japan and Australia, according to the Department for Education's (DfE) analysis of the OECD's 2009 Programme for International Student Assessment (PISA) study.” The report goes on to talk of England slipping behind other nations in reading.

Schools Minister Nick Gibb is quoted as saying: “The gulf between our 15-year-olds' reading abilities and those from other countries is stark – a gap that starts to open in the very first few years of a child's education.”

I started to smell a rat when I looked at a chart in the report, entitled “Attainment gap between England and the countries performing significantly better than England” (my emphasis). This seemed an odd kind of chart to provide if one wanted to evaluate how England is doing compared to other countries. So I turned to the report provided by the people who did the survey.

Here are some salient points taken verbatim from their summary on reading:

Twelve countries had mean scores for reading which were significantly higher than that of England. In 14 countries the difference in mean scores from that in England was not statistically significant. Thirty-eight countries had mean scores that were significantly lower than England.
The mean score for reading in England was slightly above the OECD average but this difference was not statistically significant.
England’s performance in 2009 does not differ greatly from that in the last PISA survey in 2006.

There is, of course, no problem with aiming high and wanting our children to be among the top achievers in the world. But that’s no excuse for the DfE's mendacious manipulation of information.

Reference

Bradshaw, J., Ager, R., Burge, B. and Wheater, R. (2010). PISA 2009: Achievement of 15-Year-Olds in England. Slough: NFER.

The joys of inventing data

2011-10-05T08:56:00.000+01:00

Have I gone over to the dark side? Cracked under pressure from the REF to resort to fabrication of results to secure that elusive Nature paper? Or had my brain addled by so many requests for information from ethics committees that I’ve just decided that its easier to be unethical? Well readers will be reassured to hear that none of these things is true. What I have to say concerns the benefits of made-up data for helping understand how to analyse real data.

In my field of experimental psychology, students get a thorough grounding in statistics and learn how to apply various methods for testing whether groups differ from one another, whether variables are associated and so on. But what they typically don’t get is any instruction in how to simulate datasets. This may be a historical hangover. When I first started out in the field, people didn’t have their own computers, and if you wanted to do an analysis you either laboriously assembled a set of instructions in Fortran which were punched onto cards and run on a mainframe computer (overnight if you were lucky), or you did the sums on a pocket calculator. Data simulation was just unfeasible for most people. Over the years, the landscape has changed beyond recognition and there are now windows-based applications that allow one to do complex multivariate statistics at the press of a button. There is a danger, however, which is that people do analyses without understanding them. And one of the biggest problems of all is a tendency to apply statistical analyses post hoc. You can tell people over and over that this is a Bad Thing (see Gould and Hardin, 2003) but they just don’t get it. A little simulation exercise can be worth a thousand words.

So here’s an illustration. Suppose we’ve got two groups each of 10 people, let’s say left-handers and right-handers. And we’ve given them a battery of 20 cognitive tests. When we scrutinise the results, we find that they don’t differ on most of the measures, but there’s a test of mathematical skill on which the left-handers outperform the right-handers. We do a t-test and are delighted to find that on this measure, the difference between groups is significant at the .05 level, so we write up a paper entitled "Left-handed advantage for mathematical skills" and submit it to a learned journal, not mentioning the other 19 tests. After all, they weren’t very interesting. Sounds OK? Well, it isn’t. We have fallen into the trap of using statistical methods that are valid for testing a hypothesis that is specified a priori in a situation where the hypothesis only emerged after scrutinising the data.

Let’s generate some data. Most people have access to Microsoft Excel, which is perfect for simple simulations. In row 1 we put our column labels, which are group, var1, var2, …. var 20.

In column A, we then have ten zeroes followed by ten ones, indicating group identity. We then use random numbers to complete the table. The simplest way to do this is to just type in each cell:

=RAND()

This generates a random number between 0 and 1.

A more sophisticated option is to generate a random z-score. This creates random numbers that meet the assumption of many statistical tests that data are normally distributed. You do this by typing:

=NORMSINV(RAND())

At the foot of each column you can compute the mean and standard deviation for each group, and Excel automatically computes a p-value based on the t-test for comparing the groups with a command such as:

=TTEST(B2:B11,B12:B22,2,2)

See this site if you need an explanation of this formula.

So the formulae in the first three columns look like this (rows 4-20 are hidden):

Copy this formula across all columns. I added conditional formatting to row 27 so that ‘significant’ p-values are highlighted in yellow (and it just so happens with this example that the generated data gave a p-value less than .05 for column C).

Every time you type anything at all on the sheet, all the random numbers are updated: I’ve just added a row called ‘thisrun’ and typing any number in cell B29 will re-run the simulation. This provides a simple way of generating a series of simulations and seeing when p-values fall below .05. On some runs, all the t-tests are nonsignificant, but you’ll quickly see that on many runs one or more p-values are below .05. In fact, on average, across numerous runs, the average number of significant values is going to be one because we have twenty columns, and 1/20 = .05. That’s what p < .05 means! If this doesn’t convince you of the importance of specifying your hypothesis in advance, rather than selecting data for analysis post hoc, nothing will.

This is a very simple example, but you can extend the approach to much more complicated analytic methods. It gets challenging in Excel if you want to generate correlated variables, though if you type a correlation coefficient in cell A1, and have a random number in column B, and copy this formula down from cell C2, then columns B and C will be correlated by the value in cell A1:

=B2*A$1+NORMSINV(RAND())*SQRT(1-A$1^2)

NB, you won’t get the exact correlation on each run: the precision will increase with the number of rows you simulate.

Other applications, such as Matlab or R, allow you to generate correlated data more easily. There are examples of simulating multivariate normal datasets in R in my blog on twin methods.

Simulation can be used not just for exploring a whole host of issues around statistical methods. For instance, you can simulate data to see how sample size affects results, or how results change if you fail to meet assumptions of a method. But overall, my message is that data simulation is a simple and informative approach to gaining understanding of statistical analysis. It should be used much more widely in training students.

Reference

Good, P. I., & Hardin, J. W. (2003). Common errors in statistics (and how to avoid them). Hoboken, NJ: Wiley.

How to become a celebrity scientific expert

2011-09-12T06:19:00.002+01:00

Maybe you’re tired of grotting away at the lab bench. Or finding it hard to get a tenured job. Perhaps your last paper was rejected and you haven’t the spirit to fight back. Do not despair. There is an alternative. The media are always on the look-out for a scientist who will fearlessly speak out and generate newsworthy stories. You can gain kudos as an expert, even if if you haven't got much of a track record in the subject, by following a few simple rules.

Rule #1. Establish your credentials. You need to have lots of letters after your name. It doesn’t really matter what they mean, so long as they sound impressive. It’s also good to be a fellow of some kind of Royal Society. Some of these are rather snooty and appoint fellows by an exclusive election process, but it’s a little known fact that others require little more than a minimal indication of academic standing and will admit you to the fellowship provided you fill in a form and agree to pay an annual subscription. So sign up as a Fellow of the Royal Society of Medicine, and keep good company with a range of naturopaths, homeopaths and chiropracters who have discovered this easy route to eminence. The really nice thing is that even academics can be hoodwinked by this one.

Rule #2. Find a controversial topic. This is key. You have to be willing to take a definite position on something that people have strong views about. A scare story is good - we’ve all been doing X for years but it could damage us. Finding someone to blame is also good - people who do Y are feckless. And the buzz word of the decade is neuroscience, so if you can work that in, success is guaranteed. If you're short of ideas, the list of the right might help inspire you. A recent article in the Biologist hits the spot with “The biological effects of day care”, managing to get us worried about an everyday activity, blame working mothers, and get in a neuro message all at once. It's even spiced up with a bit of conspiracy theory: experts know that day care is bad for children’s brains but nobody is allowed to speak out because it is too politically sensitive. This presses so many buttons that few journalists could resist the story.

Rule #3. Specify a causal chain. As we shall see when it comes to assembling evidence, it is particularly useful to have a causal chain with several steps. For instance:

The point here is that if you can usually find at least some studies that provide evidence for bits of the causal chain. Although it may be inconvenient if, as in this case, studies looking for a link between A and D fail to come up with clear evidence (Lucas-Thompson et al, 2010), you can rely on two things: first, few readers will be familiar with the research literature, so they will only know as much as you tell them. And second, step C, brain abnormality, is highly salient and once you start talking about that, it will distract attention from the other levels of description.

Rule #4. Avoid rigorous peer review. You don’t want to have your views critiqued by someone who knows the literature, or checks your sources. Writing books is a safe bet for avoiding pre-publication scientific critique. As far as journals go, the Biologist is ideal. This publication for the members of the Society of Biology claims to be peer-reviewed, but, as we shall see, the review process is far from rigorous.

Rule #5. Assemble supportive evidence. Note, it is important not to present all relevant studies, just those with findings that can be fitted into the causal chain.

The author of the paper in question, Dr Aric Sigman, presents us with so much positive evidence that he manages to give the impression that the whole casual chain has been validated. He starts by mentioning studies that investigated the link between A and B and find that salivary cortisol is increased in the afternoons in children attending day care. This is a product of the hypothalamo-pituitary-adrenal (HPA) axis, which is elevated in response to stress. This result is well-established, both from studies comparing groups of children who do and don’t attend day care, and from comparing the same children on days when they stay at home or go to day care. This is a potentially concerning finding, if it can be shown that there are consequences for children’s learning and behaviour. As Sigman notes “Of central concern is that the routine stress experienced at day care could cause permanent changes in the child’s neuroendocrine networks, with long-term consequences for their mental and physical health as adults.” (p. 30). The article that he cites does discuss this issue, but notes the complexity of causal relationships and cautions against assuming that the cortisol elevation is harmful. In fact the authors draw attention to an animal study suggesting a very different conclusion:

“As in the work on cortisol responses to fullday child care, these separations in squirrel monkey infants produced marked and repeated activations of the HPA axis. However, followed into the late juvenile and early adult age, animals exposed to this form of early life stress were found to be less fearful, to produce lower rather than higher cortisol responses to stressors, and to show more optimal development of prefrontal regulatory brain circuits; consistent with these findings, they also performed better on tests of executive functioning. Thus, at least for this animal model, repeated separation stress early in life fostered a form of resilience.” Gunnar et al (2010) (my emphasis)

Sigman avoids mentioning any of this and turns instead to look at the links between cortisol levels and ill health, starting with studies that show a link between cortisol and cardiovascular disease. He does not explain that these were done on people aged over 65 years, but rather implants in people’s minds the notion that this is relevant for his arguments about daycare in toddlers. Next comes the serious stuff: research linking elevated cortisol to the brain. We are told that “Cortisol is considered neurotoxic and has a global impact on cerebral size (e.g. McEwen 2007; Sheline 2003).” (p. 29). Again, we are left with the distinct impression that children who attend day care will have small brains, but to find out what actually is meant here we need to read the cited articles. When we do, we find that McEwen (2007) is a thorough research review of the physiology and neurobiology of stress and adaptation that nowhere mentions the terms ‘neurotoxic’, ‘global’ or ‘cerebral size’. Rather, in this article McEwen develops a complex theory that considers both positive and negative impact of stress. It actually has a section subtitled “Protection and damage: the two sides of the response to stressors” which discusses animal studies demonstrating how elevated cortisol can either improve or interfere with brain function, depending on context. Neurotoxicity does feature in the other article, by Sheline (2003), but this is concerned with mood disorders in adults, and discusses effects of hypercortisolemia, a condition where there is chronic elevation of cortisol, rather than a temporary increase at specific times of day. Subsequent citations are to papers that considered the role of cortisol in psychiatric disorders such as anxiety and depression: note that now the evidence is focused on a link between cortisol and adult psychiatric disorders in the opposite direction (from D to B), yet it is presented in the context of discussing consequences of high cortisol in children who attended day care. Sigman further states: “a higher cortisol awakening curve may be a biological marker for an underlying disposition towards developing depressive and anxiety disorders” (p. 30), even though the studies of toddlers attending day care show a cortisol response that develops through the day, rather than a chronically raised level: see Vermeer and van IJzendoorn (2006) for a well-balanced discussion of such evidence.

It would be tedious to wade through every cited article, but it’s worth considering just one more example. We are told “In human grey matter, the quality of a mother’s care in early childhood is thought to alter the size of the hippocampus (Buss et al, 2007)” (p. 30). Erm no. Buss et al clearly stated there were no differences in left or right hippocampal volume between those categorised as having high or low maternal care. What they did find was a complicated interaction between birth weight, gender and maternal care, such that birth weight predicted hippocampal volume only in female subjects reporting low maternal care. But even this limited result doesn’t support Sigman’s case. We are talking about ‘low maternal care’, not ‘attendance at daycare’. And guess what? When we look at ‘low maternal care’ we find it measured from a self-report questionnaire, where low care is partly identified in terms of maternal overprotection. Those categorised this way were more likely to have endorsed items describing their mother in such terms as:

Tended to baby me
Tried to make me feel dependent on her
Felt I could not look after myself unless she was around
Was overprotective of me

I’m not an expert in the neurobiology of stress. I can track down articles that appear to have been cited by Sigman (the reference list is behind a paywall) and see where he's given a misleading account, but what I don’t know is how much relevant literature has been omitted. This, of course, is what the celebrity scientist can rely on: there's only a handful of people who both have the expertise in the area, and are obsessive enough to trawl through your writing (if they can access it) and challenge any misleading statements.

Rule #6. Anticipate criticism but don't let it worry you. People who actually do research in the area you are reviewing may get irritated, but most scientists in the field wouldn’t bother on the grounds that they don't know who you are, and aren’t interested in pursuing academic debates outside the domain of mainstream journals. The worst you may get is a few nerdy bloggers such as Gimpy or Mind Hacks criticising you for lack of scholarship, sensationalism and cherrypicking of evidence. Or, if you're really unlucky, you might be up against Ben Goldacre on Newsnight. But meanwhile, your purpose as celebrity scientist has been achieved: your views are all over the media.

References

Buss, C., Lord, C., Wadiwalla, M., Hellhammer, D. H., Lupien, S. J., Meaney, M. J., et al. (2007). Maternal care modulates the relationship between prenatal risk and hippocampal volume in women but not in men. Journal of Neuroscience, 27(10), 2592-2595. dx.doi.org/10.1523/JNEUROSCI.3252-06.2007

Gunnar MR, Kryzer E, Van Ryzin MJ, & Phillips DA (2010). The rise in cortisol in family day care: associations with aspects of care quality, child behavior, and child sex. Child Development, 81 (3), 851-69. PMID: 20573109

Lucas-Thompson, R. G., Goldberg, W. A., & Prause, J. A. (2010). Maternal work early in the lives of children and its distal associations with achievement and behavior problems: A meta-analysis. Psychological Bulletin, 136(6), 915-942. DOI: 10.1037/a0020875
McEwen, B. S. (2007). Physiology and neurobiology of stress and adaptation: Central role of the brain. Physiological Reviews, 87(3), 873-904.doi: 10.1152/physrev.00041.2006 (Open Access)

Sigman, A. (2011). Mother superior? The biological effects of day care. The Biologist, 5 (3), 29-32.

Vermeer, H. J. & van IJzendoorn, M. H. (2006). Children's elevated cortisol levels at daycare: A review and meta-analysis. Early Childhood Research Quarterly, 21(3), 390-401.doi 10.1016/j.ecresq.2006.07.004

Early intervention: What's not to like?

2011-09-01T12:16:00.002+01:00

If a child has language problems, when would be the best age to intervene? At 18 months of age, when they’re just at the outset of learning language, or at five years, when they’re in school? Most people would say this is a no-brainer, with early intervention being preferred on two counts:

There are all kinds of secondary consequences of language difficulties: effects on self-esteem, educational outcomes and social interactions. Potentially, early intervention can avoid these.
It is easier to influence the course of development while the brain is still plastic. An analogy can be made with vision, where it is well-recognised that amblyopia (or "lazy eye") needs to be corrected early in life, because otherwise visual pathways in the brain do not develop normally, and the potential for good vision in the lazy eye is lost.

Currently, interest by policy-makers in early intervention has focussed mainly on children’s social and emotional outcomes, with a report by MP Graham Allen emphasising the benefits, not just for children’s outcomes, but also in economic terms. The argument is that by preventing problems from developing, we have the potential to save millions of pounds that would otherwise be spent in dealing with problems that manifest later in childhood.

The Allen report does not say much about children’s language development, but similar arguments are often made, and in some areas of the country, speech and language therapy services put most of their resources into intervention with preschoolers.

There is, however, a problem with early intervention that is easily overlooked, but which is well-documented in the case of children’s language problems. This is the phenomenon of the "late bloomer". Quite simply, the earlier you identify children’s language difficulties, the higher the proportion of cases will prove to be "false positives" who spontaneously move into the normal range without any intervention. We’ve known about this phenomenon for many years: For instance, a study conducted by Fischel et al in 1989 followed 26 two-year-olds recruited because their parents reported that they understood complete sentences but could say only a few words. Five months after initial assessment, one third still had problems, one third had made some improvement, and one third were in the normal range. Another study by Thal et al in 1991 followed ten children who scored in the bottom 10% for expressive vocabulary at the age of 18 to 29 months. One year after initial assessment, six had caught up, whereas the remaining four still had delayed language. These early small-scale studies have since been confirmed by much larger population-based studies in the Netherlands and Australia.

The late-bloomer phenomenon was neatly demonstrated in a study just published in the British Medical Journal by an Australian team headed by paediatrician Prof Melissa Wake and speech pathologist Prof Sheena Reilly. They recruited children from a large population-based study, where parents were asked to complete a Sure Start vocabulary screening measure when their child was 18 months of age, as well as a child behaviour checklist. Around 20 per cent of children were reported as having no or very limited spoken words. 301 of these children were randomly allocated to intervention or control groups. The intervention, "Let's Learn Language", was based on a widely-used approach where parents are trained to adopt strategies to enhance communicative interactions with their child. The children were then given a detailed assessment at two years of age, and again at three years. Results were striking: there were no hints of any difference between children in the intervention group and control group on any language or behavioural measures, either at 2 years or at 3 years.

The study authors noted various strengths and weaknesses of their study. Among these they discussed the possibility that the intensity of the intervention (six weekly sessions, each lasting 2 hours) may not have been enough. But they went on to note that “the normal mean language and vocabulary scores achieved by both intervention and control children by age 3 years suggest that natural resolution, rather than our intervention’s intensity being too low, explains the null findings.”

They then point out the sobering conclusion to be drawn: quite simply, if you intervene with children who are likely to improve spontaneously, there will considerable waste of government’s and families’ resources.

Does this mean we should give up on early intervention? No. But it does mean that we need to target such intervention much more carefully. At present, one of the big questions for those of us investigating late talkers is to find characteristics that will allow us to identify those children who won’t make spontaneous progress. This has proved to be surprisingly difficult.

Another important message applies to intervention studies more generally. If you provide an intervention for a condition that spontaneously improves, it is easy to become convinced that you’ve been effective. Parents were very positive about the intervention program. There was remarkably good attendance, and when asked to rate specific features of the program and its effects, around three quarters of the parents gave positive responses. This may explain why both parents and professionals find it hard to believe such interventions have no impact: they do see improvement. Only if you do a properly controlled trial will the lack of effect become apparent, not because treated children don’t improve, but rather because the control group gets better as well.

Reference: (Open Access) :-)
Wake M, Tobin S, Girolametto L, Ukoumunne OC, Gold L, Levickis P, Sheehan J, Goldfeld S, & Reilly S (2011). Outcomes of population based language promotion for slow to talk toddlers at ages 2 and 3 years: Let's Learn Language cluster randomised controlled trial. BMJ (Clinical research ed.), 343 PMID: 21852344

So you want to be a research assistant? Advice for psychologists

2011-08-25T12:21:00.001+01:00

The dire state of the academic jobs market was brought home to me recently. I’d advertised for someone to act as a graduate research assistant/co-ordinator. This kind of post is a good choice for a junior person who wants to gain experience before applying for clinical or educational psychology training, or while considering whether to do a doctorate. Normally I get around 30-40 applicants for this kind of job. This time it was 123. This, apparently, is nothing. These days, for psychology assistant jobs, which act as a gateway to oversubscribed clinical psychology doctorate programmes, the number of applicants can run into the hundreds.

One thing that strikes me is how little insight many applicants have into what happens to their job application. I hope that this post, explaining the process from the employer's perspective, might help aspiring job-seekers improve their chances of getting to interview.

With over 120 applications to process, if I allowed only two minutes for each application, it’d take me four hours to shortlist. Of course, that’s not how it works. There has to be an initial triage procedure where the selection panel views the applications looking for reasons not to shortlist. We were able to exclude around ¾ of the applications on the basis of a fairly brief scan. But we then had to select a shortlist of five from the remainder. This is done on the basis of a careful re-reading of those applications that survive triage.

So how do you get past this double hurdle and avoid initial triage, and then make it to the shortlist? Well, here are some tips. They seem very obvious and simple, but worth stating, as many of the applications we received didn’t seem aware of them.

Follow the instructions for job applicants, and read the further particulars. I gather that there are some careers advisors who recommend candidates should send their application direct to the principal investigator, rather than via administration, because it will get noticed. It will indeed, but it will create the impression that you are incapable of reading instructions.
Specify how you meet the selection criteria. Our university bends over backwards to operate a fair and transparent recruitment policy. We need to be able to demonstrate that our decisions are based on the selection criteria in the job advert, and not on some idiosyncratic prejudice. The ideal applicant lists the selection criteria in the same order that they appear in the job description and briefly explains how they meet them. It makes the job of the selection panel much, much easier, and they will give you credit for being both intelligent and considerate.
Don’t apply if you don’t meet the essential selection criteria. So, if the job requires you to drive, then don’t apply if you don’t have a driving licence (or a chauffeur). When I was young and naïve, I assumed people wouldn’t apply for a job if they didn’t meet the criteria, and ended up appointing a non-driver to a job that involved travelling to remote locations with heavy equipment. It is not a mistake I’ll make again.
Don’t assume anything is obvious. To continue with the example above, if the job involves driving and you don’t mention that you can drive, the person evaluating your application won’t know whether you’ve forgotten to tell them, or if you are avoiding mentioning this because you can’t drive. Either way, it’s bad news for your application, and in the current market, it’ll go on the ‘no’ pile.
Don’t send a standard application that is appropriate for any job. It’s key to include a cover letter or personal statement that indicates that you have read the further particulars for this specific post. Use Google to find out more about the post/employer. On the other hand, the employer really doesn’t want or need to be told about the subject matter of the research - once I had the equivalent of a short undergraduate essay, complete with references, included in an application, and though it demonstrated keeness, it was complete overkill.
Read through your application before you submit it. I’ve had applicants who describe how enthusiastic they are about the prospect of working, not in my institution, but in another university. I’ve had applications where entire paragraphs were duplicated. A melange of fonts changing mid-paragraph, or even mid-sentence, creates a poor impression.
Run the cover letter/personal statement through a spell checker, and check the English. Anyone working for me will be sending letters and information sheets out to the general public on my behalf. It creates a bad impression if there are errors, and so you’ve a very high chance of getting on the ‘no’ pile if you make mistakes on an important document like a job application.
Be honest. If there’s something unusual about your application, explain it. I have, for instance, shortlisted a person who’d had a prolonged period of sick leave, but who gave a clear and honest explanation of the situation and was able to offer reassurance about ability to do the job.
Be concise, but not too concise. The cover letter/personal statement should cover all the selection criteria, but avoid wordiness. One to two single-spaced pages is about right.

And if you get to interview? Well, this blog post has some useful hints:

10 Psychological Techniques to Help You Get a New Job

But what if you follow all my advice and still fail to get to interview? Alas, given the massive mismatch between the number of bright, talented people and the number of jobs on offer, many good candidates are bound to miss out. It certainly doesn’t mean you are unemployable. But try this exercise: look at the selection criteria and your application, and pretend you are the employer, not the candidate: An employer with a huge stack of applications and limited time. What do you think looks good, and what are the weaker points? Can you gain further experience so that the weaker points can be remedied in future job applications? Or maybe the weaknesses include something like a poor degree class, which can’t be fixed. Perhaps your specific set of talents and interests just aren’t a good fit to this kind of job, in which case you need to consider other options.

If all else fails, you may want to cheer yourself up by reflecting on how people who don’t go along with the system can nevertheless have interesting and influential lives, by reading Hunter S. Thompson's 1958 job application to the Vancouver Sun

Susan Greenfield and autistic spectrum disorder: was she misrepresented?

2011-08-12T11:22:00.005+01:00

I have had many emails in response to my open letter to Baroness Greenfield. All but one have been approving. The one exception is an eminent Professor who has chided me for misrepresenting her views. I am reproducing here our unedited email correspondence. I have anonymised the name of the correspondent, as he has not given permission for it to be used, though I will happily break the anonymity if he wishes me to do so, so he can take credit for his arguments.

As a non-celebrity scientist, I would like to get on with my day job and do some data analysis, and so have decided to reproduce the debate here, so that others can pursue it. Please feel free to comment, though please note, I will delete any comments that are off-topic, i.e. those not pertaining to issues around the validity of Greenfield’s claims, and the extent to which they have been misrepresented.

From: xxx@xxx.ac.uk

Sent: 10 August 2011 13:27

To: Dorothy Bishop

Re: Misrepresentation of Greenfield’s article

Dear Professor Bishop,

In your blog of 28 September 2010 you flattered yourself with the aspiration of being a “Paragon”. However, your blog of 4 August 2011 betrays that aspiration and violates the principles of scientific debate. You are misrepresenting Greenfield’s article in New Scientist. To claim that she is blaming what you call “internet use” for the grievous condition of autism is a travesty. The word autism does not appear in that article; Greenfield specifically refers to “autistic spectrum disorders”. Nevertheless, you implore her to “stop talking about autism” and unpleasantly characterise her comments as “illogical garbage”. For clarity I shall repeat myself: autism is not the subject of that article.

It is imperative that scientists engage with all sectors of society and do so accurately, honourably and without intemperate, personal comments. Publishing an assertion which misrepresents the evidence is unacceptable. Furthermore, your blog ignores Greenfield’s explicit references to peer-reviewed papers which provide data consistent with aspects of her general hypothesis (which is not about autism). Perhaps I should remind you of one of the key sentences in Greenfield’s article: “it is not the technologies themselves that I'm criticising, but how they are used and the extent to which they are used”.

Your failure to live up to the aspiration you expressed in your blog of 28 September 2010 saddens me and many other members of our community. In that blog you stated: “Paragons write personal letters to authors”. However, given the public pronouncements which you have made, a public retraction of your misrepresentation is now required. Your earlier experiences as an journal editor will no doubt confirm this requirement.

-------------------------------------------------------------

From: Dorothy Bishop

Sent: 10 August 2011 16:31

To: xxx@xxx.ac.uk

Re: Misrepresentation of Greenfield’s article

I have no intention of withdrawing what I have said. I am happy to defend it.

You seem to think there is a clear distinction between autism and 'autistic spectrum disorders'.

There is not; many people treat them as synonyms, and those who interpret them differently regard ASD as a milder form of the same condition. There is no justification for linking either the severe or the broader category with internet use. The argument I made about a cause needing to precede it effect applies just as much to ASD, broadly defined, as to core autism. ASD does not suddenly appear in middle childhood - the symptoms are evident from around 2 years of age, and so are not plausibly caused by internet use.

If the article is not 'about' ASD/autism, then why does Greenfield mention it at all? This really does upset parents of affected children.

And isn't she aware of the large literature debating reasons for the increasing prevalence of ASD/autism diagnosis? - if she is going to cite this to support her argument, then it behoves her to do her homework.

It is really not acceptable to use innuendo to imply associations, but then back off if challenged to produce evidence.

See: http://www.guardian.co.uk/science/the-lay-scientist/2011/aug/08/1

There is a more fundamental problem here. Susan Greenfield is listened to because she is a scientist. But unlike other scientists engaged in public communcation, she does not confine herself to explaining science to a broader audience. She uses the media to promote her own new theories. What she conspicuously does not do is to publish these ideas in the peer-reviewed scientific literature. This is a shame because it means she has become disconnected from the rest of the scientific community. I would have been happy to voice my criticism by the more conventional means of peer review, which would have been private, or as commentary on a scientific paper, but I am denied that opportunity because Susan Greenfield does not publish these ideas in the scientific literature. Since her views are widely distributed through magazines and newspapers, those of us who find them flawed have no alternative but to challenge them in the public domain. I am aware that a great many people have made 'intemperate personal comments' about Susan Greenfield, but I do not accept that I have done so; I criticised the ideas rather than the person.

I might add that yours is the first critical comment I've had. I have had numerous supportive emails and comments from scientists who have not only written to say they agree, but have thanked me for raising this.

----------------------------------------------------------

From: xxx@xxx.ac.uk

Sent: 11 August 2011 09:49

To: Dorothy Bishop

Re: Misrepresentation of Greenfield’s article

Dear Professor Bishop,

Thanks for your response.

You present yourself as sanguine about conflating Autism and Autistic Spectrum Disorders. I find this surprising and alarming.

Your case now rests on your conviction that all of the adolescents or adults who are currently being diagnosed with any Autistic Spectrum Disorder (at an increasing incidence) could have been diagnosed as such “from around 2 years of age”. Please direct me towards peer-reviewed prospective studies which support this claim.

---------------------------------------------------------------

From: Dorothy Bishop

Sent: 11 August 2011 10:49

To: xxx@xxx.ac.uk

Re: Misrepresentation of Greenfield’s article

I will send you some peer-reviewed papers when I have some free time, but meanwhile, please see Criterion C in the DSM5 proposed revision, as well as the rationale section, which explains the terminology.

You might also find it useful to talk to Professor Sir Michael Rutter, who is the world's leading expert on autism.

--------------------------------------------------------------

From: xxx@xxx.ac.uk

Sent: 11 August 2011 12:00

To: Dorothy Bishop

Re: Misrepresentation of Greenfield’s article

Criterion C in the link you have provided does not address the matter in question: namely, whether there is well-controlled evidence which supports your conviction that all of the adolescents or adults who are currently being diagnosed with any Autistic Spectrum Disorder (at an increasing incidence) could have been diagnosed as such “from around 2 years of age”.

Criterion C merely raises a circular argument, which would be susceptible to unreliable retrospection.

I will indeed raise these matters with Michael Rutter.

But, more importantly, I look forward to receiving from you peer-reviewed papers which substantiate your specific claims.

Sincerely

-----------------------------------------------------------------

Dorothy Bishop

Sent: 11 August 2011 15:51

To: xxx@xxx.ac.uk

Re: Greenfield’s article

Your initial complaint was that I had misrepresented Greenfield because I had failed to distinguish ASD and autism. I trust the DSM5 document has clarified the point for you and you now accept this was not misrepresentation.

You are now demanding that I provide peer reviewed evidence for my supposed "conviction" that "all of the adolescents or adults who are currently being diagnosed with any Autistic Spectrum Disorder (at an increasing incidence) could have been diagnosed as such “from around 2 years of age”.

I have sent you information pointing out that it is is part of the diagnostic criteria for ASD to have onset in early childhood.

This is not a circular argument. It is merely pointing out that ASD, as defined by gold standard diagnostic criteria, could not be caused by environmental influences that only start in later childhood. I reiterate the last sentence from the DSM 5 rationale section: "Autism spectrum disorder is a neurodevelopmental disorder and must be present from infancy or early childhood, but may not be detected until later because of minimal social demands and support from parents or caregivers in early years."

Note that this does not mean that all children with ASD will be diagnosed in childhood, but it does mean that they have evidence of autism in early childhood. This is typically identified by an interview instrument such as the Autism Diagnostic Interview.

To clarify my argument.

1. When asked for evidence that the internet is changing people's brains, Greenfield stated, among other things, "There is an increase in people with autistic spectrum disorders."

To most people this would imply that she is saying the internet is a causal factor in the increase in autistic spectrum disorders.

2. There has been an increase in autistic spectrum diagnoses over the years.

However, this evidence comes from epidemiological studies that do use standard diagnostic criteria including the onset criteria (see attached articles).

3. Since internet use cannot plausibly cause a disorder starting in a toddler, this is not a valid argument.

You now demand that I prove that "all of the adolescents or adults who are currently being diagnosed with any Autistic Spectrum Disorder (at an increasing incidence) could have been diagnosed as such “from around 2 years of age”. "

This is an attempt to move the goalposts. Of course diagnosis is not perfect. There may be misdiagnosed cases. The fact that you demand this evidence suggests that Greenfield's argument (as filtered by you) is now :

a) there are children who don't have autism in early childhood but who develop some kind of quasi-autism in middle childhood

b) this is caused by internet use

c) such cases account for the increase in ASD diagnoses, even though they don't meet DSM criteria for ASD

Do you have any evidence for any of these postulates ?

If that is not what you are saying, what exactly is the claim?

You have also not responded to the point I made about the appropriate place for a scientist to publish new scientific theories. Do you think it is appropriate to make statements about aetiology of a major neurodevelopmental disorder in a non peer-reviewed journal such as New Scientist, when there is no peer-reviewed work to back them up, even if the causal claims are by innuendo rather than direct statement?

If you would like your point of view have broader recognition, I would be happy to publish this correspondence on my blog, so that Greenfield's position and the supposed limitations of my arguments could be given wider publicity.

pdfs of the following papers were attached:

Baird G, Simonoff E, Pickles A, Chandler S, Loucas T, Meldrum D, Charman T: Prevalence of disorders of the autism spectrum in a population cohort of children in South Thames: the Special Needs and Autism Project (SNAP). Lancet 2006, 368 (9531):210-215.

Baron-Cohen S, Scott FJ, Allison C, Williams J, Bolton P, Matthews FE, Brayne C: Prevalence of autism-spectrum conditions: UK school-based population study. British Journal of Psychiatry 2009, 194:500-509.

Brugha, T. S., McManus, S., Bankart, J., Scott, F., Purdon, S., Smith, J., et al. (2011). Epidemiology of Autism Spectrum Disorders in Adults in the Community in England. Arch Gen Psychiatry, 68(5), 459-465.

Fombonne, E. (2005). The changing epidemiology of autism. Journal of Applied Research in Intellectual Disabilities, 18, 281-294.

Kim, Y. S., Leventhal, B. L., Koh, Y.-J., Fombonne, E., Laska, E., Lim, E.-C., et al. (2011). Prevalence of autism spectrum disorders in a total population sample. American Journal of Psychiatry.

Rutter, M. (2005). Incidence of autism spectrum disorders: Changes over time and their meaning. Acta Paediatrica, 94, 2-15.

Taylor, B. (2006). Vaccines and the changing epidemiology of autism. Child: care, health and development, 32(5), 511-519.

Williams, J. G., Higgins, J. P. T., & Brayne, C. E. G. (2006). Systematic review of prevalence studies of autism spectrum disorders. Archives of Disease in Childhood, 91, 8-15.

Wing, L., & Potter, D. (2002). The epidemiology of autistic spectrum disorders: is the prevalence rising? Ment Retard Dev Disabil Res Rev, 8, 151-161.

P.S. 13.52 on 12th August 2011
A further response from xxx

Dear Professor Bishop,

I am astonished by your peremptory decision to publish our correspondence without permission. I ask you to add the response below, without any editing, as a matter of urgency.

Dear Professor Bishop,

In your first email you stated: “ASD does not suddenly appear in middle childhood - the symptoms are evident from around 2 years of age”. This non-ambiguous statement means that all people who are diagnosed with an Autistic Spectrum Disorder after early childhood will have been displaying its symptoms from around 2 years of age.

You now point out: “it is part of the diagnostic criteria for ASD to have onset in early childhood”. The difference from your initial statement is salient. Thus, it is the case that that unless those symptoms are present in early childhood, an Autistic Spectrum Disorder may not, by definition, be diagnosed.

In this context, you draw attention to “Criterion C in the DSM5 proposed revision”. As I am sure you realise, DSM5 will not supersede DSM-IV until 2013. The criteria you describe as “gold standard diagnostic criteria” are part of a proposed revision.

I shall consider just one matter arising:

Autistic Disorder and Asperger’s Disorder are addressed separately under DSM-IV. The current diagnostic criteria for Asperger’s Disorder (DSM-IV) include the following: “There is no clinically significant general delay in language (e.g. single words used by age 2 years, communicative phrases used by age 3 years). There is no clinically significant delay in cognitive development or in the development of age-appropriate self-help skills, adaptive behaviour (other than in social interaction), and curiosity about the environment in childhood”. Indeed, a delay in social interaction is the only age-related point mentioned; no critical age is given for its onset.

I recognise that the revisions for DSM5 under current consideration are being guided by the following:

”Asperger’s Disorder. The work group is proposing that this disorder be subsumed into an existing disorder: Autistic Disorder (Autism Spectrum Disorder)”.

If this were to be enacted, diagnosis of Asperger’s Disorder would be precluded, unless its symptoms were present in early childhood (as specified by Criterion C). Again, I feel it is appropriate to ask for evidence which supports your original statement: “the symptoms are evident from around 2 years of age”. According to your gold standard DSM5, this must apply to Asperger’s Disorder. It is reasonable for me to ask whether this has been substantiated by prospective studies which are free from potentially unreliable parental retrospection. I may be in error, but I have found no such study among the papers you kindly sent me. I sincerely apologise if I have overlooked something relevant.

The immensely complex matters of aetiology and diagnosis are not given due consideration if proposed revisions (which are still subject to consultation) are presented as “gold standard”.

In my preceding email I wrote: “I look forward to receiving from you peer-reviewed papers which substantiate your specific claims”. I am saddened to note that you have chosen to misrepresent this polite request as “demanding”. It seems that our discourse will not be fruitful and that it should be closed.

An open letter to Baroness Susan Greenfield

2011-08-04T12:34:00.000+01:00

As a contemporary of yours, I have followed your career with interest over the years. I was delighted when in 1994 you were selected to give the Royal Institution Christmas lectures - the first woman ever to be so honoured. The lectures were fun and informative and delivered with enthusiasm and charisma. Since then, however, I’ve been dismayed by the way in which your public communications have moved increasingly away from science. You are frequently invited to give your opinion on topical matters, because of your status as a ‘top neuroscientist’. This leads people to assume that what you say is grounded in evidence. You have a splendid opportunity to act as an ambassador for science, but you don’t seem interested in doing that. Instead, we are increasingly treated to opinions without the evidence to support them.

I would just shake my head sadly at this lost opportunity, except that in recent years your speculations have wandered onto my turf and it's starting to get irritating. In the New Scientist this week, you mention the rise in autism as evidence for your concerns about the impact of the internet on children’s brains. Previously I’ve read that you've made similar comments about ADHD. You may not realise just how much illogical garbage and ill-formed speculation parents of children with these conditions are exposed to. Over the years, they’ve been told that their children’s problems are caused by their cold style of interaction, inoculations, dental amalgams, faulty diets, allergies, drinking in pregnancy - the list is endless. Now we can add to this list internet use. Except that here, at least, parents will be able to detect the flaw in the logic. A cause has to precede its effect. This test of causality fails in two regards. First, demographically - the rise in autism diagnoses occurred well before internet use became widespread. Second, in individuals: autism is typically evident by 2 years of age, long before children become avid users of Twitter or Facebook. You also seem unaware of the large literature discussing possible causes of the increase in autism diagnoses, most of which concludes that most, if not all, of the increase is down to changes in diagnostic criteria, (see e.g. Fombonne et al., 2005).

I wish you would focus on communicating about your areas of expertise - there’s plenty of public interest in neurodegenerative diseases, and I’m sure you could do a great job explaining this topic to a broad audience. Or you could give up the work on neurodegenerative diseases and devote your time doing research to follow up your hunches about effects of internet use, which I agree is an interesting topic. But please, please, stop talking about autism.

What does it take to become a Fellow of the Royal Society of Medicine?

2011-07-24T17:54:00.001+01:00

According to Andy Lewis, aka @lecanardnoir, the answer is around £356 for a London resident*. He revealed this discovery in a blogpost a couple of years ago. He was investigating the c.v. of Jayney Goddard, President of the Complementary Medical Association. Her website describes how she uses homeopathy, psychotherapy and hypno-analysis and ... is a Fellow of the Royal Society of Medicine. Can this be true?, you might ask. Isn’t the Royal Society of Medicine like a medical wing of the Royal Society, an organisation to which only those of the highest academic stature are elected? Er, well, no. It isn’t, and many of those working in alternative and complementary medicine are delighted at the ease to which they can gain an affiliation, and so embellish their CVs with impressive-sounding medical credentials. Perhaps this has something to do with the fact that HRH the Prince of Wales was made an Honorary Fellow of the RSM in 2005.

Here are just a few of those who mention their affiliation to the RSM on their websites, and no doubt impress members of the public by doing so. I haven't been able to find a directory of members or fellows to check accuracy of these claims.

Dr Dato' Steve Yap. Complementary medical director, DSY Wellness & Longevity Center, Malaysia. His website has the initials FRSM after his name, even though this is specifically prohibited by the RSM. His qualifications include a Masters degree in Administration from the University of Durham, and Board certification in Nutritional Medicine and Anti-Aging Medicine from the World Society of Anti-Aging Medicine, France.
Terence Watts. Founder of The Essex Institute, where students learn advanced skills in both psychotherapy and hypnotherapy, and the Association for Professional Hypnosis and Psychotherapy. His website disarmingly explains how he was a late starter who came to hypnotherapy at the age of 48, after working as a professional ballroom and Latin-American dancer, supplemented by spells as aTV engineer, electronics design, tailor, carpet-layer, computer programmer, furniture shop assistant, factory hand, salesman (fire extinguishers and alarms) and part time rock 'n' roller (lead guitar). He notes proudly that he is the first in his profession to be made a Member of the City and Guilds Institute, which he states is comparable to a British Masters degree.
Harald Camillo Gaier. Homeopath, naturopath, master herbalist, acupuncturist and author of the Encyclopaedic Dictionary of Homoeopathy.
Peter King. Principal tutor at the British School of Traditional Japanese medicine. He has an MA in ‘Sports Science & Japanese Budo Studies’and also has qualifications in osteopathy, cranial osteopathy, acupuncture, Shiatsu, and Advanced Chinese Tuina. The website also notes that the Honorary Principal of the British School of Japanese Medicine, Hatsumi Sensei, has been honoured at the RSM by the permanent inscription of his name on the 'Wall of Honour'.
The late Prof. Dr. Sir Anton Jayasuriya. Founder of Medicina Alternativa International, promoting and propagating acupuncture, homeopathy and natural medicine.
David Reeves. President of the (British) National Register of Advanced Hypnotherapists. In private practice as a Psychoanalyst, Hypnotherapist since 1991, and as a Stress Management Consultant since 1994. Before moving into the field of Hypnotherapy and Stress, his background was in the commercial sector reaching the level of Managing Director.
Dr. Lyn M. Bateman. Has a Doctorate in Clinical Hypnosis/Hypnoanalysis and Doctorate and Ph.D. (sic!) in Alternative Medicine. I'm not clear which institutions offer such qualifications. Also has a seriously illiterate website, which advertises training in medical hypnosis for non-medical persons.
Marcus Webb. Registered Naturopath and Osteopath who qualified in 1988 from the British College of Osteopathic medicine (formally the British College of Naturopathy and Osteopathy) where he served as a part-time lecturer for four years.
Iskra Harle. Naturopath offering treatment for "Fibromyalgia, Arthritis, Infertility (men & women), Acne, Hypothyroidism, Allergies, Irritable bowel syndrome, Wheat intolerance, Milk intolerance, Weight control, Fatigue, Depression, Migraine & all kinds of headaches, Early stages of Alzheimer's disease/senile dementia, Back pain, Frozen shoulder, Post surgery recovery, Post chemotherapy recovery, and many more".

Do they turn anyone away? It’s hard to tell. The closest case I could find was Ingrid P. Dickenson, BRCP EMR, Electromagnetic Pollution Consultant, who is trained in Colour Therapy, Psychosynthesis Counselling, Reiki, Oneiric (Dream) Therapy, Communication Skills with children, Group Facilitation and Electro Crystal Therapy. She describes herself as a former associate member of the RSM, noting sniffily, “Due to The Royal Society of Medicine's inability to acknowledge the effects of electromagentic pollution (despite Ingrid's multiple contacts to them) Ingrid decided to cancel her subscription in early 2011.”

*Now stands at £365. See RSM site.

How to survive in psychological research

2011-07-13T16:55:00.001+01:00

A Handbook of Skills and Methods in Behavioural Research is not the place you’d expect to find something to make you smile, but many years ago one of my graduate students pointed me to a wickedly funny piece by Ray Hodgson and Stephen Rollnick. Since then I’ve found myself loaning an increasingly dog-eared photocopy of the article to new generations of students and postdocs. Sadly, the article is not available electronically, though copies of the book can be tracked down. So here’s a summary of Hodgson and Rollnick’s laws, all of which are as pertinent to the older, seasoned researcher as to the intended readership of the ‘young, lively, questioning researcher who has great expectations but a lack of practical experience’:

Law #1. Getting started will take at least as long as the data collection

The barriers are various: perhaps the most salient for the newcomer is dithering induced by fear of commiting to a non-optimal design. Another barrier is having too many people involved; this just multiplies the dithering, as each person tries to include additional measures or graft on subsidiary projects. It’s vital to have someone who will take control for decision-making - a point emphasised in my previous post on the NationalChildren’s Study.

Hodgson and Rollnick also mention the need to get ethics approval, another topic that has featured on my blog. It’s got a lot worse in the years since they wrote their article: there’s even a kind of ‘meta-research’ in which the goal is to quantify the baleful influence of ethical scrutiny on research efficiency (e.g. Elwyn et al, 2005).

Law #2. The number of available subjects will be one-tenth of your first estimate

Note to young readers: ‘subjects’ are what we used to call ‘participants’ until someone decided that the term implied an unheathily controlling attitude to those taking part in experiment.

Quite simply, "as soon as somebody starts to research a particular condition, people with that condition leave the district". It’s totally true and totally mysterious.

Law #3. Completion of a research project will take twice as long as your last estimate and three times as long as your first estimate

This may be moderated by whether you are a pessimist or optimist, but no true pessimist would ever embark on a research project.

Law #4. A research project will change twice in the middle

Hodgson and Rollnick cite their experience with a one-year project to test the effectiveness of a Drinkwatchers program for problem drinking. “All we needed were thirty subjects from amongst the estimated 10,000 problem drinkers in South Glamorgan. One hundred and sixty problem drinkers answered the advertisement…Of these only eight volunteered to join a Drinkwatchers group, three turned up to the first meeting and one of these came to the second.” Since the study had been funded, the researchers decided the best they could do was an alternative study to discover what kinds of help problem drinkers really want. Needless to say, these days, such a change of plan would necessitate fresh ethics approval which would consume all the remaining time on the grant.

Law #5. The help provided by other people has a half-life of two weeks

Yes, yes, yes. Never do a study that depends on the kindness of strangers.

Law #6. The tedium of research is directly proportional to its objectivity

You really do need to know this when you start out in research. If you detest mundane, repetitive activities, try another career.

Law #7. The effort of writing up is an exponential function of the time since the data were collected

If the person who collected the data has left by the time you come to write it up, then it can be hard to remember exactly what was done, so you’d better be sure to have had a real obsessive in charge, who will document thoroughly every step of the research collection and data coding. Hodgson and Rollnick reckon that data that sit in a filing cabinet for 4 years will never escape.

On a more serious note, failure to get stuff written up is incredibly wasteful, especially if the funding for the study came from public funds. Sometimes the failure just comes from writer’s block, and sometimes because the researcher discovers a flaw that makes the study unpublishable. More commonly, though, the failure to write up is because the results are deemed uninteresting. This has the unfortunate effect of distorting the research literature, as null results are left in the file drawer. I'd like to see journal editors adopting a policy of determining ‘publishability’ of a paper on the basis of Introduction and Methods alone: if an interesting problem has been identified and the study is well-designed and adequately powered to answer it, then it should be published, regardless of the results.

Yet another reason for failure to publish is researchers who bite off more than they can chew. As I’ve suggested in a previous post, we need to move away from a system whereby the rewards for researchers are proportional to the amount of grant income they receive, to one that rewards thrift. And if research funders find themselves overwhelmed with far more proposals than they can fund, they should consider vetoing those who already have substantial funding, even if they are ace researchers. There is a limit to how much research someone can do and do well.

Law #8. Evidence is never enough

So you are lucky enough to get an interesting result, and are confident that this will change the field and make your reputation. And what happens? Nobody takes any notice. Hodgson and Rollnick note that research that conflicts with the prevailing view is likely to be ignored, but that’s not the only problem. You do also have to sell your science. But that does not need to mean cutting corners or distorting findings. But learn to write accessibly, get out there and give talks, start a blog (!) and, most important of all, focus on problems that are important.

Get hold of the original Hodgson and Rollnick chapter if you want positive tips on how to be a successful researcher. And for further advice, it’s hard to better Peter Medawar’s 1979 book Advice to a Young Scientist.

References
Elwyn, G. (2005). Ethics and research governance in a multicentre study: add 150 days to your study protocol BMJ, 330 (7495), 847-847 DOI: 10.1136/bmj.330.7495.847
Hodgson, R., & Rollnick, S. (1989). More fun, less stress: How to survive in research. In G. Parry & F.-N. Watts (Eds.), Behavioural and mental health research: A handbook of skills and methods (pp. 3-13). Hove, England: Lawrence Erlbaum Associates.

The National Children’s Study: a view from across the pond

2011-06-25T15:45:00.001+01:00

The National Institutes of Health, the major government funding agency for medical research in the US, is in trouble. Although a threat of a serious funding cut was largely averted, funding for 2011 is nevertheless $260 million less than for 2010. It is predicted that only 1 in 6 grant applications will be funded this year, the lowest level yet. Such low funding levels are not only disappointing for researchers with good ideas, they also are inefficient, because scientists end up spending more time writing unsuccessful grant proposals than in doing research.

So it gets interesting to see what does get funded. NIH has a great website where you can find out how they spend their money. I came across it when doing a study on the amount of funding going to research on different neurodevelopmental disorders. I revisited the site recently to look in more detail at the kinds of study that get funded in this general area, and I had a bit of a surprise. There were huge sums of money going to something called the National Children’s Study. I added them up and they came to around $500 million.

More Googling revealed that the National Children’s Study is a longitudinal cohort study that plans to follow 100,000 children from the prenatal period to 21 years of age, to investigate environmental and genetic influences on health and development. There’s no doubt that big longitudinal cohort studies have provided invaluable data in the past; they can provide samples that are large enough to detect small effects, and if the sample is representative of the population, they can be used to estimate prevalence of different conditions. They are essentially correlational, though, and therefore less conclusive than experimental studies for establishing cause and effect.

The cost of such a large-scale enterprise is bound to be high, but just how high is acceptable? The National Children’s Study website indicated that the spend to date was actually higher than the figure I had from the NIH source - $608 million, with the bulk of the expenditure, $553 million, since 2007 when the Implementation phase of the project began. And it is clear that the future spend will be higher still, as the cost of following up children and assessing their development is taken into account. The total cost was estimated in 2004 to be $2.7 billion.

The background to the study is clearly explained in an article by Nigel Paneth. During the Clinton presidency The Children's Act of 2000 was passed, instructing the NIH and other federal agencies to “. . . plan, develop, and implement a prospective cohort study, from birth to adulthood, to . . . incorporate behavioral, emotional, educational, and contextual consequences to enable a complete assessment of the physical, chemical, biologic, and psychosocial environmental influences on children's well-being . . . .” As Paneth drily pointed out, this directive was not accompanied by any funding during the 7 years of the Bush administration, so efforts were expended instead on planning and piloting the study, in anticipation of future funding. However, too much planning can be a bad thing, and “as a ship lying in port too long becomes encrusted with barnacles, so the National Children's Study became weighted down with a myriad of measures and instruments” (Paneth, 2010). But then, salvation. Under the Obama administration, the study returned to the political agenda, and $200 million per annum was allocated for 2009-2010. But problems remain. In an article entitled “Saving the National Children’s Study”, David Savitz and Roberta Ness argued that the study is in trouble. There has been debate about the best methods of recruitment of participants, and about the organizational structure of the project. “There are multiple advisory groups representing the lead federal agencies, the extramural research community; an executive committee consisting of selected Center investigators; and oversight by the National Children’s Study Program Office, the Director of NICHD, and the Director of NIH—yet how the major decisions affecting the future of the National Children’s Study will be made remains unclear.” (Savitz & Ness, 2010). Despite the huge sums of money allocated to the project, there is still uncertainty about methods. “To decide where the study should be done and who is capable of doing it, one has to know what the study will entail. The lack of a clear protocol has led to an extended, expensive, and not-fully informative pilot phase.” (ibid).

Of course you can’t know in advance exactly how longitudinal cohort data will be used: one of the benefits of previous cohort studies has been that they provide a resource to allow investigators to test their ideas against existing data, rather than having to collect a new dataset. For example, in the UK, the Avon Longitudinal Study of Parents and Children, which started in the 1990s, is now being used to investigate hypotheses about genetic and environmental influences on behaviour. When the project started, it wasn't possible to anticipate the developments in genetics, but the prescience of the researchers in gathering DNA samples mean that the data are invaluable, especially since a wealth of environmental and psychological measures are also available. Nevertheless, even allowing for such uncertainty the vagueness of the Hypotheses section of the National Children’s Study website, is worrying:

“Working hypotheses developed by the multiple teams of scientists have been summarized as an efficient and dynamic reference of the current questions to be addressed by the Study. The current list of hypotheses continues to evolve. It is expected that some hypotheses still being refined will be added and, over the long course of the Study, new questions will emerge and be added to the Study and some of the hypotheses included may become outdated and discarded. However, there is consensus among scientists planning the Study that as a group, the hypotheses alone can neither convey the true breadth of the Study nor completely guide the planning and design of the Study. To further define the full scope and topics of the Study, priority outcomes along with priority exposures are identified as health areas for the Study. The priority exposures and outcomes serve as an organizing framework for the Study hypotheses.” This sounds awfully like “We’re going to gather loads of data and hope that it pans out.”

Well, you might say, any nationally-based study of 100,000 families is bound to be complicated and expensive. But, as Savitz and Ness point out, there are examples of European studies that are comparable in size and scope, yet do not appear to have got so mired in difficulties. They draw attention to the Danish National Birth Cohort and the Mothers and Babies study in Norway, each of which is studying over 100,000 children enrolled in utero. I have not yet been able to find figures for the Danish study*, but the Norwegian study costed around $45 million over the 10 year recruitment period. Funding was announced this year for a new British Birth Cohort study that will follow 90,000 children from birth at a cost of £33.5 million. I’m sure that all these European cohort studies will ultimately cost far more than initially planned - such studies have a tendency to gobble up funds as the children grow older and the difficulties of maintaining a follow-up increase. The articles written on the Danish and Norwegian studies emphasise the logistic and ethical difficulties of doing studies of this kind, and the uncertainty about the future. But even so, the costs of the National Children’s Study seem set to exceed those of European studies by an order of magnitude.

I’m struck, as Savitz and Ness were, by the organizational differences between the European and US studies. Commenting on the Scandinavian cohort studies, they point out “These studies, each led by a very small group of investigators, took an early and consistent approach to their specifications and made tough decisions about what aspects to promote and what to curtail, based on a shared, explicit vision for the study.” This is very different from the huge committee-based structure in charge of the National Children’s Study. Committee-based decision-making even extends in the US study to writing up work for publication: “The Publications Committee will identify topics, set priorities, and facilitate the preparation of primary Study publications. The Committee will define the general scope and content of primary publications and assemble Writing Teams to prepare those publications. The Committee will define the charge of these Teams, monitor their progress, and review the manuscripts to ensure adherence to the charge given to the Teams, before the manuscript is submitted for publication in a peer-reviewed journal.” As an approach to scientific communication, this approach seems designed to kill stone dead any enthusiasm or creativity in the researchers. I think if NIH wants to get good science for its money, it would be better off taking an analogy of the scientist as an enthusiastic market gardener rather than factory farmer.

But a more fundamental question is whether the outcomes of this study will justify the enormous cost, or whether the money might be better spent elsewhere. A key question is how far we need another longitudinal cohort study, or whether the existing European studies will be able to answer many of the questions that the researchers are interested in. It’s not easy doing a cohort study anywhere, but it seems particularly tricky in a country that does not have a national health service. There are many areas funded by NIH where an individual investigator could do useful work with a grant of around half a million dollars. The funds spent to date on the National Children’s Study could have funded over 1200 such research projects. Viewed from this side of the pond it seems that, with the best of intentions, US funding for health research is simultaneously starving out the best of its scientists while feeding a bloated monster.

* Update on 2nd July 2011. I've now got a couple of estimates from researchers involved in the Danish study and both indicate a total cost since 1994 of less than $US 20 million. That is around 1/10th the cost of the National Children's Study just for one year, 2010. The Danish researchers make the point that the study benefits from the existence of national registers that minimize costs of case-finding, and that already contain much relevant information.

References
Kaiser, J. (2004). NIH Launches Controversial Long-Term Study of 100,000 U.S. Kids Science, 306 (5703), 1883-1883 DOI: 10.1126/science.306.5703.1883
Magnus, P., Irgens, L., Haug, K., Nystad, W., Skjaerven, R., Stoltenberg, C., & , . (2006). Cohort profile: The Norwegian Mother and Child Cohort Study (MoBa) International Journal of Epidemiology, 35 (5), 1146-1150 DOI: 10.1093/ije/dyl170
Olsen, J., Melbye, M., Olsen, S., Sorensen, T., Aaby, P., Nybo Andersen, A., Taxbol, D., Hansen, K., Juhl, M., Schow, T., Sorensen, H., Andresen, J., Mortensen, E., Wind Olesen, A., & Sondergaard, C. (2001). The Danish National Birth Cohort - its background, structure and aim Scandinavian Journal of Public Health, 29 (4), 300-307 DOI: 10.1177/14034948010290040201
Paneth, N. (2010). Saving the National Childrenʼs Study From Its Saviors Epidemiology, 21 (5), 602-604 DOI: 10.1097/EDE.0b013e3181ea5f89
Savitz, D., & Ness, R. (2010). Saving the National Childrenʼs Study Epidemiology, 21 (5), 598-601 DOI: 10.1097/EDE.0b013e3181e942cc

Autism diagnosis and hyper-systemizing parents: Nottingham vs. Eindhoven

2011-06-21T10:56:00.005+01:00


Family life in Eindhoven? © cartoonstock.com

With an insensitivity one might have expected from a Sun headline writer, the New Scientist summarised the findings from a recent study thus: Childhood autism spikes in geek heartlands. The bottom line is that autism is diagnosed more than twice as often in Eindhoven, a city in the Netherlands where many are employed in the IT industry, than in two other cities of comparable size, Haarlem and Utrecht. The explanation favoured by the researchers is that autism is characterised by a ‘systemizing’ style of thinking, which is part of normal human variation, and which is common in those in technical and mathematical occupations. Cities with a strong IT presence will attract an unusually high number of high-systemizer residents, and these people will be more likely to have offspring with autism. This is, then, a genetic explanation, and it’s given some plausibility by a large body of research demonstrating that parents of children with autism are more likely than other parents to show indicators of mild autistic-like characteristics, the so-called ‘broad autism phenotype’. (Although, on an admittedly quick search, the evidence that parents of children with autism tend to be high systemizers seems rather weak: mild symptoms of social and communicative problems seem a more pronounced feature of the broad autism phenotype than hyper-systemizing behaviours (Bishop et al., 2004; Wheelwright et al., 2010; Windham et al., 2010)).

As the researchers themselves note, the hyper-systemizing account is not the only possible explanation for their result. Crucially, the study relied on counting diagnoses from school records, rather than screening the population in a standard fashion. Although it’s not hard to recognise a case of classic Kanner autism, there’s far more disagreement about diagnoses for children with milder symptoms. As I argued on a Guardian blog, unless we have clear objective criteria for diagnosis, it’s hard to compare one prevalence rate with another. The different numbers could just reflect local expertise, policy or practice in diagnosing autism.

One limitation of the published study is that the researchers are quoted as saying that their study was prompted by anecdotal reports that autism was abnormally common in Eindhoven. While it is worth checking out if the anecdote is accurate, this makes Eindhoven less than ideal for testing the hyper-systemizing hypothesis, as it potentially capitalises on a chance blip. What would be better would be a study with clear a priori predictions, based solely on levels of IT industries in different cities. In theory, it should be possible to do this using publicly-available data from the UK published by the Department of Children, Schools and Families. This dataset has the advantage of being comprehensive, unlike the school report data from the Eindhoven study, which relied on schools providing the data (- the response rate was 75% for Eindhoven but only 50% for Haarlem and 46% for Utrecht). A recent report by Lindsay (2011) presented some data from this UK database on numbers of children with Special Educational Needs categorised as having Speech, Language and Communication Needs (SLCN) or Autistic Spectrum Disorder (ASD). The numbers of children with these labels varied massively from place to place, as shown in Table 1.

Table 1: Percentages of children with SEN diagnosed with SLCN or ASD;

Data from local authorities with the highest or lowest % of either diagnosis

I have no idea whether the number of IT experts is higher in Nottingham than Leeds, but it’s noteworthy that places where you might expect high levels of hyper-systemizing, such as the university towns of Oxford and Cambridge, don’t feature among the places with very high rates of ASD diagnoses. Lindsay also points out that “the two neighbouring authorities of Nottingham and Nottinghamshire have almost identical prevalence rates for SLCN and ASD, despite one being a large city, and the other a shire county” (p. 143). Clearly, my intuitions are no substitute for real data, and it’s also important to note that the data in Table 1 are not population prevalence figures, but instead are proportions of children who have already been identified as having Special Educational Needs. One would need to use the frequencies of ASD in the population as a whole to test the hypothesis of a correlation between level of IT industry and rates of autism. The data are available, and this might seem like a nice project for someone to do, except for a major problem. As Lindsay emphasised, it is impossible to conclude from the UK data whether prevalence really do vary across the country, because the definitions of disorders are inconsistent from one area to another. It’s possible that criteria for autism are more standardised in the Netherlands than in the UK, but the UK data make me suspect that it’s just not possible to draw meaningful conclusions about prevalence from data based on educational records.

References

Bishop, D. V. M., Maybery, M., Maley, A., Wong, D., Hill, W., & Hallmayer, J. (2004). Using self-report to identify the broad phenotype in parents of children with autistic spectrum disorders: a study using the Autism-Spectrum Quotient. Journal of Child Psychology and Psychiatry, 45, 1431-1436.

Lindsay, G. (2011). The collection and analysis of data on children with speech, language and communication needs: The challenge to education and health services. Child Language Teaching & Therapy, 27(2), 135-150.

Roelfsema, M., Hoekstra, R., Allison, C., Wheelwright, S., Brayne, C., Matthews, F., & Baron-Cohen, S. (2011). Are Autism Spectrum Conditions More Prevalent in an Information-Technology Region? A School-Based Study of Three Regions in the Netherlands Journal of Autism and Developmental Disorders DOI: 10.1007/s10803-011-1302-1

Wheelwright, S., Auyeung, B., Allison, C., & Baron-Cohen, S. (2010). Defining the broader, medium and narrow autism phenotype among parents using the Autism Spectrum Quotient (AQ). Molecular Autism, 1(1), 1-9.

Windham, G. C., Fessel, K., & Grether, J. K. (2009). Autism spectrum disorders in relation to parental occupation in technical fields. Autism Research, 2(4), 183-191.

Should we ever fight lies with lies?

2011-06-19T08:30:00.006+01:00

Photo from verdammelt's photostream

So here’s my moral dilemma. The pangolin is a threatened species. It is a peaceful creature which has evolved an impressive strategy of self-defence; it curls up into a scaly ball. The effectiveness of this strategy against predators can be seen here. Unfortunately, this doesn’t work when your opponent has hands, and can just pick you up. Pangolin numbers are being depleted by poachers who sell them for Chinese medicine. Pangolin scales are known as Chuan Shan Jia and are reckoned to be effective for such purposes as “expelling wind-damp from the channels”. They are also recommended for treatment of cancer, see for instance, this website.

Although sterling work is being done by conservation agencies, time seems to be running out. So I wondered about taking a different tack. As Ben Goldacre has famously remarked, the Daily Mail has a mission to classify inanimate objects into those that give you cancer vs those that cause cancer, so how about getting pangolin scales on the former list? This idea has been germinating in my brain for a while, but there are two major obstacles. First, I have a reputation for truthfulness, which I’d be sorry to lose, and second, how would I start such a rumour?

Well, a recent post by Jo Brodie gave me an answer for part 2. Jo noted that if you upload photos to Flickr, they get picked up by people doing Google searches. Her photos of ads for weight-loss tea, accompanied by her comments that there’s no evidence for effectiveness, have actually appeared on websites advertising the stuff.

But Jo’s posts don’t involve any dishonesty. I want to spread a rumour about Chuan Shan Jia that is completely fictitious. I had some misgivings, but I did it. I opened a Flickr account and made a graph. I took a figure from a powerpoint presentation on cancer rates and doctored it to be about pangolin scales. You can see it here. [No you can't, link removed. See P.S. below!]. Whether the sort of people who use Chinese medicines will actually ever find this seems unlikely, but it seemed worth a shot. If anyone would be willing to translate the slide into Chinese, I'd be most grateful.

I had a moment wondering whether I’d get sued by purveyors of Chinese medicines, but I don't think I need to worry, since the use of endangered species in medicines is illegal.

So what about my reputation? Well I am hoping that I can preserve it by declaring here that you can rely on me to be honest in my science communications on all topics except use of medicines based on endangered species. If my pangolin hoax works, you can anticipate a Flickr posting on rhino horn next.

My husband is very disapproving and thinks it is never right to fight lies with lies. What do others think?

P.S. 20th June 2011

Well, the verdict in the comments is clear, and my husband is now feeling extremely smug. To go down this route is not just unethical but also foolish. So I won't. I had hoped that by declaring myself as a hoaxer at the same time as initiating a hoax, I could defend my integrity, but I see that this would be a dangerous assumption, and the unintended consequences could be serious.
Thanks to Jo, I've discovered that my inexperience with Flickr means that my hoax graph wasn't copyable anyway.
But I'm glad I posted this: very interesting and helpful to get the reactions, and Jo has given me ideas for pursuing my vendetta against pangolin medicines without any dishonesty. I also inspired a great set of tweets from @Artvanderley that demonstrate another approach: implant the idea there could be bad effects of pangolin medicine without claiming it: e.g. "40 years ago no one believed that ingesting pangolins was linked with cancer, excessive weight gain and genital atrophy". I suspect that some of you may think that even this may be sailing too close to the wind, but it's clearly far less heinous than inventing data.
I'm left feeling just a bit disappointed, as I have always been very well-behaved and law-abiding and was rather enjoying my brief dalliance with hoaxing. But yes, the issues are serious, and so I will climb wearily back up on the moral high ground.

A gentle introduction to Twitter for the apprehensive academic

2011-06-14T20:28:00.005+01:00

If I tell people I’m on Twitter, I tend to get one of three reactions:

a) Isn’t it all about what Lady Gaga had for breakfast?

b) How do you find the time?

c) You?!!! (Implication: Twitter is for hip juveniles rather than fossilised academics)

This is unfortunate, because Twitter is a valuable resource for academics. If you’re allowing inaccurate stereotypes to deter you, you’re missing out.

First of all, you have to understand what Twitter is. It’s totally different from email, and more like a news broadcast. People all over the world are continually emitting tweets (very short messages) any of which can be viewed by anyone. You select what you want to attend to. There are two ways of doing this. The default method is to ‘follow’ particular people or organisations who tweet. Their tweets then appear in your timeline, which appears as a scrolling list when you open your Twitter page. The other method is to search for tweets that include a particular word: for instance, if you type ‘neuroscience’ into the search box at the top of the page, you’ll see all the tweets in the twitterverse that include that word, starting with the most recent.

If you want news about Lady Gaga, there’s plenty out there. But if you want information of a different kind, you can follow organisations such as the Royal Society, the Wellcome Trust, Guardian Science, the New York Times, Nature, etc. etc. Most scientific organisations, newspapers, and science journals are on Twitter, and by following them you have an up-to-date news stream about their activities.

It's OK to be a purely passive user of Twitter, just following people who interest you. In the circles I move in, a high proportion of tweets are messages pointing to a weblink, which may be a newspaper or journal article or a blog. This is where Twitter is such a useful resource for the academic: if you follow those who share your academic interests, they will point you to interesting stuff. When I first joined up I was impressed to find that within the first few days, I’d been directed to two new papers in my field that were very relevant to my work and that I hadn’t known about.

Many people remain as passive users, but you’ll get much more out of Twitter if you use it actively and emit your own tweets. Written an interesting paper? Starting up a blog? Twitter is a great way of informing people, but there’s a catch: you need to have followers, a topic I discuss more below.

How do I get started?

You can Google to find plenty of good guides to the mechanics of tweeting. See, for instance:

http://news.cnet.com/newbies-guide-to-twitter/

http://mashable.com/guidebook/twitter/

However, most of these are directed towards people who do have a keen interest in Lady Gaga’s breakfast, or who wish to use Twitter for business purposes. The suggestions here are to complement the ‘how to do it’ guides with advice geared towards academic users.

Signing up is dead easy: just follow the instructions at http://twitter.com/.

You need a username. Keep it fairly short and avoid numbers or underlines: you want others to be able to remember it and type it easily. You can be anonymous if you wish, but I’d not recommend it: you are more likely to have interesting interactions with others if they know who you are. A brief description of what you do and what your interests are will help kindred spirits discover you. You get the chance to select your avatar, a little picture that appears alongside your tweets. It’s a good idea to have something other than the default picture of an egg - if you don’t want a photo of yourself, you can pick something symbolic, but aim for something to give yourself a distinctive presence. If you want, though, you can start with the egg and change it later.

How do I decide who to follow?

I started out by following my old friend and colleague Sophie Scott, or @sophiescott as she is known on Twitter. We have similar interests and a similar sense of humour, and so the first thing I did was to see who she was following. You can check out someone’s followers by clicking on their username at the top of a tweet. You’ll see their profile on the right hand side, with an indication of how many followers they have, and who is following them. Further clicking lets you see who these people are, and read their recent tweets. So it’s easy to get an idea of whether you’d like to see their tweets on a regular basis: if yes, a single click allows you to follow them.

The people I follow divide mostly into (a) organisations/public media, such as those mentioned above; (b) academics who work in areas that interest me; (c) journalists and bloggers. Although I have friendly relationships with many of those I follow, I don’t use Twitter as a means of keeping up with friends - it’s too public and the short message format is not useful for that.

I suggest you start out by just identifying a few people that look interesting to follow, and see whether you enjoy the Twitter experience. My recommendation would be to keep the number of people you follow restricted to no more than around 100. Many people follow far more than this, but I like my twitterstream to move at a reasonably sedate pace.

Getting fed up with tweets from someone you’re following? You can just unfollow them. They don’t get a message about this, so you can do it without embarrassment.

Active tweeting and attracting followers

You can have most fun with Twitter if you tweet yourself. For the beginner, there is a major problem: if you emit a tweet, the only people who will see it are your followers, and at the outset you have no followers. You may have something very amusing to say, or a really interesting paper just published, but it’s like standing at the top of a cliff and shouting into the wind. To get started, it helps to either be well-known, or to have tweeting friends. You can look for friends and colleagues by clicking on the ‘who to follow’ button, and if you find they have a Twitter presence, send them an email with your username to let them know you are there. With luck they’ll follow you, and tell others about your presence. It’s only worth doing this, though, if they are active Twitter users with followers: lots of people sign up but never use Twitter.

You may also drum up followers by following others. This is where it is important not to be too secretive: if I get a new follower, I’ll see their name and the brief bio that comes along with it, and if they look interesting, I may check out what they’ve been tweeting to see if I want to follow them. Twitter etiquette does not require that you follow someone just because they follow you, but following someone is a way of indicating your presence to them.

Another way to draw your tweeting to people’s attention is to use hashtags in your tweets. These act as keywords and are just words with the hash sign attached at the front, such as #neuroscience or #psychology. People who are searching on these topics will find your tweets and may decide to follow you.

If you are sending interesting tweets, the message will spread around the twittersphere and you will gradually get a following. You may wonder how on earth you are supposed to generate those interesting tweets that will persuade people to follow you. You don’t always have to. You can act as a transmitter for other people’s interesting tweets, by using the Retweet button below the tweet. This will just resend the tweet to your followers, preceded by RT and your username.

You should not despair if at first you don’t have many followers. Although it’s true that a famous name will attract followers in droves, there are plenty of people who aren’t famous, but who have hundreds or even thousands of followers just because they give good value. And at the end of the day, you should not get too hung up on follower numbers. The charm of Twitter is that it lets you reach out to communicate with people all over the globe whom you might otherwise never encounter: a handful of like-minded people who appreciate your tweets is more important than a horde of followers who seldom read your messages.

What about spam?

Many newbies are worried that they will get followed by odd people. That certainly will happen. But the nice thing is that it has no impact on you. I attracted lots of provocatively dressed pouting followers when I started out. But they can follow me as much as they like; they won’t affect my stream of incoming tweets unless I follow them back. Various unsavoury characters will appear as followers for a day or two and then drop away. They hope that by following you you’ll take notice of them and buy whatever product they are purveying, but you just ignore them and they go away. Twitter discourages users who simply see it as a marketing opportunity, and is set up so you can readily report people for spam if it looks like they are doing that, but mostly the only bad thing that happens is that you have a fleeting moment of excitement at gaining a new follower, only to be disappointed to find they are someone who sees you as a potential client.

The one place where you may get more intrusive spam is if you press the @Mentions button at the top of the screen. Now, instead of your usual stream of tweets by followers, you will see just those tweets that mention you by username, and these will not necessarily be by your followers. In general, if you get people mentioning you in tweets, this creates a warm glow that others are interested in your tweets, but there are people who will try to exploit this, and so you may find tweets that mention your name in a tweet to lure you in to clicking a link to their website. In my experience, these are very rare, and when they occur they are usually easy to spot: if you click on their username, you’ll see they’ve sent the same message to many others. You should just report them for spam.

What should I tweet?

Quite simply, tweet to others things you think will interest them. Looking at tweets by others should give you an idea of what makes for a good tweet. Some very famous people are hopeless tweeters, because they just describe the mundane details of their life. What I actually want are either amusing observations, or useful information. Some people use Twitter to record their stream of consciousness. Unless you’re James Joyce, this is very dull for everyone else, and just makes you look egotistical. Come to think of it, James Joyce would have been a disaster on Twitter.

When you start tweeting, the 140 character limit seems impossible, but you learn by experience. If you want to include a link to a website in a tweet, you will almost certainly need to shorten it. There are various programs for doing this, e.g.

http://tinyurl.com/

http://bit.ly/

As mentioned above, if I read a tweet by someone that I think will interest my followers, I'll retweet it. That’s a single click operation, one of the options given below each tweet. Retweeting is what makes Twitter such an effective communication medium: if an interesting message is retweeted by several people with many followers, who in turn retweet to their followers, it can rapidly spread all over the world.

If you’ve read this far, you’ll start to appreciate that tweets are a kind of currency; your status on Twitter is tied up with the extent to which you emit popular tweets. It is therefore as important to acknowledge the source of a good tweet as it is to reference an idea in a scientific article. If you use the retweet function, this happens automatically: the retweeted tweet will be marked RT and will show both your name and that of the originator. What’s a definite no-no is to copy someone else’s tweet and resend it without acknowledging the source. Etiquette is less clear on the extent to which one should send a tweet to thank others who promote one’s tweets. This may seem polite, but if over-used, it can descend into a rather irritating form of self-promotion - in effect you are publicly drawing attention to the fact that others liked your tweet. A lot depends on how it’s done, but there’s a narrow line between being seen as courteous, and coming across as a self-congratulatory dick.

How much should you tweet? I’m more likely to unfollow someone who tweets too much rather than too little. Anyone who keeps repeating the same self-promotional message is quickly dropped from my list. If you have a blog or article you want to promote, it’s reasonable to plug it a few times on different days and at different times of day, to make sure the message gets out, but you’ll turn people off if you overdo it. If it’s interesting enough, your followers will do the work of promotion for you, by retweeting. On the other hand, there's not much point in following someone who tweets less than once a week, unless their tweets are really something special.

Remember, Twitter is totally public. I can search for someone’s name and then look at all their tweets. I would therefore strongly advise against tweeting anything at all that you would not want your friends and colleagues to see, or that could be deemed defamatory. There is a Delete option you can use on tweets that you come to regret, but by the time you select it, your tweet could have been sent all around the world.

How do I find the time?

It’s a big mistake to think you have to read every tweet that appears in your stream. I just turn to Twitter when I need distraction or entertainment. So you really don’t need to spend very long on Twitter unless you want to. The difficulty is that what’s happening on Twitter is often more interesting that what’s happening in other areas of life, and it can become quite addictive for that reason. I usually resolve not to look at Twitter during the working day, especially if I have a paper to write or an analysis to run. But sometimes the resolve faulters.

I hope that’s enough to persuade you to give Twitter a try. Happy tweeting!

P.S. 18th June 2011

The Reply Option

I’ve been asked about the Reply option that appears when you hover over a tweet.

You can use this even if you aren’t following someone. Your message will automatically be prefaced by the username of the person you reply to. It won’t appear in their regular stream of tweets, but they will see it if they look under @Mentions. This can be a way of getting yourself involved in a discussion, even if you don’t have followers. And if the discussion develops well, the person you reply to may decide to follow you.

Twitter won’t show your Reply to your followers unless they are also following the person you reply to. Complicated, huh? The reason is that otherwise your followers will be treated to just one side of a personal conversation. But if you think your reply may be of interest to a broader audience, you can make it visible to all by just putting a full stop in front of the @ at the start of the Reply. (In fact, any character will do, but . is traditional. The rule is that provided tweets don’t start with @ your followers will see them).

P.P.S.28th October 2011
I've written a short piece about blogging for academics that may interest readers of this post.

Brain scans show that.....

2011-06-11T20:42:00.000+01:00

I was set off today by a report that “fMRI scans prove music is more emotionally stimulating if you listen with your eyes closed”. What’s wrong with that? Well, if I want to know if music is more stimulating with the eyes closed, the sensible experiment would be to ask people to report their emotions while listening to music with eyes open or shut. A brain scan gives you a 3D representation of how regional blood flow changes over time when a person performs a particular task or experiences a specific mental state: it does not measure emotional stimulation.

It seems every week we have another claim that brain scans have shown something about our cognitive or emotional characteristics. To see how accurate this perception was, I tried Googling the phrase “brain scans show” or “brain scans prove”. Within minutes I had a long list of hits, most of which made unsupportable claims.

Neuroscientists have for many years been interested in documenting how brain activity in different brain regions correlates with mental activity or mental states. For some mental states, we know enough to say that when we do cognitive operation X, or experience emotion Y, brain region Z is usually activated. But what we can’t say is that if we see activation in Z, this is equivalent to cognitive operation X or emotion Y. For a start, most mental activities involve a whole network of brain regions acting together, and there can also be substantial variation from one person to another in brain structure and function. But more seriously, there is a logical problem at the heart of this argument. Consider not the brain but the skin. When I am anxious, I tend to sweat. If you observe that I am sweating, you may suspect that I am anxious. But I could be sweating for other reasons: I may have been exercising, the room may be hot, or I may have been eating chillis or taking medication. You could test your theory that I am anxious by asking me. Suppose I deny being anxious: would you believe me? Maybe not: perhaps I am really anxious but reluctant to admit it. But you’d be foolish to regard sweating as a more reliable indicator of my emotion than my verbal report unless there was good reason to believe I might be lying. You’d be even more foolish to assume that I’m not anxious if I’m not sweating. Sweating is a correlate of anxiety (and an unreliable one at that): it is not the same thing as anxiety.

But it seems that in the media, there is a belief that activity from a brain scan provides a direct measure of specific mental states; not only that, the brain scan is seen as a more reliable indicator than traditional behavioural measures. You may find it instructive to consider the claims made in the links below, and then consider how they’d look if you took away the phrase ‘brain scans show”. In general, you can see that the brain scan adds nothing: you could test each of these notions with behavioural data. It’s also worth considering how you’d react if a behavioural study and a brain scan study obtained opposing results. A prudent response would be to trust the behaviour. I've got nothing against fMRI studies as a method for studying brain-behaviour relationships: my criticism is for those who interpret the brain activity as if it was the behaviour. As Neuroskeptic put it in a post last year, “Brain scans prove that the brain does stuff”.This was the one Google result that cheered me up. Here are the others:

Future angst? Brain scans show uncertainty fuels anxiety

Brain Scans Show How Multitasking Is Harder for Seniors

Brain scans show children with ADHD have faultyoff-switch for mind-wandering

Brain Scans Show How Teens Are More 'Me-First' ThanAdults

Brain scans show how a disorder leads individuals to perceive themselves as ugly

Brain scans show meditation changes minds, increases attention

Rejection Really Hurts, Brain Scans Show

Brain Scans Show That Teens Don't Really Care About You

Brain Scans Show Married Love Can Last

Brain Scans Show Twins Think Alike

Brain Scans Show Food Addiction

Brain Scans Show Dyslexics Read Better with Alternative Strategies

Brain Scans Prove Branding Strength

Brain scans prove ‘that loving feeling’ truly is addictive

Brain Scans Show Bullies Enjoy Others' Pain

Ex-smokers are willpower kings, brain scans show

Acupuncture reduces pain perception, brain scans show

Brain Scans Show Women Feel More Pain Than Men

Brain Scans Show Why Teens Give Into Peer Pressure

Review of "How Genes Influence Behavior "

2011-06-05T09:02:00.000+01:00

by Jonathan Flint, Ralph J. Greenspan and Kenneth S. Kendler

Oxford University Press, 2010

One would expect an introductory textbook by these three eminent scientists to be good, and it is. It covers behavioural and molecular methods, using examples from the authors’ own work in psychiatric genetics. The field is put in historical context, with the false turnings explored, as well as the major breakthroughs. The authors take us on a journey through family, adoption and twin studies, on to the history of molecular psychiatric genetics, starting with the early, and usually unreplicated, linkage studies, through genome wide association studies and animal models using rodents and drosophila. Concepts are explained with a minimum of technical and statistical detail, making the book accessible to a wide readership. What sets this book apart from others on this topic are the personal and often witty accounts of doing research. Kendler’s anecdote of a family study of schizophrenia in Ireland is particularly engaging, including an encounter with an elderly grandmother who expressed indignation that he should be driving around all over the place interviewing people to find out if mental illness ran in families. “’Why everybody knows that! Take the O’Donnells for example. They are as mad as can be and it goes back generations.” (p. 12). One also gets a sense of the hard graft involved in doing research in this area: Kendler estimated that his study took 18 person years just to gather the data, and evidently required persistence and a willingness to work heroically antisocial hours by the research team.

Chapter 5, on genetic association analysis, is a sobering account of the ‘quagmire’ resulting from thousands of pounds worth of funding directed to finding genes that might explain psychiatric conditions. I’d recommend this chapter to those looking for (nongenetic) associations between individual differences in behaviour and brain markers: I see close analogies between the problems of molecular genetics and those found in neuroscience, where the sheer quantity of data makes it easy for spurious associations to be mistaken for real effects. So far neuroscientists have barely confronted this issue, though it’s pleasing to see Kriegeskorte et al (2009) making a start.

Anyone contemplating a research career in molecular genetics would do well to read the paragraph on p 34-35 debunking popular assumptions about life in the laboratory: “A combination of intelligence and technology applied to taxing but interesting biological problems leads to ground-breaking discoveries that could cure disease or change our understanding of the universe.” Well, no. The work is dull, repetitive, and usually unsuccessful. And if after years of work you do find something interesting, your laboratory head will ask: “Why is this finding wrong?”. In fact, I suspect only really good laboratory heads do that - the bad ones rush gleefully into print, which is why the field is littered with nonreplicable findings. But as Flint et al point out, you need to ask it because it’s embarrassing to publishing something that is wrong, your peers will do their best to find flaws in the work, and, most importantly, “it’s only by publishing findings that are robust to any possible criticism that we’ll make any progress”. A refreshingly old-fashioned take on the scientific process that is all-too-often forgotten in the current climate where we’re all encouraged to publish as much as we possibly can, and where a dramatic but non-replicable result may get you two papers in a high-impact journal: one for the original finding, and the other for the failure to replicate.

Reference

Kriegeskorte N, Simmons WK, Bellgowan PS, & Baker CI (2009). Circular analysis in systems neuroscience: the dangers of double dipping. Nature neuroscience, 12 (5), 535-40 PMID: 19396166

Is poor parenting really to blame for children's school problems?

2011-06-03T17:43:00.001+01:00

It’s tough being a parent of a child with developmental problems. You coo over your baby, anticipating how they’ll grow into a cheerful and contented individual, only to find that the older they get, the more the problems become apparent. Your child may be late starting to talk, and then communicate with difficulty, may struggle with reading, be clumsy, hopeless at maths, or poor at making friends. In many cases, several of these problems occur together. Gradually, as life throws more challenges at him or her, your charming toddler turns into an anxious and unhappy child, reluctant to go to school, fearing failure at every turn, and attracting ridicule from other children.

What you really don’t need at this point is to be told that it’s all your fault. Yet there is a band of self-appointed ‘experts’ who seem determined to keep this message alive, and to add guilt to the other emotions that parents experience. I know of mothers who think that their child’s problems are due to their having an occasional drink during pregnancy, going out to work when their child was young, or allowing them to eat food with e-numbers. And the media absolutely lap it up. Berating feckless and inadequate parents is a popular sport that attracts readers who can wallow in a sense of complacency that their child isn’t like that.

The latest in this line of commentators is Sally Goddard Blythe, who is described as Director of the Institute for Neuro-Physiological Psychology and child development expert. In her new book ‘The Genius of Natural Childhood’ she claims that many children are failing to develop vital physical and communication skills needed to start school, because of a lack of interaction between parents and children during the early years.

According to the Daily Telegraph “growing numbers of parents are turning to electronic toys and automatic baby walkers or rocking chairs to keep toddlers occupied. But this often impairs children’s natural development, meaning many are wrongly labelled as suffering behavioural problems when they start compulsory education”. The book notes that singing lullabies and using music gets a baby ready for language at a young age, while reading fairy tales can teach children moral behaviour and empathy. Finally, “many early developmental problems could be overcome with old fashioned one-on-one interaction between parents and children.” The account in the Daily Mail was even more stark: Poor parenting means half of five-year-olds are unfit to start school shouted the headline, with the text continuing, “Children are failing to develop vital physical and communication skills after being robbed of interaction with mothers and fathers during the early years.” Interestingly, in an interview yesterday on BBC Radio 5 Live, Blythe seemed to change tack and back off from blaming parental communication, focusing more on apparently alarming rates of motor skill deficits in primary school children. Nevertheless, links to the Mail and Telegraph articles feature prominently on her blog.

So what’s the basis of Blythe’s claims? Let’s look first of all at her credentials, as given on her website. Director of an Institute sounds impressive, but less so when it turns out to be a private self-funding institution. She has letters after her name, including MSc and FRSA. I couldn’t readily find what subject the MSc is in, but FRSA is Fellow of the Royal Society of Arts. It’s not a particular mark of distinction: you can apply to become a fellow on their website, provided you are willing to pay £150 per year and can give evidence that you “share or demonstrate a commitment to positive social change in professional, civic or personal life”. She also describes herself as a “freelance consultant in neuro-developmental education”. That fancy-sounding language gives no validation - anyone can set themselves up as a consultant. It only carries force if it is a title awarded by an organisation with some credentials, such as a university or a hospital.

Never mind. Maybe she has good data to back up the statements about parenting. Well, I've not yet had a chance to look at the book, so perhaps I will be pleasantly surprised, but I doubt it. I read widely on this topic and I know of no credible scientific support for the claims. I searched Harzing’s Publish or Perish for articles by Sally Goddard Blythe. Since 1998 she has had three papers in peer-reviewed journals, with a total of 41 citations between them. None of the papers involves research on effects of parenting. Two books were also listed, but since these aren’t peer reviewed, they wouldn’t normally be regarded as evidence of scholarly contribution, unless supported by significant peer-reviewed work. There was also a piece in an journal with the bold title Medical Veritas (The Journal of Medical Truth). I’d not come across this one before, and it’s not featured on Web of Knowledge, but suffice it to say that a high proportion of articles are concerned with vaccination, and Dr Andrew Wakefield is an editor. This paper also had no data on parenting.

One reason why this story hit such a nerve with me is that I’ve researched causes of children's language and communication difficulties for many years, and, contrary to popular opinion, parental behaviour does not seem to play an important role. There are converging lines of evidence that can be considered. The weakest kind is correlational: if you find that aspects of parenting correlate with the child’s language development, you can’t be sure of a causal link; there could be a third variable that simultaneously influences both the parenting and the child’s language. Nevertheless, this approach does allow you to assess the plausibility of potential causal routes. A recent epidemiological study looked at factors associated with late language emergence (LLE) in toddlers in Australia, and concluded “Risk for LLE at 24 months was not associated with particular strata of parental educational levels, socioeconomic resources, parental mental health, parenting practices, or family functioning. Significant predictors included familial history of LLE, male gender, and early neurobiological growth.” This study concluded that neurobiological factors and genes were important in determining which children had language difficulties. A more powerful design is the twin study, because it allows some dissociation of genetic and environmental influences. If environmental factors are important, then you’d expect two twins growing up together to resemble one another. However, twins share genes as well as environments. More specifically, we can distinguish identical twins, who are genetically the same, and non-identical twins, who are as similar as regular brothers or sisters. One of the first things to strike me when studying twin pairs where at least one twin had language difficulties was how different two twins from the same family could be - provided they were genetically non-identical. This impression was borne out by statistical analyses of twin data, which repeatedly find that the similarity of language abilities between two twins growing up together is strongly influenced by their genetic relationship (see Bishop, 2006). A further piece of evidence is obtained by turning the question on its head: if children require a certain amount of talking and singing to develop normal language, what happens to children raised in unusual environments where they have little exposure to such stimulation? Some hearing children of deaf parents grow up in circumstances where they are exposed to little intelligible speech except from a TV. When such children were first studied, everyone predicted they’d have serious language difficulties. Instead, the studies demonstrated just how robust language development usually is: most hearing children of congenitally deaf parents learn to talk quite normally (Schiff-Myers, 1998). I would not want to claim that parental behaviour is unimportant. Interactions between genes and environments make their effects difficult to disentangle (see earlier blogpost). I'd cheerfully go along with encouraging parents to sing to their children and read books. But it's simply not realistic to attribute children's school difficulties to lack of interaction with parents.

Mrs Goddard Blythe is entitled to her views. My concern is with the blurring of the distinction between opinion and evidence. When a view about effects of parenting is widely promulgated on national media, and is expressed by someone who is described as a consultant in neuro-developmental education and Director of an Institute, the natural assumption is made that (a) they are speaking from a position of authority, and (b) they have some hard evidence. In this case, neither appears to be true.

References
Bishop, D. V. M. (2006). What causes specific language impairment in children? Current Directions in Psychological Science, 15, 217-221.

Schiff-Myers, N. (1988). Hearing children of deaf parents. In D. Bishop & K. Mogford (Eds.), Language development in exceptional circumstances (pp. 47-61). Edinburgh: Churchill Livingstone.

Zubrick, Stephen R., Taylor, Catherine L., Rice, Mabel L., & Slegers, David W. (2007). Late language emergence at 24 months: An epidemiological study of prevalence, predictors, and covariates Journal of Speech, Language and Hearing Research DOI: 10.1044/1092-4388(2007/106)

Are our ‘gold standard’ autism diagnostic instruments fit for purpose?

2011-05-30T17:13:00.000+01:00

In 1985, Simon Baron-Cohen, Alan Leslie and Uta Frith published a landmark paper entitled “Does the autistic child have a theory of mind?” It described a small study that Simon Baron-Cohen completed for his doctoral thesis which, according to Google Scholar, has been cited 2800 times. If the same paper were submitted for publication today, most journals would reject it. Why? The paper stated: “The 20 autistic children had been diagnosed according to established criteria (Rutter, 1978)”. Nowadays this would be deemed inadequate. Many editors and reviewers insist that studies of autism use two diagnostic procedures, the Autism Diagnostic Interview - Revised (ADI-R) and the Autism Diagnostic Observation Schedule - Generic (ADOS-G). According to the NIH National Database for Autism Research, requiring people to use these “gold standard” assessments will “help accelerate scientific discovery”. But use of these instruments adds hugely to the time and money costs of research. Small-scale studies of autism by PhD students have become nonviable, and large-scale genetic and epidemiological studies are bogged down by the need to spend hours just establishing the phenotype for each case. Researchers from countries where ADI-R and ADOS-G are not available are at a serious disadvantage. And, as I shall argue, the end result is not a clearcut diagnosis.

Autism has three key defining features: impairments in communication, social interaction and behavioural repertoire. The latter encompasses both repetitive behaviours such as stereotyped movements, and restricted interests, e.g., an obsessive fascination with aeroplanes. After autism was first described by Leo Kanner in 1943, the diagnosis quickly became popular, but there were concerns that it was over-used. There was a clear need to translate Kanner’s clinical descriptions into more objective diagnostic criteria. The first step was to develop checklists of symptoms, and these were included for the first time in the 1980 version of the Diagnostic and Statistical Manual of the American Psychiatric Association, DSM-III. However, this still left room for uncertainty: clinicians might, for instance, disagree about interpretation of terms such as: “Pervasive lack of responsiveness to other people”.

The Autism Diagnostic Interview (ADI) was designed to address this problem. It was first published in 1989, with a revised form, ADI-R, appearing in 1994. The ADI-R typically takes 1.5 to 3 hours to administer, and covers all the symptoms of autism and related conditions. Items are coded on a 4-point scale, from 0 (absent) to 3 (present in extreme form). The scores from a subset of items are then combined to give a total for each of the three autism domains, and a diagnosis of autism is given if scores on all three domains are above cutoffs, and onset was evident by 36 months. Validation of the ADI-R was carried out by comparing scores for 25 preschool children who had clinically diagnosed autism and 25 nonautistic children with intellectual retardation or language impairment.

Right from the outset, however, there was concern that diagnosis of autism should not be made on the basis of parental report alone. Some parents are poor informants. On the one hand, they may fail to remember key features of their child’s behaviour; on the other hand, their memories may have been coloured by reading about autism. Parental report therefore needs to be backed up by observation of the child. The Autism Diagnostic Observation Schedule (ADOS), published in 1989, was designed for this purpose. It exposes the child to a range of situations designed to elicit autistic features, and particular behaviours, such as eye contact, are then coded by a trained examiner. ADOS-G, a generic version, was published in 2000, and covers a wide age range, from toddlers through to adults.

ADI-R and ADOS-G quickly became the instruments of choice for autism diagnosis. It was generally appreciated that if we use standard instruments, researchers and clinicians should be able to communicate about autism with a fair degree of confidence that they are referring to individuals who meet the same diagnostic criteria.

There is, however, a downside. ADI-R and ADOS-G were designed to be comprehensive, but they were not designed to be efficient. As noted above, ADI-R takes up to 3 hours to administer and score. ADOS-G takes about 45 minutes. In addition, testers must be trained to use each instrument, and it may take some months to find a place on a training course. Each course lasts around one week, and the trainee then has to do further assessments which are recorded and sent for validation by experts. This process can easily add another 6 months. For anyone under time pressure, such as a doctoral student or grant-holder with research assistants on fixed term contracts, the training requirements can make a study impossible to do. Inclusion of both ADI-R and ADOS-G in a test protocol can double or treble the duration of a project, especially where it is necessary to travel to interview parents who may be available only during anti-social hours.

So, does autism diagnosis need to involve such a lengthy process? This was a question I was prompted to consider when I was asked to speak at a roundtable debate on diagnostic tools for autism at the International Meeting For Autism Research (IMFAR) in London in 2008. I concluded that the answer is almost certainly no. As Matson et al (2007) put it: “Some measures emphasize the fact that they are very detailed. We would argue that detail equals time. From a pragmatic perspective, our view is that a major priority should be to develop the balance between obtaining relevant information to make a diagnosis, while parsing out items that do not enhance that goal”. (p. 49). I was surprised when I first undertook ADI-R training to find that the interview included many items that did not feature in the final algorithm. When I (and others) queried this, we were told that the interview worked in its entirety, and to pull out selected items would disrupt a natural flow. Also, the non-algorithm items might be useful for diagnosing conditions other than autism. While both points may be important in a clinical setting, they have much less force for the poor graduate student who is doing a doctorate on, say, perceptual processing in autism, and only wants to use the ADI-R to confirm a diagnosis that has already been made by a clinician. There was a short form, I was told, but it was not recommended and should only be used by clinicians, not researchers. I was even more surprised to find how the algorithm was devised. I was familiar with discriminant function analysis, whereby you take a set of scores on two (or more) groups, and find the best weighted sum of scores to discriminate the groups. You can then use the correlations between items to drop items from the algorithm successively until you get to the point where accuracy of diagnostic assignment declines if further items are dropped. I had assumed that this kind of statistical data reduction had been used to identify the optimal set of items for identifying autism. I was wrong. It seemed that the items were selected on the basis of their match to clinical descriptions of autism, and no attempt had been made to test the efficiency of the algorithm by dropping items.

There is good reason to believe that a much shorter and simpler procedure would be feasible. In 1999, a study was published comparing diagnostic accuracy of the ADI with a 40-item screening questionnaire. The accuracy of the questionnaire was as good as the interview. My impression is that this finding did not lead to rejoicing at the prospect of a shorter diagnostic procedure, but rather alarm that diagnosis could be reduced to a trivial box-checking exercise. While I have some sympathy with that view, I feel these results should have made the researchers pause to consider whether a much shorter and more efficient approach to diagnosis might be feasible.

But the problems get worse. As Jon Brock recently pointed out on his blog, Kanner’s view of autism as a distinct syndrome is no longer accepted. It’s clear that autism symptoms can occur in milder form, and that they do not necessarily all go together. The broader term ‘Autism Spectrum Disorder’ (ASD) is nowadays used to encompass these cases. The schematic illustration in Figure 1 illustrates the diagnostic problem: one has to decide where to place boundaries on the figure to distinguish ASD from normality, when in reality, all three domains of the autism triad of symptoms shade into normality, with no sharp cutoffs. The ADI-R algorithm specifies whether or not you have autism, and does not give cutoffs for milder forms of ASD. The ADOS-G does have cutoffs for milder forms, but is inappropriate for detailed assessment of repetitive behaviours/restricted interests, so it is not watertight. This means that, when it comes to diagnosis of ASD, a great deal is left to clinician’s judgements. These, rather, than algorithm scores, are used to arrive at diagnoses.

Figure 1: Schematic of autism as a spectrum disorder: Circle correspond to areas of deficit, red = social impairment, blue = communication difficulties; yellow = repetitive behaviour/restricted interests, with depth of colour indicating severity of impairment. Individuals with all three features (centre of the figure) meet full diagnostic critieria for autism, but those falling outside this region, who have milder or partial difficulties are candidates for a diagnosis of autism spectrum disorder. Note, however, there is no clearcut boundary between autistic spectrum disorder and normal variation.

There were quite a few researchers at the IMFAR meeting who complained that their papers were rejected by journals because they relied on a diagnosis made by an expert, rather than ADI-R and ADOS-G. They would be baffled to see that several recent state-of-the-art epidemiological studies use consensus judgement by expert clinicians to make their diagnoses - and these diagnoses don’t necessarily agree with ADI-R and ADOS-G. So, for instance, Baird et al had 81 cases who met a consensus clinical diagnosis of childhood autism, but only 53 (65%) were classified as autistic by the algorithms of both the ADOS-G and the ADI-R. They also identified 77 children with consensus diagnosis of ‘other ASD’ of whom 69% met criteria for autism on the ADI-R, and 38% met cutoff for PDD or autism on the ADOS. On the ADOS-G, 10% of non-autistic children scored above cutoff for either ASD or autism. This fits my experience: high scores can reflect lack of engagement, shyness, or language difficulties. Similarly, Baron-Cohen et al identified four cases of autism and seven with other ASDs in a population screening of children aged 5 to 9 years in Cambridgeshire. All the autism cases met autism criteria on both ADI-R and ADOS-G. Of the other ASD cases, five met criteria for autism on ADI-R but not ADOS-G, and two did not meet criteria for autism or ASD on either instrument. In describing these findings, I am not criticising the authors of these studies, whose methods were transparently reported and are consistent with practice as it has evolved in the field. But it is ironic that we seem to have come full circle. ADI-R and ADOS-G were developed to make diagnosis more objective, but because they aren’t geared up to diagnose ASD, we are thrown back on ‘expert clinical opinion’. This is far from reassuring, given that a recent study reported that, after months of training, researchers agreed well on scoring standardized instruments, but “consistent differences between sites in overall clinical impression were reported”.

My own recommendation is for a two-step procedure. The first step would involve a much briefer version of the ADI-R, which would be designed to pick up clear-cut cases of autism that everyone would agree on and distinguish them from clearly non-autistic cases. It’s an empirical question, but I suspect that if we were to do a stepwise discriminant analysis to identify a minimum set of diagnostic items, this would be considerably shorter than the current set used in ADI-R. The interview may then need redesigning so that it still flows fluently and follows a logical course, but this should not be impossible. In clinical settings, those identified might require further direct assessment to confirm diagnosis and identify specific needs, but this would not be necessary for determining who should be included in a research study. This would leave a group of children in whom autism was suspected but not confirmed. The question here is whether we will ever arrive at a diagnostic procedure that will clearly separate such children into ASD and non-ASD. I was involved in a study with just such a group of ‘marginal’ cases a few years ago, where we administered ADI-R and ADOS-G. The results were all over the place: some children looked autistic on ADI-R and not on ADOS-G and others showed the opposite pattern. Some had evidence of marked change in behaviour between preschool and school-age years. When I asked an autism expert how such cases should be categorised, he suggested I get an expert clinical opinion. Yet expert clinical opinion is not seen as adequate by many journal editors! And there is documented evidence that even experienced clinicians will disagree in cases where the child has a confusing pattern of symptoms, and that expert diagnoses are not stable over time. My suggestion is that in our current state of knowledge it makes no sense to try and get reliable cutoffs for identifying ASD. Instead we should aim to assess the nature and degree of impairments in different domains. Assessments such as the 3Di or Social Responsiveness Scale, which treat autistic features as dimensions rather than all-or-none symptoms, seem better suited to this task than the existing gold standards.

Finally, I must emphasise that, although I think the ADI-R and ADOS-G are not optimal for diagnosing ASD for research purposes, they nevertheless have value. They distill a great deal of clinical wisdom in the assessment process, and are cleverly crafted to pinpoint the key features of autism. Anyone who undergoes training in their use will come away with a far greater understanding of autism than they had when they started. However, these instruments are not well suited for addressing the NIH aim of “accelerating scientific discovery”. In research contexts they have the opposite effect, by making researchers go through an unnecessarily long and complex diagnostic process which does not yield suitable quantitative results for assessing the dimensional aspect of ASD.

Rondeau E, Klein LS, Masse A, Bodeau N, Cohen D, & Guilé JM (2010). Is Pervasive Developmental Disorder Not Otherwise Specified Less Stable Than Autistic Disorder? A Meta-Analysis. Journal of autism and developmental disorders PMID: 21153874

Scientific communication: the Comment option

2011-05-25T12:30:00.000+01:00

1. Think of something interesting to say

2. Hit the link saying ‘comment’.

3. Find you have to register. Enter first name, surname, title, job description, qualifiations, place of work, phone number, fax number, address.

4. Spend several minutes looking for England, then Great Britain on the drop-down list, before hitting on United Kingdom. Press enter, while wondering how often someone from Azerbaijan makes a comment.

4. Back to step 3 because failed to enter a zip code in the correct format.

5. Realise that the special security software that was installed to prevent unauthorised access to the computer has barred interaction with the site, and everything you have entered has been lost.

6. Select ‘allow this site’ in the security software.

7. Back to step 3.

8. Select a password.

9. Retype the password.

10. Retyped password fails to match password. Back to step 8.

11. Hooray! All entered.

12. Try to log in. System tells you someone with this email address has already registered with a different password.

13. Alarm husband with sudden outburst of profanities and table thumping.

14. Try to guess password.

15. Fail.

16. Loop back to step 14 several times.

17. Request new password.

18. Open email to look for message re new password.

19. No sign of email re password. Get distracted by other emails. Ponder on why as a neuropsychologist have been invited to contribute an article to Journal of Plant Membranes.

20. Delete messages with header ‘Dear Friend in Christ’. Don’t they realise I’m an atheist?

21. Password message appears in Inbox.

22. Back to login screen.

23. Login, only to find there is a saved password already.

24. Use of saved password gives error message.

25. Relogin with new password. Yes!

26. Search for original site for comments. Can’t find it.

27. Back to twitter to find the address of the original article that I wanted to comment on.

28. Find the article.

29. Hit the comment button.

30. Now what was it I wanted to say?

Autism diagnosis in cultural context

2011-05-16T07:42:00.001+01:00

A review of Isabel’s World by Roy Richard Grinker

In 1993, when Roy Grinker’s daughter, Isabel, was two years old she did not talk or gesture, flapped her hands and arms, and did not make eye contact. At 32 months, she had mastered about 70 words, which she spoke clearly, but all were nouns and the list did not include ‘mommy’ or ‘daddy’. She didn’t say ‘yes’ or ‘no’ and pulled someone to the refrigerator when she was hungry. Nowadays it’s hard to imagine a paediatrician would fail to recognise classic symptoms of autism, but for the Grinkers the process of getting a diagnosis was protracted and painful. At best they met with ignorance, and at worst with professionals who were imbued with the ideas of Bruno Bettelheim and attributed Isabel’s problems to the fact that her mother went out to work. Once a diagnosis was obtained, the Grinkers still had to fight for appropriate educational provision. They came up against people who had no experience of autism and were unwilling or unable to engage with Isabel. Gradually, though, the battle was won. Isabel, whose high nonverbal ability was eventually recognised, was able to attend a mainstream school with support. Now in her teens, she still has major problems with communication and social interaction, and her parents accept she will not able to live independently. She has, however, found a niche in her community where she has friends, can enjoy her interests in music and animals, and is accepted for who she is.

Grinker’s book is more than a parent’s first-hand account of his daughter’s autism: it is also influenced by the fact that he is a cultural anthropologist, with a special interest in attitudes to health and disability in different parts of the world. He uses his professional insights to comment on two related issues: the rise in autism diagnosis in the USA, and attitudes to autism in other cultures, particularly France, Korea, South Africa and India.

Figure 1

The rise in autism diagnosis is a hotly debated topic. In general, there is agreement about the facts: the number of children receiving an autism diagnosis in the USA has risen sharply since the 1970s. The data in Figure 1 were taken from a website relating to the Individuals with Disability Education Act (IDEA), and they show a more than four-fold increase in children identified with autism in the ten year period between 1997 and 2006. I have previously noted in a PLOS One article that research on autism, as assessed by funding and publications, has skyrocketed over the same period.

The debate is over the reason for the increase: is it due to some environmental factor that causes autism, or can it be explained simply in terms of changes in diagnostic criteria and related factors? Grinker comes down solidly in favour of the latter explanation, and a substantial part of the book is devoted to tracing the history of autism diagnosis and related socio-cultural factors. A major turning point was the publication in 1987 of the third revision of the diagnostic and statistical manual of the American Psychiatric Association, DSM-III-R. Grinker cites a study of 194 children suspected of autism which found that whereas 51% met criteria on DSM-III, 91% met criteria on DSM-III-R. He also reports a typographic error that was made in the publication of DSM-IV in 1994 that led to the ‘autism spectrum’ disorder of PDDNOS being defined on the basis of a child having impairments in social interaction or verbal/nonverbal communication skills, when it should have required impairment in both domains.

Grinker tackles a question often asked by those who think the autism epidemic is genuine: if it’s just due to a change in diagnostic practices, where were all the undiagnosed autistic children in the past? Surely we would have noticed them? On the basis of a follow-up of a small sample of UK children with severe language impairments, our group found cases who would now be regarded as unambiguously autistic, but who were diagnosed as language-impaired when they were seen in the 1980s, prior to DSM-IIIR. So are we just engaging in 'diagnostic substitution', whereby the same child who is now regarded as autistic was previously given a different label? If so, we'd expect to see that as autism diagnosis goes up, language disorder diagnosis goes down. This would be consistent with an analysis of doctor's records in the UK, which found that the proportion of children with diagnoses such as speech/language disorder went down over the same period as autism diagnoses went up. In the IDEA data, however, I found no evidence for such a process: over the 1997-2006 time period, there's actually a slight increase in diagnoses of speech/language diagnoses. Grinker, however, suggests that, in the US, children with autism would, in the past, often have been diagnosed with mental retardation. Consistent with this, the IDEA data do show a corresponding decrease in diagnosis of mental retardation over the same time period as autism diagnoses increase (see Figure 1).

Another point stressed by Grinker is that a diagnosis of autism has consequences for the child’s access to services. A diagnosis may get your child Medicaid waivers so that they can receive a host of interventions at reduced cost. Grinker describes how he lost hundreds of dollars because a speech pathologist who worked with Isabel submitted the bills under the diagnosis of “Mixed Receptive-Expressive Language Disorder”. When the diagnosis was changed to autism, the bills were automatically reimbursed. There is therefore considerable pressure on paediatricians to give a diagnosis of autism rather than some other condition.

The implications of an autism diagnosis for intervention is very different in some other countries. In France, where the legacy of psychoanalysis has been long-lasting, autism is seen as a psychodynamic disorder and there is little educational provision for affected children. In India, the diagnosis is seldom made, even if doctors recognise autism, because there seems no point: there are no facilities for affected children. Grinker has particular interest in South Korea, his wife’s birthplace, where the stigma attached to disability is so great that children with autism will be hidden away, because otherwise their siblings’ marriage prospects will be blighted. Education is seen as the route to success in Korea, and it is normal for children to spend hours after school being coached; a child who struggles at school or who does not conform to expected standards of behaviour brings shame to the family.

Grinker’s book shows that the impact of labels is immense, even if they are worryingly arbitrary. It seems crazy, for instance, that a child with educational difficulties will get insurance cover for interventions, or special help at school on the basis of a diagnosis of autism, whereas a child with equally serious needs who is diagnosed with mental retardation or receptive language disorder gets nothing. But on the positive side, he notes how the growing awareness and acceptance of autism has brought huge benefits to children and their families. Just as in Korea, it used to be common for people in the US to be baffled or even frightened by autism, and for schools to shun children with any kind of disability. The landscape has changed massively over the past twenty years. Grinker notes how Isabel’s schoolmates look out for her, and how her presence in a mainstream classroom has beneficial effects on all pupils. Cultural stereotypes are being challenged, so much so that it is no longer regarded as amazing if a university student tells you they have autism.

This brings Grinker to a final point: which kinds of cultural setting are most supportive for people with autism? He presents some fascinating data showing that rural communities are usually far more positive than urban ones for people with all kinds of disability. In a small community, everyone will know the person with a disability, and see them as an individual. In the context of neurodevelopmental disorders in general, I’ve long been advocating that we need an educational system that doesn’t just try to ‘fix’ children, but rather identifies activities they enjoy and can do well, as a basis for finding them a niche in society. This view is cogently expressed by Grincher “… in order to help people with autism we don’t always need to fully mainstream them, or pretend that they are not different, and we don’t need to simply reduce stigma. Rather, we need to provide roles in our communities for pepole with autism, some of which they may, in fact, be able to perform better than anyone else…” (p. 342).

Bishop, D., Whitehouse, A., Watt, H., & Line, E. (2008). Autism and diagnostic substitution: evidence from a study of adults with a history of developmental language disorder Developmental Medicine & Child Neurology, 50 (5), 341-345 DOI: 10.1111/j.1469-8749.2008.02057.x

The X and Y of sex differences

2011-05-11T07:37:00.002+01:00

Why and how are men and women different? My interest in this topic is fuelled by my research on neurodevelopmental disorders of language and literacy that typically are much more common in males than females. In this post, I am ranging far from my comfort zone in psychology to discuss what we know from a genetic perspective. My inspiration was a review in Trends in Genetics by Wijchers and Festenstein called “Epigenetic regulation of autosomal gene expression by sex chromosomes”. Despite the authors' sterling efforts to explain the topic clearly, I suspect their paper will be incomprehensible to those without a background in genetics, so I'll summarise the main points - with apologies to the authors if I over-simplify or mislead.

So, to start with, some basic facts about chromosomes in humans:
•    We have 23 pairs of chromosomes, one member of each pair inherited from the father, and the other from the mother.
•    For chromosome pairs 1-22, the autosomes, there is no difference between males and females.
•    Chromosome pair 23 is radically different for males and females: females have two X chromosomes, whereas males have an X chromosome paired with a much smaller Y chromosome
•    The Y chromosome carries a male-determining gene, SRY, which causes testes to develop. The testes produce male hormones which influence body development to produce a male.
•    The X chromosome contains over 1000 genes, compared to 78 genes on the Y chromosome.
•    In females, only one X chromosome is active. The other is inactivated early in development by a process called methylation. This leads to the DNA being formed into a tight package (heterochromatin), so genes from this chromosome do not get expressed. X-inactivation randomly affects one member of the X-chromosome pair early in embryonic development, and all cells formed by division of an original cell will have the same activation status. The patches of orange and black fur on a calico cat arise when a female has different versions of a gene for coat colour on the two X chromosomes, so patches of orange and black fur occur at random.
•    In both X and Y chromosomes, there is a region at the tip of the chromosome called the pseudoautosomal region, which behaves like an autosome, i.e., it contains homologous genes on X and Y chromosomes, which are not inactivated, and which recombine during the formation of sperm and eggs.
•    In addition, a proportion of genes on the X chromosome (estimated around 20%) escape X-inactivation, despite being outside the pseudoautosomal region.

These basic facts are summarised in Figure 1. Genes are symbolised by red dots, grey shading denotes an inactivated region, and yellow is the pseudoautosomal region.


Figure 1

Note that because (a) the male Y chromosome has few genes on it and (b) one X chromosome is largely inactivated in females, normal males (XY) and females (XX) are quite similar in terms of sex chromosome function: i.e., most of the genes that are expressed will come from a single active X chromosome.
Studies of mice and other species have, however, demonstrated differences in gene expression between males and females, and these affect tissues other than the sex organs, including the brain. Most of these sex differences are small, and it is usually assumed that they are the result of hormonal influences. Thus the causal chain would be that SRY causes the testes to develop, the testes generate male hormones, and those hormones affect how genes are expressed throughout the body.

You can do all kinds of things to mice that you wouldn't want to do to humans. For a start you can castrate them. You can then dissociate the effect of the XY genotype from the effect of circulating hormones. When this is done, many of the sex differences in gene expression disappear, confirming the importance of hormones.
There's some evidence, though, that this isn't the whole story. For a start, it is possible to find genes that are differently expressed in males and females very early in development, before the sex organs are formed. These differences can't be due to circulating hormones. You can go further and create genetically modified mice in which chromosome status and biological sex are dissociated. For instance, if Sry (the mouse version of SRY) is deleted from the Y chromosome, you end up with a biologically female mouse with XY chromosome constitution. Or an autosomal Sry transgene can be added to a female to give a male mouse with XX constitution. A recent study using this approach showed that there are hundreds of mouse genes that are differently expressed in normal XX females vs. XY females, or in normal XY males vs. XX males. For these genes, there seems to be a direct effect of the X or Y chromosome on gene expression, which isn't due to hormonal differences in males and females.

Wijchers and Festenstein consider four possible mechanisms for such effects.
1. SRY has long been known to be important for development of testes, but that does not rule out a direct role of this gene in influencing development of other organs. An in mice there is indeed some evidence for a direct effect of Sry on neuronal development.
2. Imprinting of genes on the X chromosome. This is where it starts to get really complicated. We have already noted how genes on the X chromosome can be inactivated. I've told you that X-inactivation occurs at random, as illustrated by the calico cat. However, there's a mechanism known as imprinting whereby expression of a gene depends on whether the gene is inherited from the father or the mother. Imprinting was originally described for genes on the autosomes, but there's considerable interest in the idea of imprinting affecting genes on the X chromosome, as this could lead to sex differences. The easiest way to explain this is by a mouse experiment. It's possible to make a genetically modified mouse with a single X chromosome. The interest is then in whether the single X chromosome comes from the mother or the father. And indeed, there's growing evidence for differences in brain development and cognitive function between genetically modified mice with a single maternal or paternal X chromosome: i.e., evidence of imprinting. Now this has implications for sex differences in normal, unmodified mice. XY male mice have just one X chromosome, which always comes from the mother, and will always be expressed. But XX female mice have a mixture of active maternal and paternal X-chromosomes. Any effect that is specific to a paternally-derived X-chromosome will therefore only be seen in females.
What about humans? Here we can study girls with Turner syndrome, a condition in which there is one rather than two X chromosomes. Skuse and colleagues found differences in cognition, especially social functioning between girls with a single maternal X vs. those with a single paternal X. There are few studies of this kind, because it is difficult to recruit large enough samples, so the results need replicating. But potentially this finding has tremendous implications, not just for finding out about Turner syndrome itself, but for understanding sex differences in development and disorders of social cognition.
3. Although most X-chromosome genes are expressed from only one X-chromosome, as noted above, some genes escape inactivation, and for these genes females have two active copies. In the main, these are genes with a homologue on the Y-chromosome, but there are exceptions, and in such cases females have twice the dosage of gene product compared to males (see Figure 1). And even where there is a homologue gene on the Y chromosome, this may have different effects from the active X-chromosome gene.
4. The Y chromosome contains a lot of inactive DNA with no genes. Recent studies on fruit flies has found that this inactive DNA can affect expression of genes on the autosomes, by affecting availability in the cell nucleus of factors that are important for gene expression or repression. It's not clear if this applies to humans.

My interest in this topic has led me to study children who do not inherit the normal complement of sex chromosomes. These include girls with a single X chromosome (XO, Turner syndrome), girls with three X chromosomes (triple X or XXX syndrome), (see figure 2) and boys with an extra X (XXY or Klinefelter’s syndrome), and boys with an extra Y (XYY syndrome).(see Figure 3).

Figure 2

Figure 3

Affected children typically don’t have intellectual disability and attend regular mainstream school. As illustrated in Figures 2 and 3 , this makes sense because the genetic differences between those with missing or extra sex chromosomes and those with the normal XX or XY complement are not great. In Turner syndrome there is only one X chromosome, whereas children with XXX or XXY will have all but one X chromosome inactivated. The extra Y in boys with XYY contains only a few genes.

Nevertheless, although children with atypical sex chromosomes are not severely handicapped, distinctive neuropsychological profiles have been described. Girls with Turner syndrome often have poor visuospatial function and arithmetical ability, whereas language skills are typically impaired in children with an extra sex chromosome. To explain these effects, researchers have proposed a role for genes that normally escape inactivation, which will be underexpressed in Turner syndrome, or over-expressed in children with three sex chromosomes (sex chromosome trisomy) - see point 3 above.

Wijchers and Festenstein note the importance of individuals with sex chromosome anomalies for informing our understanding of sex chromosome effects on development, but their account is not very satisfactory, as they state that “females with triple X syndrome (47,XXX) seem normal in most cases.” Although it is the case that many girls with XXX go undetected, surveys of prenatally or neonatally identified cases indicate that they have cognitive problems. Language deficits are found at high levels in all three cases of trisomy, XXX, XXY and XYY, with a trend for lower overall IQ in girls with XXX than boys with XXY or XYY. We did a study based on parental report, and found a diagnosis of autism spectrum disorder was more common in boys with XXY and XYY than in boys with normal XY chromosome status. But there was considerable variability from child to child, with some having no evidence of any educational or social difficulties, and others with more serious learning difficulties or autism. We currently lack data that would allow us to relate the cognitive profile in such children to their detailed genetic makeup, but this in an area researchers are starting to explore. We are optimistic that such research will not only be helpful in predicting which children are likely to need additional help, but also may throw light on more global questions about the genetic basis of sex differences in cognitive abilities and disabilities.

What are the implications of this research for the debate about sex differences in everyday human behaviour? This was very much in the news in 2010 with the publication of Cordelia Fine’s book Delusions of Gender, which was reviewed in The Psychologist, with a reply by Simon Baron-Cohen. Fine focused on two key issues: first, she questioned the standards of evidence used by those claiming biologically-based sex differences in behaviour, and second she noted how there were powerful cultural factors that affected gender-specific behaviour and that were all too often disregarded by those promoting what she termed ‘neurosexism’. I don’t know the literature well enough to evaluate the first point, but on the second, I would agree with Fine that biological factors do not occur in a vacuum. The evidence I’ve reviewed on genes shows unequivocally that there are sex differences in gene expression, but it does not exclude a role for experience and culture. This is nicely illustrated by the research of Michael Meaney and his colleagues demonstrating that gene expression in rats and mice can be influenced by maternal licking of their offspring, and that this in turn may differ for male and female pups! Genes are complex and fascinating in their effects, but they are not destiny.

Further reading
Davies, W., & Wilkinson, L. S. (2006). It is not all hormones: Alternative explanations for sexual differentiation of the brain. Brain Research, 1126, 36-45. doi: 10.1016/j.brainres.2006.09.105.
Gould, L. (1996). Cats are not peas: a calico history of genetics: Copernicus.
Lemos, B., Branco, A. T., & Hartl, D. L. (2010). Epigenetic effects of polymorphic Y chromosomes modulate chromatin components, immune response, and sexual conflict. Proceedings of the National Academy of Sciences, 107(36), 15826-15831.doi/10.1073/pnas.1010383107.
Skaletsky, H., Kuroda-Kawaguchi, T., Minx, P. J., Cordum, H. S., Hillier, L., Brown, L. G., et al. (2003). The male-specific region of the human Y chromosome is a mosaic of discrete sequence classes. Nature, 423(6942), 825-837.doi: 10.1038/nature01722

Wijchers PJ, & Festenstein RJ (2011). Epigenetic regulation of autosomal gene expression by sex chromosomes. Trends in genetics : TIG, 27 (4), 132-40 PMID: 21334089