The burqa ban: what's a liberal response?

© www.CartoonStock.com

Most liberal-thinking people regard it as a given that we should respect and tolerate the beliefs of others, even if we don’t share them. However, this can land us in difficulties if the people who hold those beliefs don’t reciprocate. This seems to me at the heart of the debate on the burqa ban.

I vividly remember the time when Salman Rushdie was receiving death threats because of the publication of the Satanic Verses. I had fully anticipated that public figures in the UK would rally around him with robust support for freedom of speech. In fact, the support was muted. Although in part this was because of fear - as noted by Christopher Hitchens, - there were others who clearly felt a tension between such support and a need to empathise with the offence caused by the book. Roy Hattersley, for instance, recommended against publication of a paperback version of the book. And the Chief Rabbi wrote to The Times (4 March 1989) that 'the book should not have been published' because of the need to respect and 'generate respect' for other people's religious beliefs. Subsequently, when Rushdie was offered a knighthood, it was that staunch liberal, Shirley Williams, who emphasised the offence to Muslims, leaving it to Christopher Hitchens and the right-wing Boris Johnston to defend freedom of speech. So an asymmetric relationship was validated: I can’t criticise you because it might offend you, but you can not only criticise what I say, but also insist that I don’t say it at all, on pain of death.

Ayaan Hirsi Ali emphasised a similar trend in her memoir Infidel: liberals, she argued, were reluctant to take action against practices such as forced marriages and genital mutilation, because they did not like to be seen to be criticising another culture. Never mind that the culture was inflicting physical and mental damage on its women. This kind of logic was taken to an extreme by Germaine Greer, who argued that attempts to outlaw genital mutilation were an ‘attack on cultural identity’.

My own views on the matter are quite simple. I will tolerate the views of others so long as they tolerate me. I will respect their cultural identity so long as it does not discriminate against others on the basis of sex, ethnicity, or sexual identity. But I expect my cultural identity and beliefs to be correspondingly respected.

So where does that leave the burqa?

Some liberals adopt the easy argument and say that the burqa is a symbol of oppression, and should therefore be banned. There’s no doubt that the burqa has been used to oppress women, most notably by the Taliban. But it is an oversimplification to argue that all women who wear a burqa are oppressed. There are some (including the young Ayaan Ali Hirsi) who choose to wear it. Yes, that choice is bound to be influenced by the attitudes of those around her, but that is equally true of any woman’s choice of attire, whether it be stiletto heels and a mini-skirt or an all-encompassing robe.

So if it comes down to a woman’s right to choose what to wear, what’s the problem? The issue was mocked on Radio 4’s News Quiz last week, as the participants called for bans on other offensive items of clothing, such as socks with sandals or culottes. Andy Hamilton described the French attitude as: “We will force them to be liberated and if they refuse we will put them in prison”.

But the burqa is different from other clothing choices in two important ways. First, it interferes with communication. Liberal-minded people in the UK have no problem with others wearing symbols of their religion such as a turban or headscarf. The real problem is that in face-to-face interactions, wearing a burqa is at best discourteous, and at worst threatening. It creates an unequal relationship, when you can’t even verify the identity of the person you are interacting with, let alone read facial cues. There’s a vast literature in social psychology looking at how nonverbal cues are important in interpersonal communication (e.g. Knapp & Hall, 2009) . We use them to judge another person’s attitude, honesty, boredom and engagement, for a start. If one person has access to these cues and the other does not, that creates an asymmetry in the interaction. Just as British people must learn to respect the culture of others by not wearing skimpy clothing in Arab countries, Islamic women should respect cultural expectations that it is important to see the face of someone we interact with in person.

The second problem with the burqa is the rationale behind its adoption. As many in the Islamic community have emphasised, the burqa is not mandatory attire for a religious woman. However, Islamic women are required to dress modestly, and some interpret this as requiring total cover-up. I suspect that for some women this is an extreme reaction to our highly-sexualised Western society. But what message does the burqa give to men? I think it is offensive insofar as it implies that they are sexual predators whose lust may be inflamed by the sight of a woman’s face. I liked the response from a participant at the UN Human Rights Council, who suggested that rather than expecting women to cover up, men should stay indoors until they learned some self-control.

So if a woman is to wear a burqa, she should be aware of the impact on others: her choice will appear discourteous to many people, especially men. Qanta Ahmed, who describes herself as a moderate Muslim, makes a related point, noting that, far from giving an impression of modesty, part of that impact is to draw attention to oneself, make others feel threatened and increase hostility to Islam.

I don’t agree with Ahmed that the burqa should be banned; this will simply increase intolerance on both sides. I’m pleased that the British government is showing no signs of going down the same route as the French.  Nevertheless, while I don’t think this is an issue that should be dealt with by law, I do think it is reasonable to exert social pressure. A woman has a right to cover up if she wishes, but she should be aware that this is regarded as culturally inappropriate in many situations in Western society. Employers have a right to expect their staff to have a sense of what is appropriate dress for a job; in many situations, a burqa is no more appropriate than hot pants or a crop top.

So in sum, I would defend the right of a woman to wear a burqa if she chooses. But I would also defend the right of someone like MP Philip Hollobone to refuse to meet with a constituent unless she reveal her face, and for an employer to require more appropriate clothing for someone interacting with the public. To do otherwise is to treat the burqa-wearing woman as someone who has rights but no responsibilities.

A short nerdy post about use of percentiles in statistical analyses

Results from psychological tests can be expressed in various ways. Percentiles are a popular format in clinical reports, because they can be explained to non-experts fairly easily, in terms of the percentage of the population that would be expected to get a score of this level or below. So if your score is at the 10th percentile, only 10% of the population would be expected to score this low.

The other format that is commonly used in reporting test scores is the standard score or scaled score. This represents how many standard deviations a score is above or below the population mean. The simplest version is the z-score, obtained by the formula:

  (X-M)/S

where X is the obtained score, M is population mean, and S is the population standard deviation.

In clinical tests, z-scores are often transformed to a different scale, e.g. mean of 100 and SD 15 in the case of most IQ tests. This is done just by multiplying the z-score by the SD and adding the new mean.  So a z-score of -.33 becomes a scaled score of  (-.33 x 15)+100 = 95.

The important point to note is that all of these different methods of reporting scores are just transformations of one another. If you want to turn a z-score into a percentile, you can do so with the Excel function:

100*NORMDIST(A1,0,1,1)

where A1 is the address of the value you want to convert.

The second value in this expression is the mean and the third is the SD, so if you want to convert a scaled score with mean 100 and SD 15 into a percentile, the function is:

100*NORMDIST(A1,100,5,1)

The normsdist function returns a cumulative proportion, so it’s multiplied by 100 to give a percentage.

You can work the other way round using the NORMSINV function, which turns a proportion into a z-score. So if you have a percentile in cell A1, then you get a z-score with:

=NORMSINV(A1/100)

If all this Excel stuff gives you a headache, you can ignore it, so long as you get the message that z-scores, scaled scores and percentiles are all different ways of representing the same information.

They are NOT equivalent, however, in their distributions. Percentiles aren’t suitable as input to statistical procedures that assume normality, such as Anova and t-tests. They should always be converted to z-scores or other scaled scores.

This can be simply illustrated. If you are into Excel, you can generate your own data to make the point - otherwise you can just look at the output from the data I have generated.

Let’s simulate data from two groups, each of 50 participants. Assume the data are reading test scores, and that group 1 has reading difficulties and group 0 hasn’t.  For group 0 I will just generate a random normal distribution of scores with mean 0 and SD 1, by typing this function in each of 50 cells:

=NORMSINV(RAND())

For group 1, I use the same formula, but subtract 0.4 from each score:

=NORMSINV(RAND())-0.4

I pasted my simulated data into SPSS, as it makes it a bit easier to generate relevant statistical output. So for each of 100 simulated subjects, I have a column denoting their group (0 or 1), a column with their z-score, and a column with their percentile score.

Here’s what you get if you do a t-test (you’ll get different values if you generated your own data as the random process is different each time - but it should show the same pattern):

So why, if the numbers are just transforms of each other, are the results different?

The answer lies in the distribution of data. If you take percentiles, you transform a normal distribution into a rectangular one, as can be seen if you plot the histograms.

 

(That hole in the middle of the percentile distribution is just a fluke in the particular dataset I generated). Another way to think about it is to consider the size of difference between two points in the distribution. In terms of z-scores, the difference between the 1st and 10th percentile is 2.32-1.28 = 1.04, and the difference between the 41st and 50th percentile is .23. But on the percentile scale, these differences are treated as equivalent. In effect, the percentile transformation stretches out the points in the middle of the scale and gives them more weight than they should have.

So percentiles are a good way of communicating test scores of individuals, but a bad choice if you are doing statistical analyses of group data.

Special educational needs: will they be met by the new Green Paper proposals?

Many children with disabilities in the UK are not getting the help they need. The campaigning group Whizz-Kidz states: “There are around 70,000 disabled children in the UK who are waiting to get the wheelchair that suits them best. They can wait months, sometimes years”.  And the situation for children with speech, language and communication needs is equally stark. According to the Bercow report, “The current system is characterised by high variability and a lack of equity. (It) is routinely described by families as a 'postcode lottery', particularly in the context of their access to speech and language therapy.”

A recent Green Paper on Special Educational Needs (SEN) released by the Department of Education takes such concerns on board, stating:
“The reforms we set out in this Green Paper aim to provide families with confidence in, and greater control over, the services that they use and receive. For too many parents, their expectations that services will provide comprehensive packages of support that are tailored to the specific needs of their child and their family are not matched by their experiences, just as frontline professionals too often are hampered and frustrated by excessively bureaucratic processes and complex funding systems.and has a wide range of recommendations.” (point 29). This all sounds excellent, so what exactly is proposed?

The two examples given above, of provision of wheelchairs and speech and language therapy are given particular focus. Specifically, the recommendation is for “the option of a personal budget by 2014 for all families with children with a statement of SEN or a new Education, Health and Care Plan” (point 6). There is an emphasis on bringing together the different agencies that are concerned with children who have complex educational needs, so that educational, medical and social agencies work together. So far so good.

Delving deeper into the document, we find the more specific statement:   
“2.41 We have consulted on the introduction of patient choice of any willing provider that meets NHS standards and price for most NHS-funded services by 2013-14. This is likely to apply to many community health services. It will give families choice, where appropriate, from a range of providers who are qualified to provide safe, high quality care and treatment, and select the one that best meets their needs. It will mean that good providers that offer innovative and responsive services are able to grow.”

Note the use of the word “patient” here; although the document talks about “educational, medical and social agencies” working together, the description of the personal budget appears to relate just to health needs.

Not surprisingly, then, the proposed solution has strong parallels with current policies on provision of healthcare, with a focus on outsourcing to private providers. The personal budget is specifically mentioned in relation to provision of such facilities as wheelchairs or speech and language therapy services, which are currently provided via the National Health Service. One can see that any policy that ensures that children get what they need in a timely fashion is to be welcomed, and there is ample evidence that the current system has not always provided this.

The key question, of course, is whether the personal budget will be adequate to give children what they need. All too often, governments have dressed up healthcare policies as providing more “choice”, when in fact they are designed to save money. It is inconceivable in the current economic climate that any extra money will be available for disability and SEN. So everything hinges on whether the personal budget will be sufficient to cover a child’s needs. No doubt the expectation is that competition between private providers will drive down costs so it will be possible to “do more with less”. I hope this works, but I'm not optimistic.

But what about the educational and social aspects of provision? I’m particularly interested in whether the “patient choice” model will be extended from the medical to the educational sphere. Here there is a potential problem. For medical provision, it is argued that NHS standards must be met. One would hope, then, that there will be vetos on spending the personal budget on such interventions as “acupuncture miracle cure” or stem cell treatment for cerebral palsy.  But in the field of special education it’s not clear what standards would need to be met.

Evidence-based education is still in its infancy, and in mainstream education there are plenty of instances where government funds have been spent on educational programmes of dubious or unproven effectiveness. Ben Goldacre, in his book Bad Science, documented the way in which Brain Gym programmes were introduced in UK schools, despite being full of ludicrous pseudoscience. Charlie Brooker’s account of this is also worth a read. Though we may laugh at these educational initiatives, they provide a nice income stream for those who are marketing them. (See also, http://wordpress.mrreid.org/2009/06/01/learning-styles-are-nonsense/)

If the plan is to give families a personal budget to spend on special education, there will be plenty of companies who will see this as a fantastic business opportunity. Some may be providing beneficial services, but there is a real risk that commercial companies will be rewarded with government funds for interventions of dubious or unknown value. Suppose your child has severe reading difficulties, language comprehension problems, or autistic features, and the classroom teacher seems at a loss to know how to help. It is easy to envisage a situation where private companies could offer attractive-sounding interventions, in anticipation that the “personal budget” could be used to support these. There are already numerous cases of such fringe interventions, but currently any parent who wants to try them has to find their own funding. In most cases, the only evidence for efficacy is anecdotal. In a few, claims of scientific support are made, but usually when investigated, the evidence proves to be weak. Randomized controlled trials are very rare in the field of education, and where these have been applied to interventions for children’s learning and educational difficulties, results have typically been much less impressive than when uncontrolled studies are done.

Clearly, if we demanded that any educational approach used in schools had to be demonstrated to be effective to a high standard of evidence, the school system would grind to a halt. Education has never been required to meet the standards of evidence seen in medicine. Furthermore, no innovations would ever occur. I’m a great fan of changing this system to one of “evidence-based education”, but I am realistic enough to realise it is not going to happen overnight. And if we do take an evidence-based approach, we need to consider carefully the way in which we measure children’s outcomes: it could be a mistake to have a narrow focus on educational attainment that does not consider other aspects of well-being. But we need to address these issues urgently if we are going to give “any willing provider” the opportunity to sell services for children with special needs.

As an example of the potential pitfalls, it is worth reading a 2008 report of the Enterprise and Learning Committee of the Welsh Assembly   It is noteworthy that some of those advising the committee had vested interests in the programmes under discussion: Prof David Reynolds had financial interests in the Dore programme, and evidence for efficacy of FastForword was provided by those who sold the programme, and a professor whose institution had made $5.5 million from royalties.  The committee were apparently unaware of independent evaluations of these programmes that gave a much less positive picture, see: http://tinyurl.com/3q4jen9  and a recent meta-analysis of FastForword, which includes studies published prior to 2008 (not to be confused with the Fast Forward wheelchair campaign).

I am not saying that private companies should be excluded from providing special education. Potentially, they have much to offer: being freed from bureaucracy of state-based organisations they have potential to develop new approaches, or to deliver traditional services in an exemplary fashion. But we need to have stringent standards in place when evaluating “willing providers” and ensure there is no conflict of interest in those advising on appropriate interventions. Some providers will see this population as a wonderful commercial opportunity. We need to ensure that limited funds are spent wisely and well.

Of course, this is only germane if a personal budget will be available for educational as well as medical interventions. I wonder whether it will be. The Green Paper is not clear on this, noting merely that “Subject to piloting, this would include funding for education and health support as well as social care”. But even if the piloting supports educational uses for a personal budget, it is not clear which children would have access to this. On the one hand, the Green Paper emphasises the numerous ways in which children currently identified with SEN have poor outcomes. Yet on the other hand it seems to imply that many of those with labels of SEN don’t have genuine problems: “Previous measures of school performance created perverse incentives to over-identify children as having SEN. There is compelling evidence that these labels of SEN have perpetuated a culture of low expectations and have not led to the right support being put in place.” (point 22). And “we intend to tackle the practice of over-identification by replacing the current SEN identification levels of School Action and School Action Plus with a new single school-based SEN category for children whose needs exceed what is normally available in schools; revising statutory guidance on SEN identification to make it clearer for professionals; and supporting the best schools to share their practices." (point 24, my emphasis). Finally, “A new single early years setting- and school-based category of SEN will build on our fundamental reforms to education which place sharper accountability on schools to make sure that every child fulfils his or her potential.” (point 5, my emphasis). 

The Green Paper sounds full of good intentions, but I’m cynical. Cutting through the fine language I see a pincer movement to cut costs of children with disabilities and SEN: first, by radically reducing the number of children who will be deemed to need special provisions, and second, by passing responsibility for the remainder over to the marketplace. I fervently hope the recommendations will do good in overcoming the obstacles currently faced by families in obtaining necessary equipment and services, but I have two worries. First, that the profit motive of those in the marketplace might conflict with the child’s best interests, and second that the net impact for children with hidden disabilities will be to reduce provision and then blame teachers for children’s educational failure.

The Green Paper is a consultation document; I'd encourage all readers with an interest in this area to read the document and give your own views: you have until 30th June 2011 to respond.

The expansion of research regulators: an evolutionary perspective

Reading about evolution has made me think about why some professions grow and thrive while others die out. I'm intrigued by the expansion in numbers of people regulating the activities of researchers. How have we got to a position where the Academy of Medical Sciences concludes: “A complex and bureaucratic regulatory environment is stifling health research in the UK”?

Consider the situation in the 1970s. If you wanted to do a piece of research you did it, no questions asked. But bad things can happen if you let people do just what they want. There are terrible examples of studies where research participants were infected, hurt or humiliated without realising what was happening or giving their consent. For examples, see Rebecca Skloot’s book, ‘The Immortal Life of Henrietta Lacks’ and Dominic Streatfeild’s ‘Brainwash: the Secret History of Mind Control’. The solution was to create a body of people, the regulators, who would scrutinise research and make sure it was ethical. Despite the regulation, every few years something bad still happened. The regulators responded by increasing their numbers and adding more regulations. In general, I’ve avoided doing studies that require me to go through an NHS ethics committee because the process is so long-winded and bureaucratic that it saps all my enthusiasm and takes up time I’d rather spend doing research. Our University ethics committee can approve studies that don’t involve patients and operates a much less complex system. Recently, though, I badly wanted to do a study involving NHS patients, and decided to grit my teeth and go through the process. It’s taken literally weeks of form-filling, and what amazed me was the sheer number of regulators I dealt with in the course of applying for approval. Then there was one set of people from the research ethics committee (REC), another set from R&D, yet more from the Comprehensive Clinical Research Network - in fact several sets of those depending on whether you were concerned with local or regional matters. There are people whose job it is to book your application in to a REC via a centralised system, and others whose job it is to do the same thing at local level when the first fail to find you a slot. These people were typically very helpful, but that isn’t the point. Why are there so many people whose sole function in life is the ethical scrutiny of researchers? Why are there so many forms to fill in that a recent article raised concerns about the environmental impact of paper use by RECs.  And how have we got to the situation described by the Academy of Medical Sciences whereby it takes an average of 621 days from receiving funding to recruiting the first patient for a trial of a cancer drug? 

From an evolutionary perspective, a research regulator is a life form with three very interesting characteristics. First, its numbers explode in response to catastrophic events regardless of how rare that event is. Second, it has few natural predators, so its expansion goes unchecked. Third, regulators multiply like bacteria: they spawn more regulations which require more regulators, so there is a rapid increase in population over time. And these three characteristics derive, I submit, from a basic human tendency to focus on emotionally-engaging events while ignoring their probability.

Catastrophe as a driving force in increasing the number of regulators
When something really terrible happens - someone is badly hurt or upset, or even worse, killed - we empathise with the victim and want to do something to prevent it happening again. All of our attention is taken up by the awfulness of the event, and we ignore the costs inherent in a solution. This kind of thinking is described by Dan Gardner in his book ‘Risk’ as due to System 1, or Gut, as opposed to the more rational System 2, or Head. Gut’s supremacy is such that if someone were to draw attention to rarity of the catastrophe or the costs of the proposed solution, they would be criticised for being heartless. It is this way of reasoning that fosters the dramatic rise in regulators.

Consider the case of Dr Harold Shipman, a general practitioner in Greater Manchester, who in 2000 was found guilty of murdering 15 of his patients. According to Wikipedia, his was one of the most prolific known serial killers in global history with 215 murders being positively ascribed to him, although the real number is likely to be higher than this. He had no obvious motive and did not appear mentally ill to his colleagues or patients. This case led to the Shipman enquiry, led by Dame Janet Smith.  It was discovered that Shipman had been sent a warning letter by the GMC but allowed to return to practice after a conviction for dishonestly obtaining pethidine in 1976.  The enquiry judged, however, that if a harsher punishment had been given, it would not have prevented Shipman from becoming a serial killer. Nevertheless, the committee called for a database to be established containing information about all doctors in the NHS, including disciplinary records, which both patients and NHS bodies could access.  They also supported a system of revalidation, whereby doctors would undergo regular checks of their competency to practise.  There is no indication that anyone ever discussed the probability of another Harold Shipman occurring. I’m sure there are many doctors who are incompetent, have massive personal problems, and there are no doubt a few who feel like murdering their patients from time to time. But I find it hard to believe that we need to set up regulations to scrutinise apparently sane doctors to prevent them from murdering their patients in cold blood.  Nevertheless, in the interests of ‘this must never happen again’ it’s recommended that a whole new posse of regulators be created to check family doctors, whose compliance will no doubt cost time that could be spent with their patients. I suspect one day there will be another doctor who does something really terrible, but I doubt these regulations would prevent this.

A much less dramatic but pertinent example was described on Jenny Rohn’s blog. I recommend you read her account of the regulations produced by her research funder that require that staff in the laboratory wear safety glasses at all times. Jenny, a woman after my own heart, took the trouble to get to the bottom of why this regulation had been introduced, and found there had been a small number of accidents, which could have been prevented if the scientist had taken commonsense precautions and worn safety glasses while performing specific hazardous procedures. A reminder to staff to do this should have been sufficient. Instead, a regulation has been introduced which costs time and money.

Regulators have no natural predators
Once regulation is established, it is remarkably difficult to remove it. This is largely a consequence of the same human tendency as discussed above: the attentional focus on catastrophe. Anyone who argues against regulation will be seen as being so cold-hearted or cavalier as to not care about the catastrophe that led to regulation being set up.

A key point here is that individual regulations often appear trivial - especially when considered in relation to the catastrophes they are designed to avert. Filling in a form, or going to an opticians is tedious, but it seems curmudgeonly to complain if someone’s life or sight can be saved. However, there are expenses in both time and money, and these can become substantial if large numbers of people are required to adhere to regulations and to administer them. We do need to consider carefully whether the measures that are put in place are effective and proportionate.

Consider another example. On 4th August 2002, 10-year-olds Holly Wells and Jessica Chapman were murdered by their school caretaker Ian Huntley in the village of Soham, Cambridgeshire. Huntley had a string of previous allegations about sexual interest in young girls when previously in the North East of England, as well as a burglary charge, but only the burglary charge was placed on the police national computer, and even this was not picked up by the routine checks that the school did, because Huntley had changed his name.  After this case, there has been massive tightening up of police checks for people who work with children. If you plan to work with children or young people, you need a Criminal Records Bureau (CRB) check.  My researcher team works in schools and we all have CRB checks. Recently, though, we’ve found some head teachers will want a new CRB check, just for their school, even if you have recently obtained one. And the regulations have been extended to individuals such as children’s authors who make occasional visits to schools.  Everyone is clearly very nervous about letting unvetted adults come into contact with children. But does it work? On 1st October 2009, Plymouth nursery worker Vanessa George admitted 13 charges of sexual abuse of children and making and distributing indecent images of children. She had completed a qualification in child care and passed a Criminal Records Bureau police check to allow her to work with younger children. 

I am aware that if I query the usefulness of the CRB check procedures it will look as if I am placing my own personal inconvenience above the welfare of vulnerable children. Regulation is tedious, and sometimes costly, but what monster would refuse to fill in a form or pay a few pounds in order to prevent a child being murdered? I can assure readers that I feel every bit as much rage and grief as anyone else every time I see that photo of Holly and Jessica that is so often reproduced in the media. If something can be done to stop children getting murdered or molested, I would be the first to endorse it. I just query whether this massive bureaucratic exercise is a cost-effective solution, as compared, say, with using resources to teach children how to identify and respond to adults who behave inappropriately.

Ultimately, the only thing that could lead to a mass extinction of regulators would be if government were to decide that the regulation was too expensive. However, in general, governments are nervous of deregulation because it will upset people who see regulation as the path to preventing another catastrophe. I disagree. It’s my belief that we can never control life so that there are no catastrophes. So from time to time bad things will happen. Every time they do, more regulators are created, but none are ever removed. Their inexorable rise seems inevitable. But as if this were not enough, there is an additional process at work.

Regulators generate more regulators
In the field of ethical scrutiny of research, a major shake-up was spawned by one rare event, the discovery that a pathologist at Alder Hey Children’s Hospital had stored body organs of deceased children without their parents’ knowledge or consent.  This lead to an explosion of regulation, and a new legal framework for the use of human tissue. But perhaps more surprisingly, it was accompanied by a broadening of the remit of research regulation to apply not just to medical research but to all research involving human participants. I suggest that a driving force here is the regulator mindset. Once you have been set up to prevent catastrophic events, you don’t just focus on the original catastrophe that started the ball rolling, you start trying to anticipate catastrophes, so you can set regulations in place to prevent them. This inevitably generates huge amounts of additional regulation. I thoroughly agree with the idea that it is better to anticipate problems than deal with their consequences, but the difficulty here is that the potential catastrophe absorbs all one’s attention and once again its probability is never considered. 

To take an example, some years ago I was part of a research group who wanted to recruit from a local maternity hospital; we simply wanted to sign up mothers who might potentially be interested in taking part in research when their children was 12 to 36 months of age. At that point, they would be contacted and invited to take part, with no obligation to do so. We did not approach mothers of babies who had any medical problems. One member of the ethics committee was concerned at our procedures. It was suggested that before writing to these parents to invite them to take part in a study, we ought to check with the family doctor whether the child had died. Well, of course, I can imagine it would be awful to receive a letter inviting you to involve your child in a study if the child had died. But what proportion of healthy babies die by 2 or 3 years of age? Is the probability of this happening high enough to justify asking family doctors of some hundred children to check their medical records before we contacted them? According to the Office of National Statistics, the mortality rate for children aged 1 to 14 years was 12 deaths per 100,000 in 2009. Deaths in children under one year of age were more common at 4.5 per 1,000 live births, but most of these were babies who would not have been recruited to our study because they were severely ill in the first week of life, and/or had very low birthweight. I am again uncomfortably aware that I will seem heartless in arguing against adopting a measure to avoid the real but rare possibility of upsetting a bereaved parent. But against this hypothetical risk we need to balance the 100 patients that won’t get seen by their family doctor in the 10 minutes it takes them to locate and check the medical records and reply to the researcher.

In case you imagine such scenarios are unique to the UK, let me give one more example, from the USA. A colleague who does brain-scanning studies of children with developmental disorders tells me she was required to conduct a pregnancy test with any girl aged 9 years or over who wished to participate in the study. This is particularly striking because it is protecting against a conjunction of two very rare possibilities: (a) that a 9-year-old girl who volunteers for a research study might be pregnant and (b) that a brain scan of the mother's head might damage a foetus.

When regulators get together with lawyers, there’s a catalytic reaction, because lawyers are even better than regulators at thinking of things that need regulating. They go beyond defending us from catastrophes and disasters to protecting against things that have the potential to upset a few people. I was interested to read in the report by the Academy of Medical Sciences that the NHS Litigation Authority had never received a claim relating to research, yet the lawyers insist we put paragraphs in our information sheets about risk, indemnity and how to make a complaint. They’ve also had a major success defending people’s rights not to have their medical records scrutinised by anyone outside the clinical care team. The problem is that if you want to do a medical research study you need to identify suitable people to take part, and that means looking at their medical records. If you insist, as current regulation requires, that no-one outside the clinical care team can look at records, you have two stark options. Either the clinical care team have to spend time not caring for patients, but trawling through records, or the research cannot get done. Richard Doll was one of the first to speak out against this kind of regulation, which makes most epidemiological studies impossible to do.  This is one point where the report by the Academy of Medical Sciences has a recommendation to allow bona fide researchers to screen medical records.

Another factor leading to multiplication of regulators is an attitude of trusting no-one. Having required researchers to give a detailed account of what their research involves, down to the last comma in an information sheet, they need squads of highly trained people to scrutinise the forms to identify possible problems. And if this is not enough, they then introduce a further stage of monitoring the research. The implication is that the researchers can’t be trusted. Unless they write regular reports to the regulators on the progress of the research, they are likely to go off the rails. Even this is not enough: the regulators also have power to visit researchers to ensure they are doing what they said they would do. This of course all creates more jobs for the regulators. Nobody ever asks whether the money might be better spent on, for instance, doing research.

How can we retrieve the situation?
Having seen the increasing drive for more and more regulation during my lifetime, I am alarmed at its unstoppable progress. I’ve focused here on those aspects that have impinged on my life as a researcher, but the trend for ever more regulation appears to infest many other areas of life. I have two suggestions for how to improve matters:

a) Before any new regulation is introduced, there should be a cold-blooded cost-benefit analysis that considers (i) the severity of the adverse event that the regulation is designed to avert; (ii) the probability of the adverse event;  (iii) the likely impact of the regulation in reducing that probability; (iv) the cost of the regulation both in terms of the salaries of people who implement it, and the time and other costs to those affected by it. I use the word cold-blooded deliberately: our normal human instincts don’t lead us to weigh up these different factors rationally. Instead, we focus solely on (i).

b) We should be more imaginative about the type of regulation that is used. For instance, research regulators increasingly play a role in training researchers in ethical conduct of research. Currently one is expected to undertake such training in addition to all the form-filling. But why not treat it more like a driving test? Once trained, researchers could be certified as competent and left to get on with it without having to fill in any forms, and without constant scrutiny and monitoring. It would save huge amounts of everyone’s time and money if we could trust people to behave professionally and treat ethical skills more like driving skills. The regulators could then focus on training researchers and offering advice to those who encountered specific ethical issues in their research. Their role would become advisory rather than policing.

I had intended to write a blogpost documenting the many stages I have gone through on the road to seeking ethics approval for my current study, but that procedure, started in December, is continuing, and I cannot tell when it will end.

The one hour lecture: How to captivate your audience in ten easy steps

1. Don’t rehearse

2. Have at least 100 slides

3. Don’t use Powerpoint’s ‘hide’ function: just rapidly flick through the slides that you don’t have time for - this creates a sensation that you could give them far far more exciting stuff if only you had more than an hour

4. Spend the first 30 minutes on your introduction - people are always more interested in introductions than in novel content

5. Even if you’ve been told your audience has little background in the area, there is likely to be one or two renowned experts in the room. Focus on the experts.  Be sure to impress them with your intricate understanding of the minutiae of the field. Don’t bore them by explaining the basics.

6. Be sure to check politely with the chair ‘How much longer do I have?’ as the 60 minute moment passes

7. Explain to the chair that you need, ‘Just five more minutes’ as the 65 minute moment passes. Your audience will be disappointed it’s only five minutes, but will be pleasantly surprised when you take longer.

8. Introduce the final set of killer experiments as the 66 minute moment passes: the audience will be delighted that you’ve saved the best material to the last

9. Have a slide saying Conclusions which isn’t the last slide. It creates exciting tension if they think you’ve finished only to find there is much, much more.

10. Spend at least 5 minutes on the Acknowledgements slide. Your audience is deeply interested in the many people whom your work depends on, and you should give their name, photograph, country of origin, role in the research, together with a quirky story illustrating their personality.

Where commercial and clinical interests collide: the case of auditory processing disorder

I’m currently writing a blogpost for the Wellcome Trust focusing on my research on auditory processing problems in children with language difficulties. While checking out links I realised there was another post I wanted to write on this topic: not on the science, but on the politics.

As I’ll explain more in the Wellcome Trust piece, auditory processing disorder (APD) is a diagnosis that is made when a child obtains a normal audiogram, i.e. demonstrates normal ability to detect sounds, yet appears not to perceive sounds normally. A common complaint is difficulty hearing speech in noise. Various experimental tests of auditory processing may show the child doesn’t appear to discriminate differences between sounds that vary in features such as pitch, duration or modulation (wobble).

APD is unusual in that there are no agreed diagnostic tests. I was pretty certain APD didn’t feature in the diagnostic bible of the American Psychiatric Association, the DSM-IV, and googling around suggests it’s not going to feature in the new DSM-5 either.   I was surprised, though, to find a mention of APD, or something very like it, in the alternative bible, the International Classification of Diseases. My searches turned up the category of “Abnormal auditory perception unspecified”, code 388.40 in ICD-9-CM. An accompanying statement on the website read: "388.40 is a billable ICD-9-CM medical code that can be used to specify a diagnosis on a reimbursement claim”. 

Given the lack of agreement on diagnostic criteria and lack of recognition in formal guidelines, it’s impossible to find sensible epidemiological data on APD. My impression, though, is that it’s a diagnosis that is quite commonly made in the USA and Australia but is much less so in the UK. A few years ago, I attended a small UK conference organised by the British Society of Audiology on APD. Many of those attending were audiologists working in the National Health Service (NHS). They wanted to update their knowledge and skills, but were apprehensive of this category, which for many of them was a new one. They were particularly concerned that scarce NHS resources might be diverted to diagnosing a condition of uncertain validity, and even more concerned at the lack of any agreed methods for treating it. The conference organisers had done their best to include a session on intervention, and had written to various American experts who were known to have developed specific approaches to APD. They did not have much joy, however. One expert explained that she didn’t give talks about her intervention, but if the organisers liked, she could run a course on it. I’d never come across this kind of thing before: for the other neurodevelopmental disorders I work on, people who have expertise in intervention will talk to other professionals about what they are doing, and be willing to present information on its rationale, methods and efficacy. Not the case here. This was closed information for which one paid money. And since there was also no published information on rationale, methods and efficacy, it was very much a case of taking it on trust. No thanks, said the organisers.

Are people in North America just less sceptical than those in the UK? The answer is no. While hunting for a mention of APD in DSM, I found a clinical policy bulletin by Aetna.They wrote a critical account of APD and its treatment, and I was pleased to see they cited a recent review by Dawes and Bishop (2009). Their overview stated: “Aetna considers any diagnostic tests or treatments for the management of auditory processing disorder (APD) (previously known as central auditory processing disorder (CAPD)) experimental and investigational because there is insufficient scientific evidence to support the validity of any diagnostic tests and the effectiveness of any treatment for APD.” Further googling revealed that Aetna is a US medical insurance company.

Putting it all together, one can’t avoid the conclusion that APD is Big Business. Not in the UK, where most of our audiologists are working for the hard-pressed National Health Service, and have no motivation to diagnose this condition. In the USA and Australia, however, audiologists in private practice have considerable incentive to diagnose APD, as they can then offer expensive treatments for it. The ICD-9-CM code opens the door to allow people to claim these expenses on medical insurance. I initially found it strange that by far the most objective and thorough analysis of APD I could find was found on the website of an insurance company, but then realised they are the ones who have an interest in being sceptical about this diagnostic category.

The sad thing about all this is that caught in the cross-fire are children whose specific difficulties may have an auditory basis. Yet none of the clinicians seems motivated to develop robust diagnostic tools, and interventions are dreamt up without adequate scientific basis or evaluation. This is a downside of a privatised healthcare system: practitioners benefit from making diagnoses but not from testing their validity.

Academic publishing: why isn't psychology like physics?

Copyright: www.CartoonStock.com

My job is to conduct research and publish the results, so I’ve become interested in discussions on the internet about models of publishing. There are three tensions in the field: (1) who pays to publish research? (2) who decides what gets published? and (3) who takes any profits?

In the traditional model, based on publishing on paper, the answers have been (1) readers pay, via journal subscriptions, (2) editors decide, advised by reviewers, (3) publishers take the profits.

But publishing on paper is in decline. Most of us read papers on the internet taken from electronic versions of journals; either directly on computer, or printed from a pdf. This has led to some re-evaluation of how publishing is done. Most publishers, with an eye to their profits, have attempted to just continue with the old system, charging subscriptions for electronic journals much as they have done for paper journals. These are largely paid for by institutional libraries rather than individuals. If your institution doesn’t subscribe, or you don’t belong to an institution, there is an option to pay for individual articles, but the charges are typically prohibitively high.

Psychologist Stevan Harnad was among the first to point out that the internet changes the publishing game completely. Way back in 1994, he suggested that academics could leave publishers out of the research communication cycle altogether. For instance, when I write an article, I could post it on my website, and anyone who wanted to read it could download it. Instead of institutions paying thousands of pounds in journal subscriptions, and individual readers being clobbered for $30 for a five-page article, academic exchange would be virtually free. What would be wrong with that? This appealing idea has over the past sixteen years fostered a huge amount of debate. What is surprising to me, though, is that there appears to be a massive discrepancy between disciplines in how far Harnad's ideas have been taken up. And Harnad's own discipline, psychology, is one that has been very slow to respond, compared, say, to physics - see http://arxiv.org/. Because many of my readers are psychologists, and many psychologists are wedded to the traditional printed journal, I'll briefly rehearse some of the arguments that have been made, before commenting on the possibilities opened up by e-publishing.

There are two obvious snags to bypassing journals. The first just concerns presentation. Traditionally, publishers have ensured that journal articles look nice: properly formatted, intelligible figures, grammatical writing, and so on. Their role in this aspect of publishing has, however, dwindled over the years. This does not mean that any academic can produce a nicely-formatted article: some skill is required. And many academics don’t write very well and benefit from the services of professional editors. But increasingly, publishers are cutting down on professional editors and graphic designers - indeed, increasingly they hold authors responsible for getting their figures formatted for direct web publication (see my post).

The second point has to do with quality control. Traditionally, journals have acted as gatekeepers. If an article appears in a peer-reviewed journal, it can usually be assumed that it doesn’t have serious flaws: the higher the quality of the journal, the safer this assumption should be. And for the top-notch journals with the highest impacts, publication of paper generally gives some guarantee that it is not only methodologically sound but also particularly important in terms of advancing the field.

Nevertheless, some have queried whether we need this traditional function of publishers, pointing out flaws in the peer review system and idiosyncratic behaviour by editors. Indeed, gatekeeping may not just be imperfect: In some fields it may damage science, by inducing a publication bias in favour of ‘significant’ (both statistically and theoretically) findings, and biasing against publication of replications (or non-replications).It's even been argued that publishing in high-impact journals may be unethical, because such journals are particularly likely to publish papers that over-state their conclusions. And unless there is a massive media stink, they resist accepting papers that fail to replicate exciting findings that they've previously published.

Such considerations have led some to argue that we should ditch not just publishers, but the whole editorial and reviewing process as well. In effect, everyone would just publish their own stuff, and it would be judged by the academic community. Although this solution appeals to the more anarchic side of my nature, I can see objections to it. The main is that we’d all be swamped by a tidal wave of information. The good stuff would be buried within a heap of rubbish. And because methodological flaws are often hard to detect unless you have done research in the field yourself, problematic but apparently important studies might have a large impact.

As Harnad noted, however, taking publishers out of the loop need not entail abandoning peer review. With the exception of a few top-notch journals, most academic publishers don’t employ editors. Instead, editors do the work for free, or for a small honorarium. It is very unusual for reviewers to be paid anything. So this gate-keeping role of publishers doesn’t need to reside with publishers. Any group of academics could decide to set up a journal, appoint editors and operate a reviewing process. Provided the journal was electronic, with no paper copies, this should be achievable at relatively modest cost.

But has it worked? Well, sadly, not for psychology. I've brooded about this issue for some years, but was stimulated to revisit it by my recent discovery of Kindle direct publishing. It’s different from an Open Access model, because the reader pays for content, but the sums involved are trivially small compared to current journal charges. The Kindle operation is designed for publishing books. Harnad has always distinguished commercial book publishing from what he terms ‘esoteric’ publishing, i.e. publication of academic articles of interest only to a small group of experts. But my Kindle experience made me wonder whether the distinction between the two types of publishing need be as sharp.

I came across Kindle e-publishing when I’d had my first novel turned down by two agents, both of whom spoke of a ‘saturated’ market for crime novels, the genre I’d written in. I’m not a typical author: though I’d be very pleased if people enjoyed reading my book as much as I enjoyed writing it, I don’t have any high expectations of a successful second career as a novelist. I write just for fun, and, being an academic, I’m used to not getting paid for what I write. And being an academic also makes me used to rejection, while at the same time recognising that I shouldn’t let it discourage me if I have a belief my work is good.

After my experience with open access journals, I’d expected that any business that offered a self-publishing option would charge authors, especially for ‘esoteric’ material. As a first novel by an unknown author and with neither agent nor publisher, my book was definitely in that category. So I was pleasantly surprised to find that not only did I not have to pay anything, but I could set my own price and get a 70 per cent royalty. For a book with low-volume sales, this is going to be negligible, especially as a great chunk of tax is taken off any US sales. (This can be avoided if you fill in a lot of forms, send away your passport to the US, and so on, but since I was not expecting more than about two US sales, this clearly was not a sensible option). Anyhow, you are walked through the process with easy instructions, which include downloading a couple of bits of software to convert your manuscript into the right format, and then you press a button and upload your book. The whole process took about the same amount of time than it typically takes to submit a journal article through an electronic portal. And  you don't need a Kindle, though the reading experience is better than on a PC or iPhone. I keep wondering if there’s a catch; no doubt someone will tell me if there is. In common with some others, I am a bit dubious about accuracy of the sales figures provided (there's a rather surreal exchange about this on the Kindle forum). But when I go to Kindle store, there it is: the Case of the Fremantle Fingers by Deevy Bishop at $2.99. This gives me a small glow of satisfaction.

So I started to think, couldn’t research papers adopt this model, but with even lower charges? I’m not saying the articles need be in Kindle format: I don’t think it’s ideal for scientific papers. But rather that the method of author download to a central site, from which others download at a minimal cost, would be an interesting one for psychology (and other discipline) journals to adopt. The difference from current practice is that the material would be affordable; instead of costing 20 to 30 dollars, individual articles would be priced at 20 to 30 cents. The Kindle store does sell some books, often out-of-print classics, for ridiculously low prices, (or even free), and the charging mechanism doesn’t seem to be a problem. Authors of academic articles don’t expect royalties, so these could be waived.

But what about the dangers of the free-for-all and lack of peer review to identify quality? I suspect that even a notional charge per download might be a help here, by identifying material that people would be willing to pay for. And the Amazon system also suggests a more explicit way in which refereeing be adapted to such a system, by using a star rating with optional comment. This is especially useful in the academic context if the reviewers themselves also have ratings. For instance, a 5 star reviewer would be one who’d published significant papers in the same area, a 3-star reviewer would be someone familiar with this or a related area but without significant publications in that area, and those with no expertise would be 1-star reviewers. It shouldn’t then be difficult for readers to see which papers had high ratings from knowledgeable people.

But would people make ratings? There has been disappointing take-up of Comments options for journals that offer this, such as the PLOS series. I don’t like anonymity, but  to encourage frank comments on papers,  there would have to be some way of registering reviewers that made it possible to check their credentials and assign them an anonymous ID. Otherwise, junior people might be scared to make adverse comments on senior figures.

The downside? For me, omitting the usual pre-publication reviewing stage would have disadvantages. As I’ve noted before, I find peer review painful but often helpful in improving papers. Sometimes it has saved me from revealing an embarrassing amount of ignorance. I’d be happier with a system that allowed an option for pre-publication review, and it would be interesting then to see whether papers that were identified as having undergone such review would attract better post-publication ratings than other papers. Learned societies, for instance, which currently produce paper journals, might instead use their existing reviewing systems for papers which are then deposited on the electronic store, with a quality kitemark.

Another drawback is that for some people, particularly those in low-income countries, even a nominal charge per download would be a deterrent. But it should be feasible to devise a system that subsidised those from resource-poor countries. Or  the charge could be waived once a certain number of downloads had been achieved, so that the most popular material becomes automatically freely available.

Since Harnad’s ground-breaking formulation of the Subversive Proposal much has been written about models of science publishing. Many changes have occurred, with increasing pressure for open access, and growing criticism of the peer review system. But there remains a massive difference between disciplines: physics embraced self-archiving rapidly, whereas scientists in other disciplines often don't even realise it's an option. Maybe this has to do with the pace of change and degree of competition in a field: physicists don't want to wait months before their work is published because it could be scooped. Psychology experiments are seldom so time-sensitive. Nevertheless, I think the pace of research in our field could be improved enormously if we broke free from the stranglehold of the traditional commercial publishers. 

btw, Tom Webb has drawn attention to a Commons Select Committee that is seeking evidence on the peer review process. This strikes many people, including me, as a bit strange, in that it would seem to fall outside the remit of government. If you want to comment, you need to do so by Thurs 10th March, 2011.

Orwellian prize for journalistic misrepresentation: an update

I've been re-reading George Orwell's Nineteen Eighty-Four to get me in the mood for looking at nominations for the Orwellian prize of journalistic misrepresentation. Bizarrely, it cheered me up. I last read it in 1968, and it made me nervous. But here we are in Airstrip One in 2010: we may be heading for mass unemployment, dismantlement of the NHS, the BBC and the Universities, we may be getting increasingly uncomfortable with the state's attitude to controlling dissent, but compared to what Winston Smith went through, this is paradise. New technologies are being used to oppress people the world over, but the internet has also emerged as a tool for fighting oppression. I can expostulate on my blog about things that concern me without the thought police carrying me off.

But, back to the prize. Why, I've been asked, did I choose that name? In Orwell's dystopian world, the press is used to achieve control, such that "nominally free news media are required to present 'balanced' coverage, in which every 'truth' is immediately neutered by an equal and opposite one. Every day public opinion is the target of rewritten history, official amnesia and outright lying, all of which is benevolently termed 'spin', as if it were no more harmful than a ride on a merry-go-round." This captured what I felt when I read the coverage of certain scientific discoveries in the press. I knew that there was an Orwell Prize to celebrate the best of British political journalism; I saw the Orwellian Prize as the inverse – a way of noting the worst of science journalism.

I wasn't so concerned by the fact that journalists sometimes make mistakes. That's inevitable when writing under time pressure on subjects where one does not have expertise. I accept too that journalists, and their editors, are not required to be neutral, and are entitled to promulgate their opinions. I'm concerned, however, when mistakes have implications for people who are vulnerable, who might be misled into adopting an ineffective treatment, or shunning an effective one, by a prominent newspaper article. Or, in the case of major environmental issues, a piece on climate change or the impact of industrial practices might sway public opinion in a direction opposite to that held by informed scientists, and risk making our planet less habitable for future generations.  The aspects of scientific writing that particularly disturb me are when science is misrepresented, and that misrepresentation is done thoughtlessly or even knowingly: sometimes for political ends, but more often to get a 'good story' that will have an eye-catching headline and so sell more newspapers.  The MMR story is the most vivid illustration I'm aware of. It's bad when hard-pressed readers are induced to spend money on ineffective treatments; it's worse when children risk disease or even death as a consequence of irresponsible journalism.

In a brilliant post last September, Martin Robbins lampooned the formulaic treatment of science stories that is so often seen in the media. His was a generic account, but in specific areas of science, one can provide even more detail. Thus, for every new advance in neuroscience or genetics, there seems to be one of two possible conclusions: either (a) if confirmed, the discovery will make it possible treat dementia/dyslexia/autism/Parkinson's disease/depression in future, or more commonly (b), the discovery will lead to better diagnosis of dementia/dyslexia/autism/Parkinson's disease/depression.  Yet most of the research in this area is a long way from translation into clinical practice. We are still learning to interpret tools such as brain imaging and genetic analysis, and the more we do, the more complex it becomes. Researchers become rightly excited when they find a neural or genetic correlate of a disease, which could help our understanding of underlying causal mechanisms. This is a vital step in the direction toward effective clinical procedures, but it's usually unrealistic to imagine these will be available soon. Yet in their desire to appeal to human interest, journalists will exert pressure to make more of a story than is justified, by focusing more on hypothetical rather than actual findings. The impetus behind the Orwellian prize is described in my earlier post, where I dissected an article describing a positive impact of fish oil on concentration in children with attention deficit hyperactivity disorder (ADHD). The study behind the story did not use fish oil, did not include children with ADHD, and did not find any behavioural benefits in the children who were given fatty acids. This article epitomised everything that made me angry about reporting of science in my area, with errors so numerous it was hard to believe they were accidental. I therefore threw down a challenge to others to see if they could find equally worrying examples. As it happens, that article was subsequently the focus of a scathing attack by Ben Goldacre, and was taken off the Observer's website.

Now, one of the things I love about blogging is its interactive nature. A number of commentators made points that made me think more. None of them were apologists for poor reporting, but they noted that journalists are not always to blame for an overhyped article. For a start, they have no control over headlines, which are  typically written by a sub-editor whose job it is to attract the reader's attention. They are also fed information by institutional press officers who may put a spin on a story in the hope that the media would pick it up. And researchers themselves are not immune from wanting their day of glory and being willing to 'accentuate the positive' rather than adopting the cautious, balanced approach that characterises a good scientist.  I have to say that, although I knew this could happen, until I started looking at candidates for the Orwellian, I had always thought that it was only a handful of maverick scientists with personality disorders who would behave in this way. I'd just assumed that for most scientists their reputation with colleagues would be far more important than a brief burst of media attention. I'd also reckoned that press offices would jealously guard the reputation of their institution for impeccably accurate science. But this proved to be naïve. Fame is seductive, and many will compromise their standards for their spot in the limelight.

Since throwing down my challenge, I've had two nominations for the Orwellian. The first was from Cambridge neuroscientist Jon Simons, who true to the spirit of my original post, scored up his submission according to the system I'd proposed, which involved giving points for each statement that was inaccurate when checked against the original source article. The nominated newspaper report was by a Washington Post staff writer and appeared on 9th September under the headline "Scientists can scan brains for maturity, potentially gauging child development". Simons's computations gave it a total of 16 points, putting it on level pegging with the ADHD article. One needs only to look at Figure 1 from the original article, reproduced below, to see one reason why he was exasperated. The prediction from the "functional connectivity Maturation Index" (fcMI), while statistically significant, is far from precise, because of the variation around the average level for each age. Consequently, there is a fair amount of overlap in the range of scores seen for adults and children. If a normal adult can get a brain maturity index of a 8 year old, and a normal 8-year-old can get a maturity index equivalent to an adult, it is questionable just how useful the index would be at detecting abnormality. Also, the regression equation was computed on the combined child and adult data, and nowhere in the paper are data presented on the accuracy of prediction of chronological age from brain measures just within the group of children. As an aside, our research group has tried similar things with the more low-tech methodology of event-related potentials, and we find it is relatively easy to discriminate child brains from adult brains, but not easy to distinguish between a 6-year-old and a 10-year-old (Bishop et al, 2007).

Figure 1 from Dosenbach et al, 2010

I would not be happy, however, giving the Orwellian award to the brain maturity piece, because the journalist does not seem to be at fault. Much of the newspaper article appears to be based on a press release from Oregon Health and Science University.  And furthermore, the Science article reporting the findings is entitled Prediction of individual brain maturity using fMRI and concludes that the method "could one day provide useful information to aid in the screening, diagnosis, and prognosis of individuals with disordered brain function."  I could not find any source for the claim in the newspaper article that the method might tell us "whether teenageers are grown-up enough to be treated as adults", though I did find a Neurolaw blogpost that could be the source of this idea: it stated "in the courtroom, this work could prove useful in determining whether children or adults should be culpable for their actions based on the maturation of the brains" (!).  Overall, I felt that the majority of mistakes in this piece were either misinterpretations that were encouraged by the journal article or press release, or involved extrapolations beyond the data that originated with the authors. And, unlike some other articles focused on developmental disorders, I did not feel this one had much potential for harm.

Another contender for the prize came from Australia, where conservation ecologist Corey Bradshaw was exercised at the way his work on frogs had been misrepresented. In a paper in Conservation Biology entitled Eating frogs to extinction, he and his colleagues called for certification of the trade in frogs' legs, which has the potential to have significant impact on frog populations. I won't attempt to summarise his arguments, which are eloquently stated in his own blogpost. All science journalists should read this – not just because it is witty, but also because it gives some insight into just how angry scientists get when they try hard to explain the science to a journalist who then gets it wrong . But this article, too, didn't seem to merit the prize. Quite apart from the fact it's really too early for the prize, dating back to 2009, there's also a sense of breaking a butterfly on a wheel. The article in question was not in a major newspaper; it was rather on a blog called SlashFood. Admittedly, it's an award-winning blog, described by Time magazine as "a site for people who are serious about what they put in their bodies". But this was clearly a journalist coming at the frogs' legs story from the culinary and not the scientific angle. And as Bradshaw himself says of the journalist, " Now, in all fairness, I think she was trying to do well…" Also, it's hard to see that the article did much harm, beyond raising the blood pressure of a distinguished conservation ecologist to dangerous levels.  Replete with inaccuracies as it was, the overall impact would have been to make environmentally conscious people think twice about eating frogs' legs. Incidentally, journalists at the Guardian will be pleased to hear that Jon Henley's coverage of the same story got a ringing endorsement from Bradshaw.

These were the only nominations I received for the prize, and so I have to announce that it will not be awarded this year, although the two nominators will receive an alcoholic token of my appreciation for their efforts. Although I could give the award to the original Observer article, this does not seem justified given that the newspaper withdrew the piece when it became aware of the scientific criticism.

I will accept  nominations for 2011, and am interested in receiving them even if they don't meet all criteria: I am fascinated by interactions between the media and scientists, in finding out more about what scientists do to irritate journalists, and vice versa. If we are going to improve science reporting, we need to understand one another. I hope that the lack of a serious contender for 2010 is telling us something about improving standards in science journalism - but maybe readers know better.

What works for women: some useful links

This is a work in progress! Feel free to suggest additions

Resources from Virginia Valian, including details of her book, "Why So Slow", a webcast and other resources

 

Athene Donald’s blog Jenny Rohn's blog Blog on "becoming a domestic and laboratory goddess" Demonstrations of schemata (see Valian for context) https://implicit.harvard.edu   Wellcome Trust report on Women in Science, Englineering and Technology Wellcome Trust summary Who Applies for Research Funding? STRIDE Faculty recruitment workshop readings STRIDE faculty recruitment, other resources

Should we ration research grant applications?

Researchers can never have enough funds.  Talented people with bright ideas frequently fail to get funded, leading to low morale in academia.  In the current financial climate, it is easy to put all the blame for this on government.  But a recent consultation document by the Economic and Social Research Council (ESRC) shows that’s not the full story.  Between 2007 and 2009, the number of responsive mode grant applications rose by 39%.  A similar picture is seen for other UK research councils.  If more people apply for the same pot of money, it is inevitable that a smaller proportion of applications will be funded: In the case of ESRC, success rates have gone from 30% to 16%.  Reductions of research council budgets in real terms means things can only get worse. ESRC reckons that the status quo is not an option: once success rates fall this low, it ceases to be worthwhile for researchers to submit grant proposals. As well as the costs to applicants in time and effort, there are also financial costs to higher-education institutions in administering grant applications, and for ESRC administrative staff.  The peer review process also starts to break down: on the one hand, it becomes difficult to find enough reviewers to handle the mounting tide of proposals, and on the other, reviewers become reluctant to say anything negative at all about a proposal, as only those with a uniform set of glowing reviews stand any chance.

One of the UK Research Councils, the Engineering and Physical Sciences Research Council (EPSRC) has put new measures in place to attempt to reduce the flood of applications. The other research councils are discussing how to do this, and the ESRC is to be congratulated for taking soundings from the academic community about possible ways forward.

Their document, however, creates an odd mental state in academic readers. We’ve all been told for years that getting grant funding is a Good Thing.  Individuals who can bring in the funds will be rewarded with promotion, tenure, and glittering prizes.  Universities pride themselves on their grant income, which is major factor in higher education rankings. Now, however, we are told that applying for grants is a Bad Thing, which the academic community needs to work with ESRC to cut back, with statements such as:

•    (ESRC has) an ambitious target of halving the number of applications submitted through its standard grants scheme by 2014
•    greater self regulation has the potential to significantly reduce the volume of applications submitted by institutions
•    (self-regulation will) probably not go far enough achieve the 50% reduction that the Council is seeking to deliver

The five options outlined by ESRC (copied verbatim below) are:
•    Researcher Sanctions: This involves limiting the number of proposals from individual researchers who consistently fail to submit applications that reach an agreed quality threshold;
•    Institutional Sanctions:  This involves introducing sanctions for HEIs whose applications fail to meet a certain success rate and/or quality threshold;
•    Institutional Quotas for ‘managed mode’ schemes.  This involves the introduction  of institutional quotas for certain schemes (e.g. early career researcher schemes, Large Grants/Centres, Professorial Fellowships).;
•    Institutional quotas for all schemes: This involves responsive as well as managed mode schemes; 
•    Charging for applications. Levying an agreed fee for institutions submitting applications, with the option that this levy is redeemable if the application is successful.

The pros and cons of each of these is discussed, and readers are asked to comment.

My response to ESRC is that they are looking for solutions in the wrong place. To fix the problem, they need to change the basic structure of university funding so that institutions and individuals are no longer assessed on amount of research funding, but rather on an output/input function, i.e. how much bang do you get for your buck.

I have argued in a previous blog that it makes no sense to reward people simply for securing large amounts of funding. Currently, a person who secures a £500K grant which leads to two publications in lower-ranking journals will be given more credit than one who generates five high-ranking publications over the same period with a £50K grant. Clearly, some sorts of research are much more expensive than others; the problem is that the current system discourages people from undertaking inexpensive research.  In my own field of psychology, there are cases of people who have published an impressive body of work based largely on student projects: they do not, however, get much appreciation from their institutions. Meanwhile, it has become almost mandatory for psychology  grant applications to include an expensive brain scanning component, even if this adds nothing to the scientific value of the research.  The introduction of Full Economic Costing (FEC) has added to the problem, by introducing incentives for researchers to add collaborators to their proposals, as this will bump up the cost of the grant.  In short, the combination of the RAE and FEC does the opposite of encouraging cost-effectiveness in grant applications  - it makes people focus solely on cost, the higher the better.

The same incentives have not only encouraged an anti-thrift mentality in higher education institutions, they have also changed expectations about the numbers of grants that academics should hold.  Doing research takes time, and applicants are typically asked to quantify this in grant applications in terms of hours per week spent on the research. As far as I know, nobody ever adds up the estimated hours of research time for a person  holding grants by different bodies. I suspect that there are cases where, if one were to total up estimated time across all a researcher's grants, it would exceed the number of hours in a week. This is particularly true for grant-holders in lectureships, who presumably are expected to spend some time on teaching activities.   The system as it stands will encourage an academic to apply for 5 grants,  spending 1 hr per week on each of them, than to apply for a single grant, on which they propose to spend 5 hr per week. Yet I’d bet that the quality of research would be better in the latter case, because high quality research takes time and thought.  Over-commitment is encouraged by the current system, yet causes stress and waste. Many research-active academics are overwhelmed by backlogs of research data, because they feel compelled to submit more grant applications rather than writing up what they have done.

I have three suggested solutions to the current crisis. The first involves such a radical change to funding structures of Universities that it is unlikely to be implemented. The other two are both feasible:

1.    For Government: Ditch the current methods for allocating funding to Universities so that cost-effectiveness and thrifty use of research funds are rewarded, rather than punished.
2.    For research councils: When evaluating research quality, be more focused on track record of outputs relative to income, so that funding is steered toward those who have demonstrated good cost-effectiveness. Given differences in costs and time-scale of outputs across disciplines, a generic metric would be unworkable, but just by asking grants panels to look at this would be a step forward. Obviously this would not apply to new investigators.
3.    For Universities:  Scrutinise all the research grants held by individuals to ensure that amount of time specified for research activities is realistic, taking into account other job demands.  A person would be debarred from putting in a grant proposal if they were over a limit in terms of hr per week already allocated to research.

If all three could be implemented, we could achieve a situation where the pressure on research councils is relieved, academics would be able to do good research with less stress on continually applying for funds, and research quality would be enhanced.

P.S. 11th January 2011
My congratulating ESRC for undertaking consultation with academics may have been premature. It seems very few academics knew about this, at least in my discipline of psychology. One who made enquiries at his institution discovered that the consultation document had gone to members of the Association of Research Managers and Administrators (whose website is the only place with a link to the document). In Oxford University, the recipient of the document circulated it to academics for their views, but I suspect in many places this did not happen. So, if rumour is to be believed, those of us who actually write the grants don't get asked about a major change in funding policy - just those who administer grant applications. Also noteworthy that the document was released on 20th December, just before Universities close for a long break.