The difference between p < .05 and a screening test

Here's a fairly standard science story. A researcher compares a group of 30 children with autism/dyslexia/depression/chilblains and a matched control group on something: genotype/reactions to pharamaceuticals/behaviour profile/brain activation. A statistically significant difference is found between the groups. The result is published in a high profile journal.
The media want to know the relevance of this finding for their readers. It is not clear that this will lead to immediate treatment for autism/dyslexia/depression/chilblains, but they do feel justified in saying that the research will be useful in improving screening. After all, we all know that the earlier we can identify something, the easier it is to treat.

What's wrong with this? Well, alas, everything. And before going further, it is important to note that it is not just the media that get it wrong; many scientists blithely assert that their findings will have application in screening with no understanding of what is required to make a good screening test.

The four key considerations for a screening programme are take-up, costs, accuracy and intervention. Take-up can be surprisingly high, even for embarrassing and uncomfortable procedures. In the UK, most women go along with screening for cervical and breast cancer because, hey, having a total stranger introduce your private parts to cold metallic objects is, on balance, less bad than finding too late that you have a terminal disease.

But there are costs, of course. First, to the individual being screened, who has to take time out of a busy life to go along for screening, and second to the NHS, which must pay for the staff, equipment, facilities and infrastructure to do the screening. None of this is going to be worthwhile unless the screening is very accurate. If you are screening for cancer, then you cannot afford to make errors of omission: giving the all-clear when in fact the person has disease. But equally, you will be very unpopular if you make errors of commission – if you tell healthy person they have failed the screen, causing anxiety and leading them to have further tests that then fail to confirm the diagnosis. This has been a problem for the PSA screening test for prostate cancer.

Quite apart from the accuracy of diagnosis, the screening would not be worthwhile if the disorder you are screening for was not readily treatable – or preventable. There would be little point in identifying large numbers of people with disease X, even if you could do accurately, if nothing could be done about X. Indeed, it could be counterproductive.

In this light, let us consider recent stories in the media about using automated analysis of vocalisations to screen for autism. The study, by Oller and colleages, was published in Proceedings of the National Academy of Sciences in July 2010. It is an impressive and innovative study: the researchers had children wearing body-microphones so that all their utterances could be recorded, and they developed software that was able, with a high degree of accuracy, to distinguish the child's own utterances from those of other people and background noise. Using their extensive knowledge of the acoustic characteristics of babble and early language, they derived algorithms to characterise aspects such as pitch and rhythmic features in the vocalisations. They compared results for 106 typically-developing children, 77 children with autism and 49 children with language delay, and demonstrated that on a combination of these indices the groups were well discriminated. They concluded "We interpret this as a proof of concept that automated analysis can now be included in the tool repertoire for research on vocal development and should be expected soon to yield further scientific benefits with clinical implications." Screening is mentioned twice in the paper; in the abstract it states: "automated analysis should soon be able to contribute to screening and diagnosis procedures for early disorders", and in the introduction it is noted that autism is currently diagnosed mainly in terms of negative features, such as joint attention deficits and communication deficits and "abnormal vocal characteristics would constitute a positive marker that might enhance precision of screening or diagnosis." Note that these claims are pretty cautious: the vocal analysis is seen as 'contributing to' and 'enhancing precision' of screening. Nevertheless, the potential of the method as a screening tool has been emphasised in some media reports.

So how accurate is it? The key statistics for any screening instrument are sensitivity and specificity. Sensitivity is the proportion of 'hits', i.e. cases with autism who are correctly identified as having autism. Specificity is the proportion of 'true negatives', i.e, non-autistic children who are correctly identified as non-autistic. These two indices can only be defined in relation to a specific population, and their interpretation will depend crucially on the base rate of disorder in that population.

Oller et al reported identification of autism versus typical development showed sensitivity of 0.75 and specificity of 0.98. This corresponds to the situation shown below: 104/106 (98%) of typical children are accurately diagnosed, and 58/77 (75%) of autistic children are accurately diagnosed.

		Actual diagnosis
		Typical	Autistic	% autistic
Diagnosis	Typical	104	19
from vocal	Autistic	2	58
analysis	Total	106	77	42%

But their study sample was selected to contain a high proportion of children with autism, 42%. In the general population, rates of autism are much lower than this. The most liberal estimate, which includes children with far less severe problems than those included in this study, is around 1% (Baird et al, 2006). If we apply the sensitivity and specificity figures from Oller et al to a population with a base rate of autism of 1% - just by multiplying the number of typical children while keeping specificity constant – then the numbers look like this:

		Actual diagnosis
		Typical	Autistic	% autistic
Diagnosis	Typical	7471	19
from vocal	Autistic	152	58
analysis	Total	7623	77	1%

We see that in this simulated general population sample around three quarters of those who'd be diagnosed as autistic on the basis of vocal analysis are non-autistic. In reality, accuracy is likely to be lower than this in such a sample, because it would include children with milder cases of autism, including those with Asperger syndrome, who were excluded from the study by Oller et al.

It gets worse. A population sample would also include other non-autistic children with poor communication skills, including those with specific language difficulites, or conditions associated with more general developmental delay. Oller et al did include a non-autistic sample with language impairments, and it is clear from the presented data that there was considerable overlap between the autism and language impaired groups; my reading of the plotted data suggests around 43% of children with language impairment fell below the cutoff for abnormality on the vocal measures. The notion that vocal abnormality is a 'unique signature' for autism, as reported by some of the media, is clearly false.

Might there be situations where the vocal analysis could be clinically useful? Potentially, yes. Autism diagnosis is particularly difficult in young, low-functioning children, and even our gold-standard diagnostic tools can have difficulty distinguishing autism from non-autistic mental handicap in children under 18 months of age (Lord et al, 1993). The vocal measures of Oller et al might provide useful supplementary information in such cases, but bear in mind that the data presented so far suggest the vocal abnormalities can occur in conditions other than autism. Another context where the vocal analysis could be of value is in evaluation of very young children at high risk of autism, in particular young siblings of children with autism, around 10-20% of whom will develop autism or a related neurodevelopmental condition. It is possible that vocal analysis will be useful in identifying those likely to have problems. However, as noted by Zwaigenbaum et al (2006), even if we are able to identify such children early on, we then have to confront the fact that little is known about effective early intervention for autism. A further issue in the study of infant sibs is that vocal behaviours could be abnormal in such a sample if children imitate their older autistic sibling. Overall, automated vocal analysis could prove useful in early diagnosis of high risk groups, but we cannot be certain without further research. And even if we did find it could be used in this role, we would want to demonstrate that it provided information that could not be obtained by other simpler means. For instance, a brief parental questionnaire can be pretty effective in identifying autism (Berument et al., 1999).

Despite these caveats, I want to stress that I think the Oller et al study is an excellent piece of science. The method they have developed could provide important insights into the nature of communication problems in young children, and also be used to study how typical children learn to speak. It is a scholarly piece of work built on many years of painstaking research.

I've used the example of the Oller et al study because it provides a recent example of a good study whose clinical implications have been overstated, but there are many, many other instances. Take for instance, media coverage of findings of genes associated with risk for dyslexia. I just did so by Googling 'genes dyslexia screening', and found this article in the New York Times, which is typical of how genetic associations are reported in the media. The newspaper article states " Researchers said a genetic test for dyslexia should be available within a year or less. Children in families that have a history of the disorder could then be tested, with a cheek swab, before they are exposed to reading instruction. If children carry a genetic risk, they could be placed in early intervention programs". Now, I recently reviewed the research on genetic studies of dyslexia (Bishop, 2009). Even though I thought I knew the literature, I was pretty surprised by what I found. I looked at one of the more robust results in the field, on a gene called KIAA0319, described in a paper with the title " Strong evidence that KIAA0319 is a susceptibility gene for developmental dyslexia". The authors compared children with different haplotypes (allele combinations) in relation to dyslexia status. Here's my summary:

"The most common form with alleles 1–1 was equally frequent for affected versus unaffected cases, but two other common forms, 1–2 and 2–1, showed contrasting effects. (Conventionally, 1 is the more frequent allele, and 2 is the less frequent). The 1–2 haplotype was found in 35% of those with dyslexia and 27% of those without, whereas for 2–1, the figures were 24% in those with dyslexia and 36% in unaffected controls.... Extrapolating to the general population (i.e. taking base rates into account), one would expect that most individuals with the 1–2 risk haplotype would not be dyslexic, and most dyslexic individuals would not have the 1– 2 haplotype."

Note that, even without taking base rates into account, the strength of the association with genotypes is considerably lower than the Oller et al association between vocal behaviour and autism. This result causes excitement among geneticists because it has been replicated, and the evidence for association is strong in terms of p-values in a very large sample size. This finding potentially can tell us something about the development of brain regions that lead to risk for dyslexia, and it indicates that biology plays a role in dyslexia, and it is not just the result of poor parenting or poor teaching. But there is a difference between "strong evidence" and "strong effect": in fact, the effect size for individual genotypes is very small – far too small to play a useful role in screening. Since the 1970s we have had reasonably accurate ways of predicting which preschoolers are at risk for reading difficulties, based on simple behavioural tests and family background (see e.g. Satz & Friel, 1978). They are unlikely to be surpassed by any genetic tests in the foreseeable future.

Nevertheless, the belief that genes are destiny and are somehow more accurate than simple behavioural or observational measures in predicting problems is persistent. This encourages disturbing developments, such as firms offering genetic screening to identify your child's talents. Apparently you can pay to have your child tested for genes related to a host of personal characteristics, ranging from intelligence to optimism to dancing. Just one small problem: for many of these characteristics there is no published data linking gene variants to the trait. And for those that exist, the effects are tiny.

The arguments I present here are very old; I was taught about the importance of base rates by the late A. E. Maxwell when training as a clinical psychologist in 1975. However, knowledge of this aspect of epidemiology is not widespread, even among academic psychologists who are otherwise sophisticated users of statistics. I recently asked a group of MSc students if any of them knew about base rates, and they thought I was talking about the Bank of England.

So, the bottom line: just because someone finds a statistically significant difference between two groups, don't assume that will translate into a useful screening test.

(Red underline denotes corrections made 19th August 2010).

Cited references

Baird, G., Simonoff, E., Pickles, A., Chandler, S., Loucas, T., Meldrum, D., et al. (2006). Prevalence of disorders of the autism spectrum in a population cohort of children in South Thames: the Special Needs and Autism Project (SNAP). Lancet, 368 (9531), 210-215.

Berument, S. K., Rutter, M., Lord, C., Pickles, A., & Bailey, A. (1999). Autism screening questionnaire: Diagnostic validity. British Journal of Psychiatry, 175, 444-451.

Bishop, D. V. M. (2009). Genes, cognition and communication: insights from neurodevelopmental disorders. The Year in Cognitive Neuroscience: Annals of the New York Academy of Sciences, 1156, 1-18.

Lord, C., Storoschuk, S., Rutter, M., & Pickles, A. (1993). Using the ADI-R to diagnose autism in preschool children. Infant Mental Health Journal, 14, 234-252.

Satz, P., & Friel, J. (1978). Predictive validity of an abbreviated screening battery. Journal of Learning Disabilities, 11(6), 347-351.

Zwaigenbaum, L., Thurm, A., Stone, W., Baranek, G., Bryson, S., Iverson, J., et al. (2006). Studying the emergence of autism spectrum disorders in high-risk infants: Methodological and practical Issues. Journal of Autism and Developmental Disorders, 37, 466-480.

Further reading

Loong, T. W. (2003). Understanding sensitivity and specificity with the right side of the brain. (10.1136/bmj.327.7417.716). British Medical Journal, 327(7417), 716-719.

Postscript, 12^th August 2010

An article by Ecker et al that was published yesterday in Journal of Neuroscience comprehensively illustrates the points above.  The researchers gathered MRI brain scans on 20 adults with autism, 19 adults with ADHD, and 20 control adults with no neuropsychiatric diagnosis. Previous research has reported various differences between brains of people with autism and controls, but there have been few findings that are sufficiently strong and convincing to discriminate the groups. Indeed, it has been surprisingly difficult to find any consistent abnormalities on brain scanning, even in the case of children with severe impairments. Ecker et al adopted a different approach, by arguing that a constellation of features would be more effective than individual indices in distinguishing autism from control brains.

The three groups did not differ in terms of three measures of brain capacity: intracranial volume, total brain volume, and gray matter volume.

They used five measures to derive a 'classifier': an equation that puts the measures together in a way that maximally distinguishes the autism and control groups. These measures were (as far as I understand the highly technical account):

1. The average convexity or concavity of large-scale features of the brain surface. This measures sulcal depth and gyral height
2. Mean (radial) curvature. This assesses number of smaller convolutions on the surface of the brain.
3. A measure of the degree of cortical folding in terms of metric distortion relative to an average template.
4. Cortical thickness
5. Pial area, a measure of the grey matter surface.

To avoid the bias that can arise when the same data is used to derive and test a classifier, they adopted a 'leave one out' method, whereby one person with autism and one control were omitted when deriving the estimation of the equation. The resulting equation was then applied to classify the left-out cases. This was done separately for the left- and right sides of the brain.  Only the left-sided classifier was effective, giving the following results.

		Actual diagnosis
		Autistic	Control
Diagnosis	Autistic	18	4
from MRI	Control	2	16
classifier	Total	20	20

When the same left-hemisphere classifier was appplied to the ADHD group, the picture was similar. 4 of 19 cases were classified as autistic, and the remainder as non-autistic.

In the discussion of the results, the authors note the potential for clinical application as follows:

"... the existence of an ASD biomarker such as brain anatomy might be useful to facilitate and guide the behavioral diagnosis. This would, however, require further extensive exploration in the clinical setting, particularly with regards to classifier specificity to ASD rather than neurodevelopmental conditions in general. "

In other words, the classifier needs to be validated on other samples, and it is as yet unclear whether it would be as accurate if people with autism were contrasted, say, with people with language impairments. The authors also noted that the adult sample used here was largely composed of high-functioning adults with Asperger syndrome, and it was not clear whether results would generalise to other cases of autism.  In the final sentence of the paper it is stated: "classification values and specific patterns we report must be considered as preliminary".

In sum, the paper illustrates a promising approach to understanding more about brain development in autism, but it does not demonstrate the utility of this approach for differential diagnosis of autism from other conditions, nor for assisting diagnosis in marginal and difficult cases.

Encouraged by an enthusiastic press release from the Medical Research Council, who funded this work, the media picked up the story yesterday.  It was featured prominently on BBC television and radio news, made the front page story on the Independent, and was given extensive coverage in other national newspapers.  The account of the work showed little evidence of the caution and qualification in the published account.  In this case, I don't think the media can be blamed for misrepresenting the work, as much of what was said was taken from the MRC press release.  Here are some statements from the Independent, under the headline: "Brain scan promises to identify the hidden sufferers of autism":

• Autism could in future be diagnosed in 15 minutes from a brain scan

• Scientists ... have devised a method which can distinguish the autistic brain from the normal brain with 90 per cent accuracy.

• The method could lead to the introduction of screening for the disorder in children.

Dr Ecker was quoted as saying:

"The [computer programmme] says yes this is autism or no it is not, and also gives an indication of the severity."  She also contrasted the 15 minutes it took to do an MRI scan with conventional diagnostic procedures which "takes the whole day and involves asking embarrassing questions such as how many friends they have and how they are doing at school"

Dr Ecker seems to be implying that we should stop using diagnostic interviews to diagnose autism, because a brain scan is more accurate.  Since autism is currently defined in terms of a pattern of impairments in different domains of functioning, this is a remarkably bold claim, especially since her method awarded an autism diagnosis to 20% of controls.

Much is also made of the potential of the method for diagnosing autism in children. It's possible this may be effective, but children's brains are very different from adult brains, and one would need a whole new study to evaluate this. Furthermore, although Dr Ecker says of brain scanning "It doesn't hurt and you can even go to sleep", it is not an easy procedure to apply to young children without sedation.  Lying still in a very noisy tube is not something that comes naturally to a child, even if you can explain the procedure to them clearly so they know what to expect. Most children with autism have poor language comprehension and may also become anxious when in an unfamiliar situation. It is not impossible to do brain scanning studies with them, but it certainly is not easy.

A final thought. In recent years, brain research on autism has increased exponentially. There are other MRI datasets for adults and children with autism and other clinical groups as well as controls. It should be possible for Ecker and colleagues to cross-validate their findings fairly rapidly against other datasets. I hope that others will be willing to share their data to make this possible.

Book Review: The Invisible Gorilla

Chabris, C., & Simons, D. (2010). The invisible gorilla and other ways our intuition deceives us. London: HarperCollins. Psychology is a much misunderstood discipline. If you go into a high street bookstore, you will find the psychology section stuffed with self-help manuals and in all probability located next to the section on witchcraft and the occult. This is partly the fault of the Dewey Decimal Classification, which sandwiches Psychology firmly between Philosophy and Religion. For those who regard experimental psychology as a scientific discipline with affinities to medicine and biology this is a problem, and some psychology departments have dissociated themselves from the fluffy fringes of the discipline by renaming themselves as departments of cognitive science or behavioural neuroscience. An alternative strategy is to reclaim the term psychology to refer to a serious scientific discipline by demonstrating how experimentation can illuminate mental processes and come up with both surprising and useful results. This book does just that, and it does so in an engaging and accessible style. The book starts out with the phenomenon referred to in the title, and which the authors are best-known for, i.e. the Invisible Gorilla experiment. This has become well-known but I won't describe it in case the reader has not experienced the phenomenon. Richard Wiseman has a nice video demonstrating it. This is perhaps the most striking example of how we can deceive ourselves and be over-confident in our judgement of what we see, remember or know. In all there are six chapters, each dealing with a different 'everyday illusion' to which we are susceptible. My personal favourites were the last two chapters, which consider why people continue to believe in notions such as the damaging effect of MMR vaccination, or the beneficial effects of brain training for the elderly. Sceptics tend to dismiss those who persist such beliefs in the face of negative evidence, and denigrate them as stupid and scientifically illiterate. Chabris and Simons, however, are interested in why scientific evidence is so often rejected and consider why it is that anecdotes so much more powerful than data, and why we are sucked in to assuming there is causation when only correlation has been demonstrated. My one disappointment was that they did not say more about the reasons for wide individual variation in people's scepticism. After a rigorously sceptical undergraduate course in experimental psychology at Oxford, I assumed that all my peers on the course would be sceptics through-and-through, but that is far from being the case: I have intelligent friends who learned all about the scientific method, just as I did, yet who now are adherents of alternative therapies or psychoanalysis. I find this deeply puzzling, but it makes me realise that the satisfaction I find in the scientific method is in part due to the fact that it resonates with the way my brain works, and there are others for whom this is not so. In sum, I enjoyed this book for the insights it gave into how people think and reason, and for its emphasis on the need to adopt scepticism as a mind-set. Its avoidance of jargon and clear explanations give it broad appeal, and it would make an ideal text for undergraduates entering the field of experimental psychology, because it illustrates how a good experimenter thinks about evidence and designs studies to test hypotheses.

An exciting day in the life of a scientist

or: How to kill a few hours trying to get publication quality figures out of Matlab This is really just a boring moan; blogging as therapy. Well, yesterday I had the excitement of getting proofs for an 'in press' article. Virtually no errors to be corrected, but what was this list of queries from the publisher? Ah, the figures. Resolution too low. Well, that should be easy to fix - I'd do it first thing. Or so I thought. 9.30 a.m. The paper has an unusually large number of figures, eight, some in colour. All were created in Matlab and saved as .tiff format. I was pretty proud of generating the figures in Matlab. Graphics in Matlab is a bit of a nightmare and takes some time to learn, but once learned, you can generally create figures that are more complex than those produced by the other applications that I know. The proofs have come from Developmental Science, but they tell me the figures are too low resolution, even though I'd selected a 'no compression' option when saving them. Coincidentally, I have another article that is under consideration by Journal of Neuroscience, who also mention their stringent requirements for figure quality, and point me to a website, Cadmus, that will explain what is required and how to do it. Oh good, I think. Someone will help walk me through how to get good quality figures. Ha ha ha. Cadmus has a list of programs and formats that are supported. Alas, Matlab not among them. But Adobe Illustrator is. We have a copy of that. I used to have it on my machine, but uninstalled it, because I never used it and I got fed up when graphics files defaulted to opening in it, which took ages. Tracked down the CD, reinstalled it (compact version). Right, I think, Matlab will allow me to export a file in .ai format, and then I will be OK. Ha ha ha 9.45 I start with the simplest figure I have – a simple black and white line drawing with a couple of text labels. I save it with .ai format. When I click on it, Adobe Illustrator tries to open it, but first tells me it has 'unrecognised fonts' (Arial?) and then says it can't open it. OK, I think. But I can open an .eps or .tiff file in Adobe, and I can also save my Matlab figure in those formats. But once again, I get strange messages about wrong fonts, and for the .eps version, what appears on the screen is unrecognisable from the original. I look again at what Cadmus says about Adobe Illustrator. Oh dear. "PLEASE NOTE: When creating graphics in illustration programs such as Adobe Illustrator with the intention of outputting to an imagesetter or platesetter, it is extremely important that the person creating the illustration have a thorough understanding of the details of imaging in a prepress environment. There are an abundance of complex problems that can occur at output if paths are set up improperly, colors are indicated incorrectly, or other elements are constructed improperly. Trapping issues can also present problems if not addressed. The more complicated your illustration becomes, the greater the probability of problems at output, and therefore the need for more expertise and experience in creating the files." Decide that I had better try another option, since I have never used Adobe Illustrator and would not recognise a platesetter if I stumbled over one. But, I think, there is a helpful application associated with Cadmus that allows you to check your files. And I can just open my .eps file from there. Having gone through the usual round of registering, thinking of a password, getting email confirmation of the account, etc. I am in to 'Rapid Inspector'. I try opening my .tiff file. FAIL says Rapid Inspector. Resolution too low. OK, how about .eps version? Ah, says Rapid Inspector"Rapid Inspector found an image with CMYK color. CMYK color is not supported. Acceptable color space include(s): Spotcolor, Lineart, Grayscale, RGB." But this is a black and white figure! I spend some time in Matlab trying to sort this one out, but with no success. My own fault, but I can't find the script that I made to generate the figure in the first place, and I will need to redo it with different fonts etc. So waste 10 mins tracking it down and resolving once again always to save my programs in sensible places with sensible names. I go on to the web to find out how to change the colormap to gray. Re-run program, save the figure, and try it again in Rapid Inspector. It still tells me I have CMYK color. It also complains about my fonts. "Rapid Inspector detected that some or all fonts are missing from this file. To pass inspection, all fonts must be embedded. The following fonts are not embedded: Helvetica. " That's odd, as I was using Arial, not Helvetica. Try a few more runs of the program with different fonts. It still doesn't like my fonts. 10.45 Time to do a Google search about how to save a Matlab figure with embedded fonts. Well, it is nice to know I am not alone, and that many others have had this problem over the years. Several complain that it is about time Matlab did something about it. One helpful person, Oliver Woodford, has written a routine called export_fig, which is freely available: http://www.mathworks.com/matlabcentral/fileexchange/23629 Excellent. But, he explains, if you want to use it to create the kinds of files I need, you need to download two other applications from other sources. Fortunately, I already have the first, but the second, xpdf, is one of those applications that makes the non-geek's heart sink when you go to the download webpage and find, instead of clear instructions about what to do, a whole list of possibilities. I fear that the one I probably need ends in .tar.gz. I've tangled with these things before but can never quite remember what to do with them. 11.30 After a bit of fiddling about, I save the .tar.gz file, then try to extract the contents. A few failures as I do something wrong, and then at last I have it. But I am not sure I have it in the right place and no indication is given as to where it should be saved. I've just stuck in my Matlab program folder. 12.00 OK I should be all set, so now let's look at the examples of how to use export_fig. Nice helpful man who wrote the script clearly has been through everything I have, and more. He writes: "Exporting a figure from MATLAB the way you want it (hopefully the way it looks on screen), can be a real headache for the unitiated, thanks to all the settings that are required, and also due to some eccentricities (a.k.a. features and bugs) of functions such as print. The first goal of export_fig is to make transferring a plot from screen to document, just the way you expect (again, assuming that's as it appears on screen), a doddle." This is looking more promising.... Print out the instructions – 13 pages of them. 12.30 Took a break to look at some interesting data: what I ought to be doing instead of this rubbish. 14:30 OK back to export_fig. First attempt failed. Matlab can't find export_fig. I need to put the script somewhere else. OK,eventually sorted that by putting all the export_fig m files onto the Matlab folder in My Documents. All going very well so long as I am exporting to .png format. But I want .eps. When I try that the program complains it needs pdftops . SO where the hell is that? I will have a hunt. Found it, but it is a .cc file. does not seem to be recognised by matlab. So I have now spent more time on the website looking for a .m file. Doesn't appear to be one. Gave up and decided to try a .tiff file. Hah! nasty bossy Rapid Inspector says PASS. Hooray! But turned out I was reading in a different file created with the same name in May. Back to my .tiff option in export_fig This fails resolution test, even though it is specified as max quality. 15.00 Have a cup of tea. Back to trying the .eps option. Can't work out how to use the xpng file. Program stops and asks for pdftops. I have located pdftops.m and pdftops.cc but neither seems what it wants. As far as I can see from looking at the code, it wants an .exe file. The web tells me that a .cc file is a C++ file. In some desperation I tried renaming the .cc file as .exe, but that did not work. Decide to write to the author of the script, having read all the comments on the program and found that nobody else is having problems. Send the email. It bounces. I had mistakenly included a full stop at the end of the email address. Try to resend to correct address: email keeps autocompleting to the address with the full stop. After 2 tries, get into 'frequent contacts' in address book and delete entry so can now send the email. 15.35. I need another cup of tea to calm down. So now trying figure 2 , coloured headplot. Already have as .tiff; it looks very nice. Rapid inspector tells me FAIL! resolution is too low. I try saving as .png. Get lovely looking picture. Rapid inspector won't read it. Try exporting from microsoft image reader to .tiff and then reading in. Now I get: "alpha_planes: Rapid Inspector found extra color channels within this image. Extra color channels are also known as Alpha channels. Alpha channels are not supported. Please use an image editor to remove alpha channels from this file.resolution: Rapid Inspector found a low-resolution (RGB) image (96 DPI). The minimum required resolution for this type of image is 300 DPI. " 16:30 The wonderful Oliver Woodford replied and explained patiently how to cope with the pdftops thing. I downloaded. Still did not work. Downloaded again to the location he had said his was in . It works!! And the figures it creates are acceptable to the wretched Rapid Inspector. Verdict I'm really grateful for those who have produced free software that helped me deal with this. But I am really annoyed on two counts. First, Matlab is an expensive package. It does wonderful things and I love it to bits as a programming tool, but its graphics are not easy to use. People have been complaining for at least 2 years about the difficulty of generating high resolution output, yet nothing has been done. It should be high priority for the Matlab developers to fix this so that there is a simple command to generate this kind of output. Second, the Journal of Neuroscience exemplifies a trend in many journals to make authors do a lot of work that would, in the old days, have been done by copy-editors and other professionals. Scientists are supposed to have skills in graphic design and programming on top of all their other accomplishments. Some journals do still accept figures in a range of formats and look after any conversion from their end. But increasingly, the onus is put on authors. There appears to be no correlation between the wealth of a journal and the amount of help it will give to authors – in fact, if there is a correlation, I suspect it is inverse. Journal of Neuroscience charges hefty fees for just submitting a paper, let alone publishing it, with added costs of $1000 per figure unless first and last authors are members of the Society for Neuroscience – we are not and we have lots of colour figures. So our grant will be spent on shoring up J. Neuroscience rather than employing a vacation student for a few weeks. I reckon that on a 1-10 scale of geekiness I am a 6-7, and I am struggling. I am a full-time researcher with good support. I am a reasonable programmer. But I've got lots of colleagues who are trying to produce papers who are closer to a 1 or 2 on the geekiness scale, have little or no support, and are trying to fit research around busy teaching commitments. How on earth can they cope with all of this?

Journalists and the "scientific breakthrough"

There has been some animated discussion about science journalism by journalists and bloggers relating to the Fish Oil story that featured in my 1st June blog on the Orwellian prize. In summary, Ben Goldacre wrote a piece criticising the lax journalism behind the story, and was then roundly criticised himself by Jeremy Laurance of the Independent, who felt Ben was going too far in attacking hard-pressed journalists. There is a good piece by Ed Yong that gives the background and offers a cogent analysis. Here I don't want to rehearse the arguments – I've already added a comment to Yong's blog underlining my support for Ben Goldacre. Instead I have a constructive suggestion from a scientist to science journalists about how they might do things differently when handling science stories. This is prompted by my experiences when attending meetings of the British Science Festival. The British Science Association, which runs the festival, has a distinguished track record: founded in 1831 as the British Association for the Advancement of Science, its meetings included highlights such the first use of the term 'dinosaur' (1841), the debate on Darwinism between Huxley and Wilberforce (1860), Joule’s experiments (1840s) and the first demonstration of wireless transmission (1894). Times, however, have changed. Science has become increasingly specialised. The British Science Festival plays a key role in fulfilling the stated goals of the Association: "connecting science with people: promoting openness about science in society and affirming science as a prime cultural force through engaging and inspiring adults and young people directly with science and technology, and their implications". It is, however, not a venue where scientists reveal to the world a hitherto unreported "breakthrough". In general, the material that is reported at the Festival will either be already published in specialist journals, or will be "work in progress". In fact, most scientists find the whole concept of the "breakthrough" suspect: most research proceeds incrementally rather than by a sudden leap forward. The media, however, don't seem to grasp this essential point. They descend on the meeting in droves where they summon presenters to press conferences based on the one-paragraph press releases that the scientists are encouraged to supply. They don't attend the lectures. If my experience is anything to go by, they focus on at most 5% of the presentations, selecting those that they judge will make a good story of interest to the general public. Then during the week of the Festival, there is a plethora of newspaper articles which report on the selected presentations, inevitably talking about them as if the work is entirely new. Scientists then get uncomfortable as they feel their research has been "talked up" to make it into a newsworthy "breakthrough". The following week, the newspaper coverage of science reverts to its previous low level. Now, while I can understand some of the reasons for this, it seems to me a terrific waste of an opportunity. Gathered at the Festival is a subset of Britain's scientific elite, selected in large measure because they are doing interesting research and can communicate their science to a general audience. (Inevitably there are some who are hopeless at doing this and misunderstand the nature of the audience, but they are a minority). Their work doesn't stop when the Festival comes to an end. They will return to their labs and continue doing the interesting research. So if I were a science journalist, I would go to the Festival not to frenetically write stories to be published that week, but to make contacts with researchers who could supply me with good science stories for the rest of the year. I'm sure most journalists don't have time to tour the country interviewing scientists, but if they could make a personal contact with a scientist and persuade them they want to write a serious piece, they would find most would be amenable to a future telephone interview. And instead of having to respond rapidly to a press release, journalists could take a more relaxed approach, which would reduce their stress levels and give them time to check things carefully. I suspect that many journalists will reply to say this is all very well, but it's not news and so editors would not stand for it. Their job is not to provide the public with a general science education, but to tell people about breakthroughs. I disagree. There is far too much science for any one person to grasp, and so most scientific research will be novel to the majority of readers. The difficulty is to explain what is often technical and complex material to a lay readership. We have some excellent science journalists who are good at doing exactly that. If they could build up a network of relationships with scientists, they'd find a great many fascinating stories out there that are currently appreciated by only a minority of specialists.

Book review: biography of Richard Doll

Smoking kills: the revolutionary life of Richard Doll by Conrad Keating Signal Books: Oxford, 2009 I only once had the opportunity of hearing Richard Doll speak. It was in 2004, after a hard-hitting lecture by Charles Warlow on over-regulation of research (http://tiny.cc/m1n7v). Doll, aged 92, looking physically frail but mentally alert, stood up to point out that under current ethics regulation, none of the epidemiological studies that he had done to demonstrate a link between smoking and cancer would have been possible. The only way to rescue epidemiology from the trend for increasingly strict and bureaucratic constraints, he felt, was for someone to deliberately flout the law and be prepared to go to prison for the cause of liberating research. "I am prepared to volunteer myself for that role", he said, prompting a spontaneous outburst of applause. This engaging biography of Doll paints a picture of a man who could have been trusted to follow through on this offer. But as well as telling us about the remarkable life of Doll himself, the book is fascinating for its account of the history of evidence-based research in medicine. Doll would have gone to Cambridge to read mathematics were it not for the fact that he flunked one of the examinations after being plied with strong alcohol the night before. That fateful event led him to study medicine at St Thomas's Hospital, but he never lost his love of mathematics, and was quick to see the potential for using statistical methods and randomisation to evaluate treatments. Together with Austin Bradford Hill he developed the method of randomised controlled medical trials which has become the cornerstone for evaluation of efficacy of treatments. He subsequently took up a post researching disease prevention, a niche that not only suited him well, but was also one of the few options open to him, as his membership of the Communist party made him virtually unemployable as a hospital doctor. Remarkably, Home Office regulations meant he was also banned from teaching undergraduates, although he was allowed to teach postgraduate students. The account of his seminal work on smoking and cancer was a revelation to me. I had no idea that in the 1950s it was commonplace for doctors to offer a cigarette to a patient to calm them down before an examination. Nor that there had been a six-fold increase in lung cancer mortality in males between 1930 and 1944. Many of the discussions around this epidemic are reminiscent of those currently seen in the field of autism – was there some factor that could explain the increase, or was it simply that detection of the disease had improved, especially with new X-ray methods for visualising tumours? Doll was remarkable for his single-mindedness and energy in assessing large numbers of patients, and for his ability to see the need for controls. He was ever alert to the possibility of misinterpreting data and developed methods of guarding against unwitting bias. Calculations were done by hand and painstakingly checked. Doll was concerned not to publish until certain of his facts, leading to some delay in publication of the first results. Keating emphasises the extent to which Doll felt it important not to become a campaigner for public health measures, but merely to present the results of his findings. It is a view nicely summed up by a comment attributed not to Doll, but to his collaborator Hill, who was confronted at a cocktail party by a doctor who said "You're the chap who wants us to stop smoking." "Not at all," replied Hill. "I'm interested if you go on smoking to see how you die." As each new study by Doll found further support for a strong link between smoking and lung cancer, and also revealed an unexpected link with heart disease, initial apathy towards the results turned into acceptance by the Medical Research Council, criticism by the tobacco industry, who derided the findings as 'unscientific', and alarm by the government, who made substantial sums in taxes on smoking. One of the harshest critics was the eminent statistician and heavy smoker R. A. Fisher. Fisher was exercised by a key limitation of epidemiological studies, the lack of experimental control. "Correlation does not mean causation" was his mantra, and he argued forcefully for alternative explanations of the association. The debate turned acromonious, with Fisher accusing Doll and Hill of failure to make their data available, and others accusing Fisher of conflict of interest, owing to his lucrative consultancies with tobacco companies. Again, aspects of the debate have some modern resonances: in my own field of neuropsychology, it is often argued that the goal is to find the necessary and sufficient cause of a disorder, much as Fisher had argued. On this view even a single case of lung cancer in a non-smoker could overturn a causal theory. But gradually it became clear that a factor such as smoking could act as a risk factor for disease without acting in a deterministic fashion. In my own writings on developmental neuropsychology I have used the example smoking and cancer to make the point that causal mechanisms may be multifactorial and probabilistic (http://tiny.cc/i8x5l), but until I read this book I had not realised that there was a bitter debate around this very issue in the 1950s. Keating describes it as a shift from reliance on Koch's postulates to Hill's specification of criteria for causality. Koch, working on infectious diseases had specified that, among other things, to prove an organism was the cause of a disease the organism had to be discoverable in every instance of the disease, and that experimental animals could be given the disease by inoculation with a pure culture. Hill's criteria involved nine viewpoints, none of which on its own gave indisuptable evidence for a causal effect, but which taken together would help determine whether causality was a reasonable hypothesis: strength, consistency, temporality, biological gradient, plausibility, coherence, experiment and analogy. Doll is best-known for his work on smoking, but he also played a pivotal role in discovering the health risks of asbestos, ionising radiation and the contraceptive pill. Keating tackles head-on the accusations that emerged after Doll's death that he was in the pay of companies such as ICI and Monsanto, and refutes firmly any suggestion of malpractice. Doll believed that if one wanted to obtain access to data it was important to have some relationship with companies. This emerged most strikingly in his early work on asbestos, where his preliminary findings caused shock and alarm. He resisted attempts to suppress the data, but negotiated with the company where he had done his studies with the aim of doing further analyses. The company were willing to implement changes to their working practices and Doll judged that more good could be done by working with them than attacking them: after Doll's findings, measures for containing asbestos fibres were taken so that the workers could at last see that there was a clock on the far-side wall. Doll made no personal profit from this company, nor from his work for chemical firms, whose fees were donated to charity. Nevertheless, the book raises big questions about how to handle potential conflict of interest and illustrates the importance of modern practices of full disclosure – which only became commonplace after his retirement, and of which Doll fully approved. I can thoroughly recommend this book as providing a stimulating account of a fascinating life. It puts into historical perspective aspects of medical statistics and epidemiology that are now largely taken for granted, but which may have taken much longer to be developed without Doll's intelligence and dedication. A final word of thanks to the Book House, Summertown, the small, independent bookseller where I happened upon this book. Please support them if you live in Oxford: I can guarantee that if you wander in and have a browse, you will find something interesting and unexpected on the shelves. (And no, I don't have any vested interests – just keen for the survival of a local amenity!)

Orwellian prize for journalistic misrepresentation

CALL FOR NOMINATIONS


I am offering a prize each year for an article in an English-language national newspaper that has the most inaccurate report of a piece of academic work.

The prize will consist of a certificate and statuette and I would welcome suggestions for the design of both of these

.

The prize will be awarded in January of each year.

Rules

1. The article must purport to report results of academic research, and judgement will be based on a points scoring system, as follows:

    * Factual error in the title: 3 points
    * Factual error in a subtitle: 2 points
    * Factual error in the body of the article: 1 point

2. Factual errors must be ones that can be judged against publicly available documents – i.e. not just opinions or reports of interviews.

3. Nominations must be posted on this blog. The nomination should contain:

    * Web addresses for both the nominated article and the academic source that is misrepresented.
    * Name and email contact of the nominator. Anonymous nominations are not allowed
    * A scored copy of the article, as illustrated below

If a nominated article is not available electronically, then the nominator should provide a list of the points used to score the article, and retain a photocopy of the article, which should be provided to the judges on request.

4. If there is more than one plausible candidate for the prize, then additional criteria will be used, such as:

    * The seriousness of the error, e.g. could it damage vulnerable groups?
    * Relevant undisclosed vested interests by journalist or his/her newspaper
    * The ratio of accurate to inaccurate content
    * The presence of irrelevant but misleading content
    * The size of the readership

and mitigating circumstances, such as

    * Whether there was a misleading press release from the academic's institution
    * Whether a scientist colluded in 'talking up' the findings and going beyond data

Nominators are encouraged to comment on these points also, but final judgement will be made by a panel of judges.

Illustrative example of nomination:

The following article is a strong contender for the prize in 2010, and illustrates the scoring system.

Inaccurate sections shown in square brackets with N points in round brackets.

Nominated article: http://www.guardian.co.uk/science/2010/may/30/fish-oil-supplement-concentration

Misrepresented source article: http://www.ajcn.org/cgi/content/abstract/91/4/1060

Nominator: DVM Bishop, dorothy.bishop@psy.ox.ac.uk

[Fish oil](3) helps schoolchildren [to concentrate](3)

US academics discover [high doses of omega-3 fish oil](2) combat [hyperactivity and attention deficit disorder](2)

* Denis Campbell

* The Observer, Sunday 30 May 2010

Children [can learn better at school](1) by taking [omega-3 fish oil supplements](1) which [boost their concentration](1), scientists say.

Boys aged eight to 11 who were given doses [once or twice a day](1) of docosahexaenoic acid, an essential fatty acid known as DHA, showed [big improvements in their performance during tasks involving attention](1).

Dr Robert McNamara, of the University of Cincinnati, who led the team of American researchers, said their findings could help pupils to study more effectively and potentially help to tackle both attention deficit hyperactivity disorder (ADHD) and depression. The study, reported in the American Journal of Clinical Nutrition, is important because a lack of DHA has been implicated in ADHD and other similar conditions, with poor maternal diet sometimes blamed for the child's deficiency.

ADHD affects an estimated 4%-8% of Britons and can seriously impair a child's education because they have trouble concentrating and are often disruptive in class. A lack of DHA has also been associated with bipolar disorder and schizophrenia.

"We found that, if you take DHA, you can enhance the function of those brain regions that are involved in paying attention, so it helps people concentrate," said McNamara. "The benefit is that it may represent an intervention that will help children or adults with attention impairments."

The researchers gave 33 US schoolboys 400mg or 1,200mg doses of DHA or a placebo every day for eight weeks. [Those who had received the high doses did much better in mental tasks involving mathematical challenges](1). Brain scans showed that functional activity in their frontal cortex – which controls memory, attention and the ability to plan – increased significantly.

The results, and fact that many people eat too little fish to get enough DHA through their diet, meant it could help all children to improve their learning, added McNamara. "The primary benefit is to treat ADHD and depression, but it could also help people with their memory, learning and attention," he said.

--------------------------------------------------------------------------------------------------------------------

12th September 2010

New nomination by Jon Simons, who has pointed out to me that there is a word limit on Comments, which makes it difficult to post nominations there. If you have a nomination, please save as a text file and send to me (email above) and I can post it here.

Nominated article:

http://www.washingtonpost.com/wp-dyn/content/article/2010/09/09/AR2010090904116.html

Original article: http://www.sciencemag.org/cgi/content/abstract/329/5997/1358

Nominator: Jon Simons (jss30@cam.ac.uk)

Scientists can [scan brains for maturity](3), potentially [gauging child

development](3)

By Rob Stein, Washington Post Staff Writer, Thursday, September 9, 2010; 6:13 PM

Scientists have developed a scan that can [measure the maturity of the brain](1), an advance that someday might be useful for [testing whether children are maturing normally](1) and for [gauging whether teenagers are grown-up enough to be treated as adults](1).

A federally funded study that involved [scanning more than 12,000 connections](1) in the brains of 238 volunteers ages 7 to 30 found that the technique appeared to accurately differentiate between the brains of adults and children and determine roughly where individuals scored in the normal trajectory of brain development.

 While much more work is needed to validate and refine the test, the technique could have a host of uses, including [providing another way to make sure children's brains are developing properly](1), [in the same way doctors routinely measure other developmental milestones](1). [The scan could, for example, identify children who might be at risk for autism, schizophrenia and other problems because their brains are not maturing normally](1).

"If you are worried about a kid's development, in five minutes you could do a scan and it would spit out a measurement of their brain maturity level," said Nico Dosenbach, a pediatric neurology resident at St. Louis Children's Hospital who helped develop the technique described in Friday's issue of the journal Science. "That's sort of the future."

The technique developed by Dosenbach and his colleagues uses magnetic resonance imaging, already commonly used to measure activity in the brain by correlating increases and decreases in [blood flow](1) to various brain regions. The scans are considered safe because they do not use radiation.

In this case, the technique was called functional connectivity magnetic resonance imaging, or fcMRI, because it [measured connections](1) in the resting brains of the subjects. The researchers used a computer program to analyze how [connections in the brain changed as the mind matured](1), pinpointing 200 to produce an index of maturity. They found that [close connection weakened while distant connections strengthened as the brain matures](1), until about age 21 or 22.

Dosenbach estimated they were able to distinguish between the brain of children ages 7 to 11 and that of adults ages 25 to 30 with 90 percent accuracy. They were able to differentiate between adolescents and adults with 75 percent accuracy, Dosenbach said in an e-mail.

But Dosenbach warned that it would be premature to start using the technique to measure individual maturity levels.

"I would not endorse that," he said.

Postscript : Academic mobbing in cyberspace

Remarkably, if you scroll down to the end of the petition you find a couple of entries where people have signed up in order to warn others not to sign! It seems that the petition is really treated more as a blog. Academic Fraud by Evans is the the charge. This petition has nothing to do with Academic Freedom. This is a travesty and we hope Dawkins and Pinker come down very hard on Evans Look at the facts and and evidence-- we have been fooled Evans is a pariah from now on-- His outrageous fraud by Web 2.0 is beyond contempt. Violet Memills. Do NOT sign this petition. It is possible that the bandwagon of backlash against political correctness is being used to cover bullying and abusive behaviour. It is best to let the situation resolve itself without externally fabricated public pressure to judge one way or the other. UCC staff member

Academic mobbing in cyberspace

On 14th May 2010, an online petition was set up for supporters of Dr Dylan Evans, an academic at University College Cork.

http://www.ipetitions.com/petition/freedebate/blog

The blog accompanying the petition stated that:

---------------------------------------------------------------------------

The article that caused offence was published in PLoS One a leading international peer-reviewed scientific journal. It was entitled "Fellatio by fruit bats prolongs copulation time". To read the full article, click on this link:

http://www.plosone.org/article/info:doi/10.1371/journal.pone.0007595

The article was covered extensively in the international press. See, for example, this piece in The Guardian:

http://www.guardian.co.uk/education/2009/nov/10/oral-sex-bats-improbable-research

In early November 2009, a lecturer at University College Cork showed this article to over a dozen colleagues. One of those colleagues later filed a formal complaint of harassment against the lecturer, even though in this particular case the article was shown in the company of third person, and was brought up only in the context of an ongoing academic discussion about the relevance of evolutionary theory to human psychology. The colleague displayed no signs of offense at the time, and indeed requested a copy of the article. Nobody else complained.

External investigators were appointed. Their report exonerated the lecturer. Despite this, the President of UCC has imposed harsh sanctions on the lecturer.

------------------------------------------------------------------------------------------------

By the end of the day on 15th May, 1455 individuals had supported the petition. After online publication of documents relating to the case on 17th May (http://felidware.com/DylanEvans/), and further blogs by Dr Evans, more critical commentaries began to appear, challenging whether this really was a case of an attack on academic freedom. Nevertheless, petition supporters numbered over 3700 by 29th May.

My goal here is not to argue the rights and wrongs of the case, which have been debated at length on the website

http://scienceblogs.com/pharyngula/2010/05/bat_sex_is_not_protected_by_ac.php

Rather, I am concerned by the way in which Evans was able to rally support from a large number of academics on the basis of a single blog.

My personal reaction on hearing of the petition was to check a few facts to see if there was another side to the story. The first point to emerge was that neither Evans nor his colleague worked in evolutionary biology or zoology. His post was in a Medical Education Unit in the School of Medicine, and his PhD was in philosophy. I then looked at the bat sex paper, which was conveniently published in an open-access journal. The abstract gave a graphic description of fellatio in bats. I thought that if a male colleague were to come into my office to show me the paper, I would find it rather embarrassing, unless it were in the context of prior discussion of animal sexuality. However, I'm of an age where explicit discussion of penises doesn't crop up much in conversation and I thought I'd run a straw poll past some more junior female colleagues. Essentially, there was broad consensus that interpretation of Evans' behaviour was entirely dependent on context. There were three ways it could be taken:

a) as a genuine attempt to use the bat paper as part of an ongoing academic debate;

b) as a jokey interaction– of the "wow, look at this wacky article" kind; this could be OK but ill-advised unless you knew the colleague well enought to be able to predict she'd find it amusing rather than embarrassing or offensive

c) an attempt by Evans to initiate a sexually explicit conversation with a female colleague, comparing bat and human behaviour in a way that could be construed as harrassment.

Reading the background material, it is clear that Evans presented the situation as (a), the complainant presented it as (c), (occuring in the context of prior inappropriate behaviour), and the university authorities decided to interpret as an instance of (b) – but inappropriate because the colleague did not feel comfortable with Evans showing her this material, hence they concluded he needed some counseling.

Now what interests me is that, if all the women I contacted could identify (c) as a possible context, why didn't the petition supporters see it? Note that neither I nor my straw poll respondents assumed (c) is what happened, but we all saw it as a possible scenario. Academics are supposed to weigh evidence and use their brains rather than emotions when evaluating evidence, yet here there seems to have been a collective failure to think clearly among those signing the petition.

To look at this further, I decided to analyse the responses to the petition in more detail. Here are some statistics.

Gender of signatories and proportions commenting

After removing duplicates, I found 3123 signatories.

364 (12%) were anonymous

293 (9%) had names that did not reveal gender.

Of the remainder, 79% had male names.

It would be interesting to know how this compares with gender ratios for other online petitions. It looks like a strong male bias, but it is possible that men are generally more likely to respond to online petitions than women, so we should be cautious in interpretation.

58% of the signatories just put their name to the document without adding further comment.

I classified the comments (N = 632) for the 1455 respondents who signed on 14th-15th May (would have liked to read all, but ran out of time!). Each comment was categorised according to any or all of the categories that applied.

Uncertainty

The first thing I considered was whether the endorsement of the petition was qualified by any expression of uncertainty or hedging. Just 2% of respondents made such a comment (my italics, below), .e.g

1. " If this is true as presented, then it's scandalous conduct by UCC's HR department." Dr Steve Rossiter

2. "Looking at the situation as reported and the article, it looks like they were judging quite fairly" Ed Sketch

3. I am acquainted with the facts only as related in this petition, but I find it hard to imagine any extenuating circumstances that could render any other judgment than that this is an outrageous violation of academic freedom Daniel C. Dennett

However, a similar number expressed certainty about the events that had occurred, without giving any reason for the certainty (e.g. being present at the events, or having personal knowledge of Evans and trusting his integrity)

4. "I can see that such an article could be used deliberately to upset someone but I am convinced that this did not happen in this case" Professor Susan Blackmore.

5. Academic Freedom should not be impeded. I am sorry someone got upset about the article, but I am certain that it was a personal issue and not sexual harassment. Steven Platek

6. Dr. Evans' actions were obviously not harassment, nor were they some kind of implication of harassment. The colleague in question is kind of obviously being intellectually dishonest about their position, and is attacking the messenger rather than face the implications of the message. This is shameful behavior for faculty, and far worse to be implicitly sanctioned by the president of the college. Bryan Elliott

Academic freedom, political correctness and injustice

The most common theme in the comments was concern about an attack on academic freedom.

163 of 632 comments (26%) mentioned this explicitly. Five drew parallels with Galileo/the Inquisition

7. This action is absurd and shameful. It runs contrary to the principle of intellectual freedom and freedom of speech, to say nothing of common sense. Steven Pinker

8. It is unbelievable that scientific discourse is threatened in this almost theocratic way at a European Institution. These sort of censorship must stop now. Sincerely, Björn Brembs

9. That the president of a university wouldn't do everything in his power to promote academic freedom is offensive to me. I demand that Professor Michael Murphy be sanctioned immediately! Brian Neau

10. This was a ridiculous move by the HR and deeply worrying to the progress of science in general. It is no better than the persecution of Galileo for having the 'audacity' to suggest that humans occupy a planet that is not the center of everything. Have we progressed no further in our conceit since then? Benjamin Machanik

137 (22%) complained that UCC was affected by inappropriate political correctness, prudery, or religious sensitivity. Five mentioned creationism.

11. First the blasphemy law and now this! What's wrong with Ireland? Such ridiculous moralizing has to stop! Freedom of speech is the cornerstone to any free and democratic society, and protecting people's sensibilities over their right to free expression will only lead to oppression. Jeff Klinger

12. I have not heard both sides of this case, but I suspect creationist bigotry. I would be interested to know if this is true. D. Cameron

13. To use political correctness as a means to stifle free discussion is unpardonable. William Wright

Virtually all respondents appeared to accept Evans' account (subsequently disputed) that he had been exonerated by the external investigation but then sanctioned by the President of UCC. The injustice of this was mentioned in 46 (7%) comments.

14. I think what is really shocking here is that Dr. Evans has already been put on trial and exonerated. This is therefore not only a slap on academic freedom, but also on UCC's own rule of law and system of justice.

Distortions

Virtually all the signatories accepted the account of the context given by Evans, but some went beyond it. The most common of these was the belief that Dr Evans and his colleague were researchers in a zoology department, with a specific research interest in evolutionary biology, although some seemed to think he was a student, or that the colleague who complained was male:

15. The proper course of action for the offended party is to publish their own study that refutes the claims of the article, or outlines the reasons for their offense. Dr S. Killings

16. As a fellow zoologist, I am appalled at your treatment. Emma Sherratt

17. Good heavens! Sanctioning a student because he showed a peer-reviewed scientific paper on biology? Vernon Balbert

18. It is astonishing to think someone would be offended by a biologist, of all people, using animal sexuality as an example in a debate. Russell Tanton

19. I support Dr Evans in his campaign to have his work openly discussed and published without the restriction on his professional activities. Patrick

The account by Evans, which maintains that the bat paper was produced in the context of an 'ongoing academic discussion about the relevance of evolutionary theory to human psychology' was interpreted by several signatories to mean that he and his colleague were in the throes of a public discussion of this issue when he produced the bat paper as evidence for his viewpoint. The subsequently published documents reveal this to have been erroneous; the complainant maintains that he appeared in her office with the paper out of the blue.

Accusations against the President of UCC

Professor Michael Murphy, the President of UCC, came in for considerable personal criticism. 116 (20%) of comments criticised him, some in fairly foul-mouthed and abusive terms (though it has to be said that these were in a minority).

20. Professor Murphy, you are truly a moron. Individuals like you are going to be the end of human progress. Please give some bats fellatio. Thank you. Anonymous

21. This is truly bizarre. I don't know who is the more to be pitied - the colleague who took offence, or Professor Michael Murphy. Geoff Coupe

22. Mr. President, Have some cahones. Catherine Partyka

23. What is truly "upsetting," beyond the petty, prudish, and unprofessional behavior of the person filing the so-called "harassment" complaint in question, is the apparently craven, bureaucratic reaction by the UCC HR office and President Murphy, who are now the deserving targets of scorn and ridicule worldwide. James S. Hart

24. I am shocked by this example of PC gone nuts. Prof. Murphy needs to take a valium and rationally rethink his position. If one cannot discuss the sex habits of bats, or any other type of sex, in a scientific manner, we might as well all go back to caves. Alison Streight

25. The president should be ashamed of himself for penalizing someone for distributing scientific articles. Chuck Goecke

26. It is clear to me that the lead here is being taken from a position of weakness, rather than clear headed thinking, and calls not Evan's actions into question, but those of Prof Murphy. An enquiry into his suitability for this post is now necessary Annabel Huxley

27. Professor Murphy should be removed from his position if he doesn't understand science and is opposed to academic freedom. Jonathan Cornick

28. This is what happens when religious fanatics like Murphy are in charge of academic institutions. Science and scientists will always be the pawns of the religious fruitcakes. Just for being a fool, Murphy should be tossed out on his ear for sanctioning a scientist who has done nothing wrong in anyone's book. Good lord, almighty, what has become of us? Michael J. Raymond

Accusations against the complainant

86 (14%) of the comments attacked the complainant, who was accused of being prudish, unscientific, and malicious.

29. This smacks not merely of restraint of academic freedom but intimidation and bullying. Equally disturbing, what serious academic could possibly be offended by an article about the sexual behaiviour of animals? Unless UCC are replacing staff with nine-year-old schoolgirls in an effort to save costs... Nick Mickshik

30. If university professors and administrators are afraid of scientific evidence, they should not have the privilege of teaching young minds, nor should they be permitted to sanction the discussion of other scientists. Susan Pinker

31. If Dr Evans' colleague is so sensitive — due, perhaps to some dreadful trauma he suffered in the past — then he needs to consult with a mental health professional of some kind. Nelson Cunnington

32. If you want to censure someone, try doing so to the overly sensitive person who complained. Radar Pangaean

33. The only one who should have gotten any kind of scolding should have been the immature prude who found such material "offensive" in the first place. If anyone needs counseling, it's that individual. Anonymous

As noted above, a major concern expressed by signatories was that religious or political sensitivities might have stifled academic debate. An additional inference made by some signatories was that the complainant had made a malicious sexual harrassment charge because she could not win the academic argument about evolutionary biology.

34. Academic work is being stifled all because some toffee nosed prudish twat, who it seems is working in a similar area to Dr. Evans, couldn't handle the idea of fruit bats blowing each other. IT HAPPENS. DEAL WITH IT. But don't go complaining of fucking sexual harrasment when someone proves you wrong. Tadhg Murphy

35. This coddling of the so-called faint-hearted must stop, especially as the only reason for crying foul is the lack of a well reasoned argument. No one has the right to not be offended. Virginia Ford

36. Since the external investigators concluded that Dr. Evans was not guilty of harassment, we are left to speculate on the reason someone might find the peer-reviewed article upsetting enough to lodge a formal complaint. What a convenient method by which to stifle debate or suppress research one does not agree with. Robert Waite

Some implicitly criticsed the complainant for doing other women a disservice by using harrassment legislation to make a trivial complaint.

37. Let's not belittle the seriousness of harassment by applying it to any and all conversations that occur. Kelly Amodeo

38. This does not constitute harrassment, and framing it as such only dilutes the term "harrassment" and downgrades the suffering of real victims. Rose Strickland-Constable

Sex differences in issues raised

In general, the content of comments was broadly similar for male and female respondents. I wondered whether women would be less likely to criticise the (female) complainant than men, but this was not the case: 19.4% of comments criticising the complainant came from woman (comparable to the overall proportion of women signatories). Interestingly, women were rather less likely to criticise the UCC President: 8% of the criticisms against him came from women.

Overview

A large number of academics were persuaded to sign a petition in support of Dylan Evans, purely on the basis of information on a blog written by Evans. Although a few of those signing the petition expressed themselves in terms of mild concern, and hedged their comments with qualifiers about needing more evidence, the majority appeared certain of their ground, and expressed outrage at an abuse of academic freedom. Furthermore, some 14-20% heaped personal abuse on the President of UCC and/or the complainant, for their perceived bigotry, prudery and unscientific attitude. Reading the comments at times reminds one of cases where a pedophile is 'outed' by the media and attacked by a mob.

Why did so many rush to support the cause? The answer is surely that it was presented as a cause close to their hearts. The comments made it clear that the motivations for most signatories were ones that all good academics would support: freedom of speech, scientific knowledge unimpeded by political correctness, and fairness. However, for some signatories there appeared to be another argument they were all too willing to endorse, that someone who complains of sexual harrassment must be a hysterical prude. The failure to consider that there might be another side to this story is worrying, and the readiness with which some signatories embellished the story to fit such an account – accusing UCC staff of religious bigotry and worse – won't in the long run help the battle against political correctness.

As the blogosphere becomes an ever more important source of information, there will be more and more information out there that has not been through any kind of peer review. The role of academics as impartial evaluators of evidence will become ever more important. This sobering episode has taught us that we need to be especially vigilant when the information confirms our pre-existing prejudices.