Sunday, 29 January 2012

2011 Orwellian Prize for Journalistic Misrepresentation

© www.CartoonStock.com
So the time has come round for the announcement of the 2011 Orwellian Prize. The prize is given for an article in an English-language national newspaper that achieves an unusually high level of inaccuracy. Only articles that describe a piece of published scientific research are eligible. Points are given for every statement in the article that does not match the original source, as follows:
  •     Factual error in the headline: 3 points
  •     Factual error in a subtitle: 2 points
  •     Factual error in the body of the article: 1 point
Last year, I got two nominations, but, as described here, neither adequately met the criteria. This year, I’ve had just one nomination, from Neurobonkers, but it’s set a standard for inaccuracy that will be hard to beat. The article that I first selected in 2010 to illustrate the scoring system scored 16 points. This one achieves a startling 23 points. The source article by Kucewicz et al (2011) can be found here. Here is a screenshot of the account in the Daily Mail, with errors marked in red (3 points), orange (2 points) and blue (1 point).
There is a detailed analysis of errors in this blogpost by Neurobonkers, which I urge you to look at. Suffice it to say,  the academic paper is not about cannabis, smoking or schizophrenia. Rather it is about an artificial compound that is not present in cannabis, which was injected into rats, and which led to changes in their brain waves.

There were some complaints to the Press Complaints Commission, and presumably in response to this, the article was modified. The headline, which originally read Just ONE cannabis joint 'can bring on schizophrenia' as well as damaging memory was altered to Just ONE cannabis joint 'can cause psychiatric episodes similar to schizophrenia' as well as damaging memory. Perhaps even the Daily Mail found the notion of a schizophrenic rat implausible. But the rest of the article remains, as a scare story about cannabis. And here is what makes this article such a prime candidate for the Orwellian Award: this is not about a hyped press release by a university, or misunderstanding of complex science. It's not even about sensationalising a scientific finding to draw readers in. No, this is about using a scientific paper as a prop in the Daily Mail's anti-cannabis campaign. A ploy that the newspaper has previously used in another ideological battle, on climate change. When reporting research, no respect is given to the truth: scientists are simply used to bolster a preconceived opinion, and if they don't do that, their findings are distorted.

Twelve of this article's 23 points came from the headline. Journalists don’t write the headlines. They therefore dislike my scoring system because it penalises errors in headlines more than errors in the body of the text. My view is that it’s the headlines that count for most: far more people will read the headline than the text, and for many readers it's the only part of the article they will process. It’s important that it's accurate. Although few would defend frank lying, some editors seem to think it doesn’t matter if a headline is hyperbolic, provided it sells the paper or gets someone to read further. This very issue has been a topic of debate in the recent Leveson Inquiry into culture, practice and ethics in the UK media. I feel strongly that it's a cop-out to just wash one's hands of it and blame anonymous sub-editors for misleading headlines. I shall therefore continue to award points in proportion to the prominence of the material. But I appreciate it’s not then fair to make the award to the journalist. Indeed, given the Mail's agenda on cannabis, the journalist in this case may well have been under duress to write a scare story.   I will accordingly be making the award to Paul Dacre, Editor of the Daily Mail. I will be happy also to send the token of appreciation for the nomination to Neurobonkers if he/she is willing to email me to tell me where to send it.

I'm pleased not to have had more nominations this year: it suggests that, despite all the grumblings about science journalism, the field is in rude health. I've certainly read a lot of good science reportage in our national newspapers, and where articles have made me angry, it's often because of hype by a press office or scientist, rather than distortion by the press. There are, however, still a few topics, among them drugs policy, where the political stakes are high and scientific reporting is cynically exploited to support an otherwise weak argument.

Coming up with an award certificate and item turns out to be an excellent way of avoiding doing serious work.....




Reference
Kucewicz, M., Tricklebank, M., Bogacz, R., & Jones, M. (2011). Dysfunctional Prefrontal Cortical Network Activity and Interactions following Cannabinoid Receptor Activation Journal of Neuroscience, 31 (43), 15560-15568 DOI: 10.1523/JNEUROSCI.2970-11.2011
at

15 comments:

  1. Grant Jacobs29 January 2012 at 09:27

    I would like to see Paul Dacre’s acceptance speech.

    ReplyDelete
  2. Neuroskeptic29 January 2012 at 10:05

    I agree about headlines. People say that headlines are there to attract attention to the main text. But many people don't read the main text, just the headline. It's impossible to know what the percentages are but I'd be surprised if more than say 10% of people who see the headline even read the first paragraph and fewer will read to the end (I'm basing that on the fact that I read less than 10% of headlines I see)...

    Plus even if you do read the main text, you've been primed to interpret it in terms of the headline.

    Would be very interesting to do an experiment actually: take a fairly balanced article about some complex issue, give it two very different headlines, randomly assign people to read one of the two versions and then see what the make of it.

    I don't know but I would be very surprised if people weren't swayed by the headline, even if they did read to the end.

    ReplyDelete
  3. deevybee29 January 2012 at 10:26

    I like the idea of Neuroskeptic's experiment. In fact, when writing the post I had a vague memory of relevant research on this kind of thing, but I tried a quick Google and it didn't turn up anything. But there's is relevant research showing that if you give a complicated passage a title, that influences how people comprehend it in terms of the inferences they make, e.g. Bransford, J. D., & Johnson, M. K. (1972). Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of Verbal Learning and Verbal Behavior, 11(6), 717-726. doi: 10.1016/s0022-5371(72)80006-9

    ReplyDelete
    Replies
    1. Neuroskeptic30 January 2012 at 13:37

      Ah, right. That's the kind of thing I mean, I suspected someone must have done it before.

      Would be good to do it with real world stimuli though. Take, say, an article from a "balanced" newspaper, and then replace the title with the headlines used by two sensationalist newspapers of opposing political views... that kind of thing... i.e. make it as naturalistic as possible so critics can't just say "Oh, that's just the lab, not real journalism."

      Delete
    2. Neurobonkers11 February 2012 at 03:42

      I've just been catching up on the Leveson Inquiry and it turns out some research has been done! Blogspot's spam filter stops me hyperlinking but if you Google the video of the "Full Fact" team's evidence they describe the "Paragraph 19" effect. Ben Goldacre did a post on the research titled "The caveat in paragraph number 19". The research found that on average readers read only half the story, never reaching the (PCC satisfying) caveat which renders the headline void.

      Delete
  4. Neurobonkers29 January 2012 at 19:21

    Incidentally, Paul Dacre is also chairman of the PCC. Another weird coincidence is that the PCC have refused to address the complaints about this article.

    ReplyDelete
    Replies
    1. Neurobonkers29 January 2012 at 22:52

      Make that the PCC Editors' Code of Practice Committee. Paul Dacre is the man writing the rulebook. This certainly explains the hopelessness of the organisation (who have been investigating this article since October).

      Delete
  5. Anonymous29 January 2012 at 19:24

    this is brilliant! its great to see someone going after the media who just try to sell papers while in the mean time are poisoning the easily manipulated minds of the mass populace. bravo

    ReplyDelete
  6. RAJ29 January 2012 at 20:51

    Catchy headlines sell and improve circulation whether in science reporting or in crime reporting. My favorite was a headline in the Rupert Murdoch owned New York Post. NY Police discovered a decapitated man in a strip joint.

    The Post headline 'Headless Man Found in Topless Bar'.

    ReplyDelete
  7. Mike Fogg30 January 2012 at 12:59

    Sub-editors do occasionally come up with genuinely brilliant headlines, though. Two that spring readily to mind are:

    1) Super Cally Go Ballistic, Celtic Are Atrocious (when Inverness Caledonian Thistle knocked Celtic out of the Scottish Cup some years ago); and

    2) Kim Wild! (when the now late Beloved Leader was doing some pretty impressive sabre rattling about nuclear testing).

    Also at the same time as the headline in 2) the 'Sun' came up with "How Do You Solve A Problem Like Korea?" - bear in mind that this was during the Sound of Music reality competition 'How Do You Solve A Problem Like Maria'.

    ReplyDelete
  8. Anonymous30 January 2012 at 13:27

    Just ONE Daily Mail article can bring on symptoms of rage and despair...

    ReplyDelete
  9. Anonymous1 February 2012 at 17:49

    How the **** can these **** that write all this **** be the ones who pretty much moderate whether they can get away with writing such tosh and then be on the board that says if (or if not as the case may be) the complaints are addressed. Which it seems they have not.
    Despite this article being awarded a certificate for 'The most Infactual', based on an error-point system used on other articles too, and had an abnormal amount of complaints put forward about it, the PCC decides to palm off the matter?

    Another prime example of how politics has gone to ****. I call, Government for the people and not for the corporation. Despicable state 'the system' is in now. And its affected the people, they have a voice its going to get used. Bent systems like this is whats making the people speak out in anger against. Revolt?

    ReplyDelete
  10. The Editor8 February 2012 at 07:08

    Re great headlines, I like the British wartime effort:
    "British push bottles up Germans"

    Thank you Professor Bishop for highlighting this tripe.

    ReplyDelete
  11. Anonymous28 May 2013 at 09:53

    Politics is indeed the problem. If even science can be manipulated this way, imagine how much more the laws are being manipulated

    ReplyDelete
  12. Anna13 August 2013 at 08:01

    This is in the event that the Orwellian prize is bestowed on a yearly basis ... I just came across a great example of journalistic misrepresentation of scientific findings:

    http://www.bbc.co.uk/news/science-environment-23672150

    ReplyDelete

New comments are not allowed.