Saturday, 15 October 2011

Lies, damned lies, and spin


The Department for Education (DfE) issued a press report this week entitled “England's 15-year-olds' reading is more than a year behind the best”. The conclusions were taken from analysis of data from the PISA 2009 study, an OECD survey of 15-year-olds in the principal industrialised countries.

The DfE report paints a dire picture: “GCSE pupils' reading is more than a year behind the standard of their peers in Shanghai, Korea and Finland….Fifteen-year-olds in England are also at least six months behind those in Hong Kong, Singapore, Canada, New Zealand, Japan and Australia, according to the Department for Education's (DfE) analysis of the OECD's 2009 Programme for International Student Assessment (PISA) study.” The report goes on to talk of England slipping behind other nations in reading.
Schools Minister Nick Gibb is quoted as saying: “The gulf between our 15-year-olds' reading abilities and those from other countries is stark – a gap that starts to open in the very first few years of a child's education.”
I started to smell a rat when I looked at a chart in the report, entitled “Attainment gap between England and the countries performing significantly better than England” (my emphasis). This seemed an odd kind of chart to provide if one wanted to evaluate how England is doing compared to other countries. So I turned to the report provided by the people who did the survey.
Here are some salient points taken verbatim from their summary on reading:
  • Twelve countries had mean scores for reading which were significantly higher than that of England. In 14 countries the difference in mean scores from that in England was not statistically significant. Thirty-eight countries had mean scores that were significantly lower than England.
  • The mean score for reading in England was slightly above the OECD average but this difference was not statistically significant.
  • England’s performance in 2009 does not differ greatly from that in the last PISA survey in 2006.
There is, of course, no problem with aiming high and wanting our children to be among the top achievers in the world. But that’s no excuse for the DfE's mendacious manipulation of information.

Bradshaw, J., Ager, R., Burge, B. and Wheater, R. (2010). PISA 2009: Achievement of 15-Year-Olds in England. Slough: NFER.


  1. Worse. Research shows that 50% of our children are BELOW AVERAGE!!

    1. No, 50% are below the median!!

  2. There is, Dorothy, another problem with making these kinds of comparisons. The English alphabetic code is significantly more difficult to learn than the alphabetic writing systems of other European countries. For example, Spanish has around twenty-two to twenty-four speech sounds and around thirty some odd spellings for those speech sounds. This makes Spanish (as well as Finnish, Italian, German) pretty easy to learn to read and to write. It's also why many schools don't bother teaching reading and writing at the beginning of school - because they know that in the second year everyone learns to read and write in a relatively short period of time. English, on the other hand has around forty-four sounds and, by common agreement, at least a hundred and seventy-five or so common spellings. This makes it much harder to learn - which is why trying to make comparisons between pupils' reading abilities in different countries is deeply problematical.
    Oh, and you being a statistician, if you look at the stats at the end of the full PISA reports, I think you'll find that many of the countries cheat.
    I wonder what you think about the government's initiative in promoting synthetic phonics?

  3. Ah, John, thanks for that comment! I was looking for a good reason why the English spelling should be overhauled to remove the many different ways the same letter combination is pronounced, and you've given it!

    I suggest a complete overhaul of the English language a la Kemal Ataturk (i.e. in one year) to remove all these inconsistencies! Well, that will never work, but perhaps someone will start paying attention to this problem now.

  4. John: The problem with that theory is that the best-scoring area was Shanghai, and the Chinese writing system is non-phonetic, simply a set of 1000s of symbols connected with concepts. These are then connected with phonemes due to the meaning the word holds in spoken Chinese. Many common characters have more than one, entirely different, phonetic association.

    So logically, if reading is all about how easy it is to learn the phonetics, the Chinese (& Japanese & Taiwanese) should be illiterate.

  5. Thanks for commments guys. There are numerous possible reasons for differences in attainment across languages, ranging from schooling practices through to the orthography of the language (see e.g. Ziegler JC, Goswami U (2005) Reading Acquisition, Developmental Dyslexia, and Skilled Reading Across Languages: A Psycholinguistic Grain Size Theory. 131: 3-29.
    ), not to mention specific tests and sampling used.
    But my main concern in this post is the selective presentation of the data by a government department. If they'd prefaced their report by saying "English kids are average, but we want them to be better: let's look at those countries which score higher" it'd be fine. But instead they selectively report the numbers to make it look as if England is at the bottom of the heap and getting worse. This has to be deliberate. I wonder exactly what their motivation is.