Saturday 5 October 2013

Good and bad news on the phonics screen



Teaching children to read is a remarkably fraught topic. Last year the UK Government introduced a screening check to assess children’s ability to use phonics – i.e., to decode letters into sounds. Judging from the reaction in some quarters they might as well have announced they were going to teach 6-year-olds calculus. The test, we were told, would confuse and upset children and not tell teachers anything they did not already know. Some people implied that there was an agenda to teach children to read solely using meaningless materials. This, of course, is not the case. Nonwords are used in assessment precisely because you need to find out if the child has the skills to attack an unfamiliar word by working out the sounds. Phonics has been ignored or rejected for many years by those who assumed that if you taught phonics the child would be doomed to an educational approach that involved boring drills in meaningless materials. This is not the case: for instance, Kevin Wheldall argues that children need to combine teaching of phonics with training in vocabulary and comprehension, and storybook reading with real texts should be a key component of reading instruction.
There is evidence for the effectiveness of phonics training from controlled trials,  and I therefore regard it as a positive move that the government has endorsed the  use of phonics in schools. However, they continue to meet resistance from many teachers, for a whole range of reasons. Some just don’t like phonics. Some don’t like testing children, especially when the outcome is a pass/fail classification. Many fear that the government will use results of a screening test to create league tables of schools, or to identify bad teachers. Others question the whole point of screening: This recent piece from the BBC website quotes Christine Blower, the head of the National Union of Teachers, as saying: "Children develop at different levels, the slow reader at five can easily be the good reader by the age of 11.” To anyone familiar with the literature on predictors of children’s reading, this shows startling levels of complacency and ignorance. We have known for years that you can predict with good accuracy which children are likely to be poor readers at 11 years from their reading ability at 6 (Butler et al, 1985).
When the results from last year's phonics screen came out I blogged about them, because they looked disturbingly dodgy, with a spike in the frequency distribution at the pass mark of 32. On Twitter, @SusanGodsland has pointed me to a report on the 2012 data where this spike was discussed. This noted that the spike in the distribution was not seen in a pilot study where the pass mark had not been known in advance. The spike was played down in this report, and attributed to “teachers accounting for potential misclassification in the check results, and using their teacher judgment to determine if children are indeed working at the expected standard.” It was further argued that the impact of the spike was small, and would lead to only around 4% misclassification.
However, a more detailed research report on the results was rather less mealy-mouthed about the spike and noted “the national distribution of scores suggests that pupils on the borderline may have been marked up to meet the expected standard.” The authors of that report did the best they could with the data and carried out two analyses to try to correct for the spike. In the first, they deleted points in the distribution where the linear pattern of increase in scores was disrupted, and instead interpolated the line. They concluded that this gave 54% rather than 58% of children passing the screen. The second approach, which they described as more statistically robust, was to take all the factors that they had measured that predicted scores on the phonics screen, ignoring cases with scores close to the spike, and then use these to predict the percentage passing the screen in the whole population. When this method was used, only 46% of children were estimated to have passed the screen when the spike was corrected for.
Well, this year’s results have just been published. The good news is that there is an impressive increase in percentage of children passing from 2012 to 2013, up from 58% to 69%. This suggests that the emphasis on phonics is encouraging teachers to teach children about how letters and sounds go together.
But any positive reaction to this news is tinged with a sense of disappointment that once again we have a most peculiar distribution with a spike at the pass mark. 
 
Proportions of children with different scores on phonics screen in 2012 and 2013. Dotted lines show interpolated values.

I applied the same correction as had been used for the 2012 data, i.e. interpolating the curve over the dodgy area. This suggested that the proportion of cases passing the screen was overestimated by about 6% for both 2012 and 2013. (The precise figure will depend on the exact way the interpolation is done). 
Of course I recognise that any pass mark is arbitrary, and children’s performance may fluctuate and not always represent their true ability. The children who scored just below the pass mark may indeed not warrant extra help with reading, and one can see how a teacher may be tempted to nudge a score upward if that is their judgement. Nevertheless, teachers who do this are making it difficult to rely on the screen data and to detect whether there are any improvements year on year. And it undermines their professional status if they cannot be trusted to administer a simple reading test objectively.
It has been announced that the pass mark for the phonics screen won’t be disclosed in advance in 2014, which should reduce the tendency to nudge scores up. However, if the pass mark differs from previous years, then the tests won’t be comparable, so it seems likely that teachers will be able to guess it will remain at 32. Perhaps one solution would be to ask the teacher to make a rating of whether or not the test result agrees with their judgement of the child’s ability. If they have an opportunity to give their professional opinion, they may be less tempted to tweak test results. I await with interest the results from 2014!

Reference
Butler, Susan R., Marsh, Herbert W., Sheppard, Marlene J., & Sheppard, John L (1985). Seven-year longitudinal study of the early prediction of reading achievement Journal of Educational Psychology, 77, 349-361 DOI: 10.1037//0022-0663.77.3.349

15 comments:

  1. Thanks for this Professor Bishop. There is a tendency for the likes of Christine Blower to conflate issues. It is highly doubtful that the Finnish example is an appropriate model for Britain to adopt. Finnish is considered to be a more phonetic language than English. The idolisation of Finnish education among certain educationalists masks the fact that there are stark differences in attainment especially between Finnish and Swedish-speaking children. Children in Finland may actually be being taught how to read well before they formally start school.

    There are issues concerning the spike. Is it the case that a minority of teachers are passing more children than can be justified or is it a systematic bias amongst teachers more generally? If teachers cannot be trusted to deliver the screening test more effectively should the tests be administered by Ofsted or by a panel of trained assessors appointed directly by the Department of Education? Again there are issues over the validity of Ofsted inspections and I doubt that any decision to take the administration of tests outside the hands of teachers will be popular. The problem is that the leadership of some trade unions is so intransigent that they cannot even envisage an alternative to whole language teaching and if you advocate phonics you are considered to be a reactionary.

    I would like to know what goes on in teacher training. Is there any systematic bias against phonics teaching in teacher-training colleges? Also any improvement in results should not mask the fact that Gypsy/Roma and children from Irish Traveller heritage both do very poorly.

    ReplyDelete
  2. Unfortunately I think that this reflects the usual 'teach to the test' approach that has been taken with this - teachers get hammered if their pupils get a poor pass rate on the test, to avoid this children are being specifically coached on reading 'alien words' and practicing doing tests which I personally think defeats the object of the test, which is to identify children that are struggling with phonics rather than test-taking. I think the blip perhaps reflects the pressure that teachers are under to achieve high percentages of children passing the test. As an informed (psychologist) mum of a child who achieved exactly the pass mark in the test, despite not having great phonic decoding skills I think it is a shame that the test is not being allowed to do what it was designed to do because of the constant drive to hit prescribed targets.

    ReplyDelete
    Replies
    1. speaking as a Y1 teacher children don't need to be coached to read "alien words" just taught to accurately decode words.

      Delete
  3. This comment has been removed by a blog administrator.

    ReplyDelete
  4. This comment has been removed by a blog administrator.

    ReplyDelete
  5. I do agree that speech must be active before reading can have value. This is why phonics is seen as the basis of good reading skills .Where infants are taught to recognise objects and words which go with them they are disadvantaged because they are not combining phonics in the process. http://helpchildrenread.net/

    ReplyDelete
  6. Being able to make sound out of letters allows a child to be able to join them up into words. There is nothing as pleasing to a parent as hearing their little one sounding out basic words. It is a wonder that anyone would find it not suitable. info@easylearnreading.com

    Regards
    Daniel P. Warren

    ReplyDelete
  7. It is helpful if parents help their kids learn how to read and speak. i bought HOP program for my 1year old through helpkidsread.net. My elder one is fluent and could read almost 50 pages per half an hour, without any pronunciation error.

    ReplyDelete
  8. To all who are trying to sell reading programs through my blog, I can only say that I think it is entirely counterproductive to teach a child to read before the age of 3 years minimum, and then only if the child has evidence of a desire to learn to read and sufficient phonological awareness to be able to relate letters to sounds.

    ReplyDelete
  9. Deevybee..I agree. I am an English teacher and I'm doing phonics with my grandson (4). He gets the cards out and we are slowly progressing with lots of enthusiasm. The minute he is tired or stops I pack it all away. So far I've never had to suggest doing it, he is the who gets the set off the shelf.

    ReplyDelete
  10. Thank you for this it's very interesting and I have been following your blog as part of my studies to become an Early Years Teacher this year. It has been interesting reading about the controversy over the use of non words. In practice I have found them very valuable l in raising the phonological awareness e.g. discriminating between the sounds in "Shell" and "Chell" as children often have to listen carefully and are unsure which is correct. The games in school using non-words always make the nonsense value of the word obvious - for example the non-words going in the dustbin and the words going in the treasure chest. Non words also help increase vocabulary by getting the child to think about whether it is a real word - have they heard of it/seen one - what does it mean/look like?

    ReplyDelete
  11. The problem with the phonics screening test is the inclusion of real words such as 'crowds' - which many children who have english as a second language will pronounce with the 'ow' as in 'low' - if they dont know the word 'crowds', then either pronounciation as in ow(cow) or ow(low) should be valid. However, it is marked as wrong even if phonetically viable if it is in a real word - but if the children don't have that word in their vocab then how can they possibly know which way to pronounce it?
    Another one of the real words was 'nigh' - many children will not know that this is a real word and as they are told it must be read as a real word they will fix it to 'night'. Surely if the government are going to insist on correct pronounciation of real words then they need to make sure that they are words generally in the vocab range of 5/6yr olds including those for whom English is not a home language. My son was able to read these in his test and to tell me about them, but many in his class are EAL children and will 'fail' because these words are testing there vocabulary and not their phonics. Daft as ever - just let the teachers teach.

    ReplyDelete
    Replies
    1. Do you realise that the words in the phonics check are highly confidential until the two week window is closed and should discusses even within school with anyone administering the check, never mind online where anyone can few it! If you are a teacher then this is really bad form. If you're not then how have you heard about them?

      Delete
    2. That should have said 'shouldn't be discussed even within school with anyone not administering the check, never mind online where anyone can view it!'

      Delete
  12. Or use entirely non-words in the check so that there is no disadvantage at all to any children with regard to their existing oral vocabulary.

    ReplyDelete