The key question that arises from much of the data that’s generated about pupils’ attainment is what test best assesses pupils’ ability in reading. We know that the current government like the PISA (Programme for International Student Assessment) a great deal and are suspicious of Key Stage 2 tests, which they don’t like because they show our teachers are doing a great job in improving children’s reading — pass rates have doubled in fifteen years.
The PISA reading test is predominantly multiple choice and highly problematic in my view. If we examine a sample question, we’ll begin to see why:
“Read the text and answer the questions which follow.
IN POOR TASTE
from Arnold Jago
Did you know that in 1996 we spent almost the same amount on chocolate as our Government spent on overseas aid to help the poor? Could there be something wrong with our priorities? What are you going to do about it?
Yes, you.
Arnold Jago,
Mildura
Source: The Age Tuesday 1 April 1997
Arnold Jago’s aim in the letter is to provoke:
• Guilt
• Fear
• Amusement
I suppose the obvious answer is guilt, but you could see how some children with reasonably reading abilities might answer “amusement” or “fear” because there are elements of the letter which are faintly ridiculous and/or menacing. Other reading questions ask pupils to fill in the details for a Warranty Card, read an old fashioned fairytale, an informative booklet about bees and a library timetable. Very dry stuff indeed. It struck me that many students might not know what a “Warranty” card is, particularly if they are from more socially disadvantaged backgrounds. The questions that are asked are either multiple choice question or require very “closed” answers; there’s little room for broader readings of the texts, certainly nothing that allows for a personal response. It’s a very mechanical test that means computers can mark it.
Let’s compare with the Key Stage 2 reading SATS. In my view, although flawed, they are much better than the reductive PISA reading test. For the reading test, pupils read a selection of different texts and then answer some closed, short questions — quite similar to the type that PISA ask — but then are required to answer questions like this, which appeared in the 2009 Key Stage 2 reading SATs:
“15a. The idea of a child living in a tree is quite humorous.
Explain what else is funny about Norman’s situation.”
In my view, there’s no question that the Key Stage 2 SATS are a much better test of reading; they check to see if pupils have understood the underlying meanings behind texts, have understood high level concepts like irony and are able to articulate personal responses. The PISA does none of this. That’s why we need to trust our home-grown test results much more than the flawed ones devised by quantitative researchers who are looking for correct, mechanical responses rather than personal, thoughtful ones. Furthermore, the SATs’ levelling system is quite robust; pupils attaining a Level 4 overall in reading will be competent readers and will have done well on this test. Attainment below Level 3 indicates pupils are having great difficulty reading.
For all their faults, the SATs tests are much more enlightened than the PISA. Let’s hope that Gove doesn’t bring in tests like the PISA and allows room for personal responses. Otherwise, he won’t be testing reading ability in any meaningful fashion.