Away From the Numbers

In which Alice takes a tea break and feels there might be more to life than measuring it.

Alice decided that. after her frustrating conversation with the rabbit she needed some fresh air to work out why his description of setting targets depressed her so much. She found a door that led out into the woods behind the school. It was a warm afternoon with the sun streaming through the trees and playing interesting and changing patterns on the leaves on the ground.

Being English, and a teacher, she immediately thought of tea and set off to find some. She recognised the clearing before she entered it and the familiar voices brought a smile to her face. The Mad Hatter and the March Hare were sat round the same table she had seen all those years ago. It was just as piled with cups and saucers and teapots as before.

“Oh, Hatter and Hare, how happy to see you holding on to habits!”

“Don’t get all alliterative on us” called the Hatter, “there are no extra marks for that you know”.

“I am just pleased to see you” said Alice, “It will be so nice to be with some comfortingly reassuring madness after all the tedious sanity that has just been explained to me.” She slumped into a chair and poured herself a cup of tea.

“4” shouted the Mad Hatter.

“6” retorted the March Hare

“5” came a small dreamy voice from inside a teapot, which must have been the Dormouse.

“Your problem”, shouted the Hatter, pointing a bony finger in the direction of his long eared friend “is that you are too generous with your grading. If you grade too high then everyone will think they are good at everything!”

“I do not have ‘a problem’ “, said the Hare, “At least not one that I want to talk to you about. Research shows…”. This was met with a deep groan from inside the teapot. “Ahem! Research shows that low grading is demotivating for children and puts them off the subject.”

“It shows that I am a more rigorous teacher”, said the Hatter with pride “and my subject is a more rigorous subject. I am interested in excellence and so I need to show them where they can improve.”

“Stop, stop, stop” cried Alice, “What on earth are you talking about? What have I done that you are grading with these numbers?”

“Pouring Tea, of course” They chorused.

“But why on earth should you grade that? I got the tea in cup didn’t I? The quality of the tea is in its taste not in a numerical grading of the pouring of it.”

“Ha,” harrumphed the Hatter, “You might as well say the value of a painting is what it looks like!”

“Or the value of a piece of music is what it sounds like” continued the Hare.

“Or the value of a teapot is how comfortable it is for sleeping in” came the small voice from inside the pot.

“It probably is for you” said the Hare to the pot.

The Hatter explained. “The point is that the value of anything can only be understood if it is properly assessed”

“With criteria and rubrics and moderation” said Hare

“Without that we have no way of knowing how good it is you see. But once we have done this we have numbers” The Hatter sounded almost reverential when he said the word numbers. “They are called numbers, you know, because without them we would be numb. We would have no feelings or understanding of the world around us. With numbers we can do anything.”

“We can do averages, trends, standard deviations, regression and correlation”

“We can measure value added”

“And value subtracted”

“We can compare with standardized data, against background populations, students with students, teachers with teachers, schools with schools and countries with countries”

“That last one is called PISA. It is named after a tall building that is falling over because it is built on poor foundations.”

Alice looked from the teapot in her left hand to the teacup in her right hand with a completely mystified look on her face.

“You see”, said the Hare which Alice most certainly did not, “that you would have no idea how well you had poured the tea unless we had given it a proper assessment. That is called summative assessment because it sums up what you have done.”

“Now you know that you can do better next time. That is called assessment that is formative, which is like summative assessment except that we say it in a nicer way and smile when we give it. We pretend we haven’t given you a number, when we really have, and try to tease you into working it out for yourself.”

“But where do the numbers come from? Asked Alice

“Why rubrics, of course” Hare and Hatter chorused.

“The rubrics list the criteria by which things are assessed. We start with a clear description of a perfect piece of work and then gradually make the description worse until we have a dreadful piece of work” explained the Hatter.

“How do you mean”

“Well, let’s say that a perfect cup of tea might be described as ‘Warm, but not too hot, with a balanced flavour of tea and milk. Filling the cup but not too much so that you would spill it down your front, carefully placed at the centre of the saucer’. Then we would take that and say what might describe a cup that was nearly as good. ‘The tea shows some understanding of temperature without being exactly right. There are flavours of tea and milk but one dominates the other excessively. The tea shows good filling and positioning qualities.’ Now we might then degrade the criteria even further to give: ‘The tea has either some understanding of temperature or some understanding of flavour. Positioning and filling are rather approximate’ and finally we would say ‘A cup of tea with no real flavour of tea that is overflowing or largely empty’ and there we have our rubric.”

“But that last one isn’t really tea at all” said Alice

“Exactly, so it would get the lowest grade.” Said the Hare, smugly.

“I think I get it. So it really is about the tea and not about the numbers”

Hare and Hatter looked shocked. “You don’t get it at all” said the Hatter, “Think, dear girl, what exactly is a rubric?”

“A list of criteria describing different levels of quality of a cup of tea.”

“Go on.”

“Er…” Alice continued uncertainly, “and each level is given a number?”

“Exactly so. Now what do you think comes first? The criteria or the numbers?”

“Well, I suppose, if we want to grade out of 10, we start with the numbers 1 to 10.”

“Indeed”

“And then we draw up the criteria to match the number of numbers we have. Finally we assign the criteria to the work and read off the number.”

“And there you have it,” concluded the Mad hatter with his smug smile firmly in place. “the numbers come first and the numbers come last. The criteria are just a set of sentences created to fill the slots next to the numbers. Everything is Number, as I suspect Pythagoras and his bunch of mad cultists once must have said.”

“Oh dear. That is depressing” sighed Alice.

“On the contrary! “, retorted the March Hare, “It is the numbers that give the statements meaning. We can have any set of statements and, if we allocate numbers to them, they take on importance. Without the numbers they are just a set of random sentences.”

“But surely the sentences have to mean something. In any assessment or measurement or survey the value of the data depends on the criteria you use of questions you ask.”

“Wrong again!” said Hare, “Once you have the numbers then it doesn’t matter where they came from. That is the magic of number.”

“There is a namesake of mine, well almost a namesake, certainly a distant relation I am almost sure, who attributes numbers to all sorts of things with great precision.” Said Hatter. “He gives everything two complete places of decimals! How about that for numerical authority? For example ‘Feedback‘ gets 0.73 which is very high, whereas ‘Homework’ gets 0.29 which is very low. So it is clear that Feedback has much more effect on learning than Homework does.”

Alice was very impressed. “But how does he get the numbers?” she asked.

“Well, that is the beauty of it.” Continued the Hatter, “nobody has the time to find out! It is called a meta-analysis because it takes a bunch of other numbers that other people have come up with in their own studies and creates new numbers from them. So if they are wrong it’s not even his fault!”

“And nobody bothers to question if all the different numbers he has used to justify the 0.79 for feedback are from equivalent studies. They just accept that feedback must be good because it has a number associated with it.” Said Hare.

“And you can’t argue with numbers!” concluded Hatter.

“But is it really all numbers in this school?” Alice asked

“Well no.” said Hare, “The point is that there are two types of things going on in any school. The important things and the unimportant things. The important things are so important that we attach numbers to them. The unimportant ones are those that do not have numbers. Important things include exam and test scores.”

“And coursework and continuous assessment” added Hatter,

“And effort, and attendance”

“Things that do not have numbers and are therefore unimportant are usually called Extra Curricular Activities”

“Or sometimes Character Education or CAS”

“Although,” continued Hatter, “I have heard that there are some schools that attach numbers to character education and even behaviour as well. If they do that they become important too.”

Alice decided she had had enough tea and she left them as they carefully stuffed the dormouse back into the teapot she had emerged from.

Chapter 6