Understanding by Intelligent Design. Are we not Gods?

This morning, while talking to a colleague, I thought of an interesting analogy to some of my more persistent beliefs about education. We were looking at a book which quoted John Coleman’s “Six Components of Great Culture” from 2013 (Harvard Business Review). Coleman’s first component is that “Your culture starts with your values”.

Well, I didn’t get to the other five components because this one had me wondering if he has it the right way round. Don’t our values come out of our culture? Encapsulated in this dichotomy is the debate that I return to over and over again. Should the direction of growth be determined by the end point or the start point? Are development, learning, movement and progress natural phenomena that build out of where we are now and what our initial conditions are, or are they planned to reach a pre-determined outcome? In short, and here is the analogy, is change evolutionary or determined by intelligent design? Do we grow naturally and incrementally from where we are or do we constrain that growth towards a predefined goal? Is Darwin or Creation the better analogy for our lives?

“Some people work by deciding what they want to get to and heading for that. I don’t work like that. I look at what I have now and see which parts would be interesting if I developed them.” Brian Eno in his Peel lecture if 2016.

My tendency is to believe that the evolutionary approach is certainly more creative, probably more effective and possibly more honest. The intelligent design model stifles original thought, causes friction through failure and deceives us into thinking we know things we don’t. Historically, evolution seems the accepted path whether it be biologically, psychologically or sociologically. Why do we become creationists when we replace the past by the future? Here are some examples.

  • Target setting for teachers

We know that too often people do not agree to goals that they weren’t going to achieve anyway. But when the goals are challenging they are usually about filling a gap, learning a skill that they do not have or improving areas of weakness. The is the end-determined model that has a picture of “the good teacher” that we should all aspire to. However the research suggests that this is not the most productive way forward. Rather than looking at the design of the ideal end point, we should look at the strengths of the teacher now. What makes them good already? Develop those skills even further. Not all teachers will be the same, but students will encounter a range of very well developed and individual expertise.

  • Exam based courses

This is the classic model for education. Define where we need to get to, and even when we need to get there. All students studying this subject need to know how to do these things at this point in time. Of course, if we were really successful they would all score 100%, but we are not and the range of marks measures their and our failure to achieve the ideal. How many times have we heard that there is something really interesting that we do not have time to look at because it is not on the syllabus? The students were really interested in this but we didn’t have the time to pursue it. It might even be more justifiable if we could agree that the content was vital for all, but we know it isn’t. The students could just as valuably study different periods in history, different topics in science, different books in literature, so why do we set the end point first, rather than let the students have a say in where they want to expand their knowledge from what they already know.

  • Learning outcomes

This is the same as the exams really, but broadened to more trendy terminology. We are still saying where they need to get to even though we might be stressing skills and concepts more than content. Once we say that the outcome of a task will be a presentation to the class we stifle the creativity of those who could have responded with a song, a play, a film or even an essay. Of course you can say that the presentation could be one of those but it so seldom is because the teacher has clamped it down from the start by defining the expectation. Why not just ask how they might best respond to this task, and leave it to them?

  • Criteria based assessment

This is a big one. The prevailing dogma is that you should never set a task without a clear description of the criteria by which it will be judged a success. Did anyone tell Picasso that before he painted “Les Desmoiselles D’Avignon’? or Stravinsky before he wrote “The Rite of Spring”, or Zuckerberg before Facebook, or Wales before Wikipedia? I once had a student who wrote her Theory of Knowledge essay as a poem. It was beautiful and brilliant and showed a deep understanding of the issues, but it was nowhere near the criteria and would not have scored at all. Criteria can be fine on occasion, but be aware they stifle creativity and use them with caution

  • Maths investigations

This is a bit subject specific, but I think there are equivalences in other areas. A genuinely open ended Maths investigation will begin with a stimulus, a situation or a problem. Students will start from the same point but have complete flexibility in what direction they decide to explore, which parameters they will change or relax as they pursue a generalisation that casts light on the situation. They can then make conjectures, prove and refute them. It becomes a genuinely creative process that mirrors real mathematical research. It is the beginning of how Maths is actually discovered. Imre Lakatos’ wonderful book on mathematical discovery “Proofs and Refutation” contains the great observation “Columbus did not find a route to the East Indies, but he did discover something quite interesting”. For the casually mathematically minded who may not be up to Lakatos, my slightly dumbed down version can be found here, with an introduction here.

  • Control and accountability

I remember a Grant Wiggins talk many years ago at a conference somewhere. He was a great educational thinker with some excellent ideas and I do not want to put him down. However in this particular talk, Education By Design all seemed to me to be about control and accountability. The fundamental use he described was about setting measurable targets so that we have the indisputable evidence that a teacher is not performing. “Nothing personal,” he would say “but this is where we agreed you should be getting to, and the evidence shows you are not.” To me there was something a bit fundamentalist about this which made me feel rather uneasy. Now I realise that the EDB advocates, whatever they may think intellectually about the merits of evolution, are emotionally Intelligent Design Creationists.

‘Cheshire Puss,’ she began, rather timidly, as she did not at all know whether it would like the name: however, it only grinned a little wider. ‘Come, it’s pleased so far,’ thought Alice, and she went on. ‘Would you tell me, please, which way I ought to go from here?’

‘That depends a good deal on where you want to get to,’ said the Cat.

‘I don’t much care where–‘ said Alice.

‘Then it doesn’t matter which way you go,’ said the Cat.

‘–so long as I get SOMEWHERE,’ Alice added as an explanation.

‘Oh, you’re sure to do that,’ said the Cat, ‘if you only walk long enough.’

Lewis Carroll – Alice in Wonderland

Evidence for the Prosecution

I was at a discussion or meeting a couple of weeks ago about one of my favourite topics – Assessment. The format was that representatives from 7 schools described where they were going with assessment and what problems they were grappling with. We looked at assessment policies and practices from the theoretical to the procedural. We looked at software systems and how they were used, how you put in data and got it out in a variety of different forms. The usual words were being used “Summative”, “Formative”, “Evidence”, “Reflection” and of course “Data”, when I began to notice that I understood different things by some of these words compared to the others. I explained that at the meeting in a rather haphazard way, but thinking about it since has crystallised the problem a bit more for me.

chart

Some Spurious data from the wonderful tylervigen.com

 

My two main areas of concern are the use of the words “Data” and “Evidence”. I worry enormously about the way data is used whether it is big data, personalised data, averaged data, data-over-time, data-as-snapshot or data as the start of a conversation. The other schools in this conversation were all very keen to make it easy to put in lots of data. Systems are judged by the lack of impediment they create to a teacher putting in data. They were then very keen to use the data as much as possible. There were tables and graphs, coloured cells popping up on spreadsheets to indicate anomalous data, letters sent, meetings arranged, reports written and targets set on the basis of all this data.

Reading the news headlines of recent years should make us terribly wary of data. There is a whole profession of people whose expertise lies in collecting and drawing conclusions from data. They study in minute detail the best ways to collect accurate normed and unbiased data. They are not just a bunch of teachers whose real profession is teaching and who use a bit of data on the side to help them. No, these are the true data junkies. Unfortunately the evidence (that word again) indicates how unreliable this process is. They have got most major global predictions wrong. From the complete absence of a prediction of the 2008 crash, to the Trump and Cameron elections, to Brexit and Leicester City, the analysis of data has been completely wrong. Not just a bit wrong, but completely wrong. How wrong can you get a choice between win and lose?

So why aren’t we amateur data addicts a bit more worried about the way we use data? It is just too seductive isn’t it? After the meeting I described at the start, a colleague from another school said to me that he was surprised that I wasn’t so keen on using real data to describe progress, because I was a mathematician after all. Perhaps that is the problem. I have a romantic relationship with the products of mathematics. I love the processes. I can sit over tables on spreadsheets for hours as happy as the bunnies in the Fibonacci story. But, I can also see my lover’s faults! If you take a large bunch of numbers and perform some interesting and quite complex maths on them, you can produce conclusions. The main criterion for the value of the conclusions is the sophistication of the maths. The more complex, the more seductive. The second criterion is the quality of the presentation of the results. Colours and graphs, animations and alerts are just so persuasive. We often fail to question the nature of the data it has come from. Beautiful constructions from bad bricks still fall down.

A weather analogy. My colleagues were looking at students targets based on predicted grades. They had a wonderful system that showed up when the deviation of performance from expectation was large or significant. If performance was lower, then it clearly showed something was up. Red lights should flash. Alarms should ring. Reasons need to be found for the underperformance of the child. Note that it is always the underperformance of the child and not the overperformance of the prediction. If yesterday’s weather forecast had predicted a sunny day, but today turned out rainy, would we be looking to see why the day had underperformed? What is wrong with the day that it has failed to live up to its expectation? I suggest we are more likely to call the prediction into question than our observation of the weather. We would blame the weathermen or the Met office, or the website or the newspaper. They have got it wrong again. But with kids we would rather blame performance of the child than the inaccuracy of the prediction. Strange isn’t it.

So, does all this data really constitute evidence? I must come clean and say the correct answer is “Yes, but only a bit!”. If a child is graded and the grades go steadily down, we should try and find out why. If a whole class of children’s grades are going down we should try to find out why. But what we must not get sucked into doing is thinking that the grades are the complete picture of the child’s leaning, just because we can plot a graph. We should not think that this constitutes evidence of learning whether it goes up or down. Evidence is much more complex than that. These trends are evidence only of grades.

chart-1

More correlation from tylervigem.com

Another weather analogy. Describe yesterday to me, from a weather point of view. Just have a think about what you would say if I asked you to describe the weather in a couple of sentences. You could do it couldn’t you? Now describe the weather yesterday in a number – just one. A bit harder? Will you give me the temperature, and if so at what time, or would it be an average? Or perhaps you would choose the hours of sun, or the millimetres of rain? The pollen count or the relative humidity? Difficult to imagine summing up the weather in one number isn’t it. So if we cannot sum up a day in one number, why do we attach such credibility to a lot of days summed up by a lot of numbers? A graph of average temperature over all of the days of a month is a collection of a lot of single days reduced to numbers, and no more than that. We can look at trends of course, and therein lies the power. But it is only the trend of a single number on a single day. The reduced encapsulation of a day into a number is no less a reduction of data just because we do it a lot.

Let’s take a step backwards. You gave me a couple of sentences to describe the weather yesterday and we probably happier with that than with any single number you produced. Even then, I wonder if it captured the whole of the glorious complex changing weather of the whole day. Did you catch all the subtle changes of light as the clouds move, the sun drops and moon appears? Did you capture the different nature of the rain as it moved from a few sporadic drops, accompanied by a drop in wind and a lowering of temperature, to a tumult that washed away the leaves left on the ground by the wind of the morning? The point I am trying to make is that we can only experience the whole day be being in the whole day. Anything else is a reduction. The same is true with a piece of student work. They may write an essay, paint a picture or write up an experiment. Any comment we make is a reduction from the complexity of the work. To reduce further to a number removes evidence rather than creates it.

With the weather we are unlucky. If you arrive in my town and I tell you that yesterday was a glorious day, you can never experience it. Not that exact day. It is gone. But in education we are largely very lucky indeed. A piece of student work usually still exists, and we can go back to it again and again. I say usually, because a performance is not usually the same. I unfortunately missed a play at my school last week due to some unavoidable meetings. I am told it was wonderful, but I will never see it. Shame!

Here is the nub of my concern. When my colleagues spoke about evidence they meant sets of numbers, and occasional comments that summed up some spurious reductionist data and had value only because we can draw pretty graphs. When I say evidence, I mean the work itself. That is where the evidence lies. Let us resolve never to reduce the glory of a child’s learning to a number, but look at the work itself and see that here is the evidence of the progression of his communication skills or her analytical skills. Let us never send to parents a number that we pretend describes their child unless we have shown them the piece of work in all its complexity, with all its strengths and weaknesses. Then we can engage in reflection on learning that actually means something.

Don’t try to encapsulate the development of Picasso’s artistic career in a set of numbers and pretend that it describes his art. We are often guilty of doing no more than the analysists who would do this by listing how much the paintings sell for. The cost of everything tells us the value of nothing.

Assessment article

I have just had an article published in the International Teacher magazine. It is not really particularly new, and is more of an edit of the assessment pages on this site. However if you want to read it, it is here.

It is no longer available on the site, so I have added it here.

Assessment

Is there any topic in education that exercised more chapters in books, more conferences and workshops and more column inches in the papers than assessment? How we do it, what it is for and the validity of what it shows us are all keenly debated. At one end we have the interpretation of results:  “Trend for grades to increase shows steady dumbing down of educational standards” or “Trend for grades to increase shows improved teacher training.” Take your pick. The interpretation depends more on your preconceptions than the numbers. At the other end assessment is not about measuring at all. It is formative and relies on feedback to students. How do we make sense of this forest of meanings for assessment, and the jungle of opinions about how it should be done?

We are guilty, in schools, of creating a very artificial environment for our children. This happens in many ways, from uniform to bells, in grouping by age or by ability. None of these things happen much to most people in adult life, or in childhood outside school. Nowhere is this more true than in the realm of assessment. You would be right, at this point, to turn the tables on me and ask what I mean by assessment, but I will refuse to take the bait. I will use various examples where the word is used to define a certain practice, but the definition will elude us for a while. In the end, perhaps we can get rid of such a confusing term altogether.

A wonderful example of assessment comes from Mary Poppins. I am not necessarily recommending you see the musical but near the beginning Mary, the new Nanny, meets the children for the first time. She takes tape measure from her bag and holds it up against the boy and reads “A noisy, mischievous, troublesome little boy.” She then does the same to the girl, which reads “Thoughtless, short-tempered and untidy”. When it is put against Mary, it reads “Practically perfect in every way.”

We may laugh at the magical properties of the tape measure, but are we so far away from that when we use numerical evidence as a basis for a discussion of a child’s learning? For many teachers it is a short step to using the same data to justify conclusions about their personality. Perhaps these are the teachers who would like the students to think they are practically perfect in every way.

Let’s get back to the real world outside school. We know that most people in most jobs are rarely, if ever, examined or tested, certainly not on a weekly or even daily basis. We also know that most of these people are not regularly graded with averages calculated to see how they are progressing. Yes, there are many cases where sales figures and production targets are used to assess an employee’s worth or even salary, but that is not the same thing. To see why not we need to look more deeply at what assessment is.

 

 

 

Assessment is essentially reductionist and distant.

Look at this picture. How do we think the artist might have arrived at this piece of work?

Hong Kong Skyline – Joyce Hadiwibawa

 

Details from Hong Kong Skyline – Joyce Hadiwibawa

Details from Hong Kong Skyline – Joyce Hadiwibawa

I have seen the sketches and photographs that the artist worked on leading up to this painting of a fictional skyline. But there is more than just the artwork. This painting comments on political and social issues of the day. At the time of painting there was a lot of discussion about illegal structures being added to buildings in Hong Kong. Many of these were on roofs as the details show. The two candidates for the post of Chief Executive Officer, the effective leader of of Hong Kong, were both accused of creating illegal extensions to their homes.

The question is how do we assess this work of art and what value does assessment have? For any piece of work, the general pattern goes something like this.

  1. The teacher discusses the idea of the work with the student
  2. The teacher comments on the sketches and research as the student progresses.
  3. The teacher gives formative feedback to help the student progress with the work.
  4. Once the work is complete, the teacher comments on the work, giving suggestions how the student can progress with the next piece.
  5. The student reflects on what they have done and what they have learned.
  6. The teacher assigns a grade to the work. This may be a number or a letter. It may be related to a rubric, or a set of criteria.
  7. The student judges the success of their own work by the grade that they have got.
  8. Parents see the grade and congratulate or comment to the student. Sometimes they see the work.
  9. The grade goes into a calculation of another number or grade that includes other pieces of work.
  10. This term grade or course grade is viewed by parents, colleges, universities or other relevant outside

Now, how much of this is actually assessment is rather open to different points of view. The purist traditional teacher will say that assessment means that you take a piece of student work and assess it’s value by giving it a mark or a grade. Anything else is feedback or discussion. Those brought up in the era of formative assessment will want to include all of the steps as being part of the assessment process.

Semantics aside, the point I am trying to make is that each step takes us further away from the student and their piece of work. Each step reduces the piece of work first to discussion, then to a comment, then to a reflection, then to a number, then to a component of another number. This is why the process of assessment is essentially both distant from the work and a reduction of it.

What works for adults works for children

We would never consider assessing work that is done by adults in this way. To take the Art example again, imagine going to one of the great galleries and seeing a great piece of work. When I go to the Raina Sofia Museum in Madrid and stand in front of Picasso’s Guernica, I can scan the work and soak in its majesty. The idea that a comment from someone, however erudite the critic, could stand in for the work is ludicrous. To go even further, any suggestion that I should reduce this piece of Art to a number is patently ridiculous. Yet we do it for students! All the time!

This is just as true in other domains. Einstein’s work on relativity is completely useless if we just comment on how good it is, or give it a grade. We need the work itself to be able to send rockets to distant planets or design a GPS system. We cannot use the assessment of the work in place of the work. Only in school do we say to a student or a parent or a university that a student is working at a numerical level and expect that to stand in place of the actual work.

Even as teachers we are not normally assessed by reduction to numbers. It is true that some systems do this, but there is no evidence that I have seen that indicates this helps teachers improve their practice at all. As we have seen in the previous chapter, the discussion about our work is helpful as it helps us reflect but an assessment, in the pure sense, doesn’t.

Why can we not apply adult behaviour to students?

This may seem as an argument for formative assessment over summative assessment, but I think we can go further than this.

An Interesting Article

I came across a good article by Dylan Wiliam in the TES the other day. It is called “The 9 things every teacher should know“. None of this is earth shattering, but it is clear and supported and well written by one of the most sensible and honest of the pack of educational gurus that roam around our schools.

I first came across Dylan when I first started teaching back in the early 1980s. We were both Maths teachers in London and collaborated writing a couple of O level and GCSE papers if my memory serves. He went on to be the pioneer of Assessment for Learning with groundbreaking work such as Inside the Black Box. His 2010 BBC TV series The Classroom Experiment is on YouTube.

There are obvious things about learning in this article, that we sometimes forget.

“It is important to remember that unless students are struggling to make sense of what they are learning, little knowledge is likely to be retained in the long term.”

“How well students perform in a learning task is inversely proportionate to how much they will remember weeks later.”

“For successful collaborative learning there should be group goals: students should be working as a group, rather than just in a group. And that each member has to be accountable to the group for the quality of their contributions.”

Have a read, and go to Dyan’s website for more papers and videos.

A suitable response?

detention

After a discussion at a meeting at Island School, I was given to write down some thoughts about detentions and other punishments. Here they are. I don’t think any of it is earth shattering – fairly obvious really.

I start from the principle that any measure that we implement in response to a behavioural issue must serve to prevent the student from repeating that behaviour. If it doesn’t do that then it shouldn’t happen. This view stems from the idea that we are in the business of teaching children how to behave positively.

What is interesting about a lot of the literature about detentions is that the argument for their effectiveness often comes down to the fact that it is something students do not like. Phrases such as “Detentions at lunch time are particularly effective because it removes social time and they do not like that.” And “No one likes having their free time removed” abound. The argument goes that if the threat is something they do not like, then they won’t do it.

I think this view comes from a paradigm that is wrong on several levels. It is based on the idea that education over behaviour is a conflict that we, the adults, must win. We must make children feel worse as a consequence of what they have done. I have worked in schools where this atmosphere is very much the case. The conflictual nature of behavioural transaction between teachers and students has been so deeply engrained in the school and the prevailing community that it is very hard indeed to see a way to break that cycle. In that sort of school getting students to behave well, even for the wrong reasons, is a victory. I am lucky in that I work in a school that is not like that. We are lucky enough to be able to strive for more and aim for children to behave well for the right reasons. We want students not only to display responsible behaviour but to do so because they care for others and can take responsibility for their own well being.

The paradigm of retribution is expressed in many forms. We say, sometimes, that people must suffer the consequences of their actions. I do feel uncomfortable with an education policy based on people suffering or deliberately making then do what they do not like. Of course we all have to do things we do not like, and so should children, but that is different from making them do them because they do not like them. There is something inhumane and rather misanthropic about this viewpoint.

The next concern I have is that I have never seen evidence that this process actually improves behaviour. If the argument is that of a deterrent, then it becomes a deterrent to being caught rather than the behaviour. Getting away with it is therefore equated with success.  The number of repeat offenders in any detention system would rather indicate that it has a rather low success rate. We have to ask why this might be.

Research and my own observations indicate that there is a strong correlation between students who feel connected to their community, feel valued and, crucially, liked and those who behave well or positively. Similarly, those who feel alienated, dissatisfied and unappreciated tend to behave worse, and take less notice of the codes of the community. They have less motive to behave well to and for others, if they do not feel respected by them and part of their society.

Now there could be a chicken and egg argument here. Does alienation lead to bad behaviour or does bad behaviour, and its suffered consequences, lead to alienation? Actually it doesn’t matter. If the former we need to work on the alienation, if the latter we need to work on the consequences.

We value the conversations that help students understand the harm their behaviour can cause to themselves, other individuals or the community. This is true whether they miss lessons, disrupt others’ learning, take drugs or act violently. The success of these conversations leads to a growing understanding of the effect their actions have on themselves and the community and their part in it. I would argue that the detention has a tendency to undermine all the good work.

We want students to believe we care about them, because we do and because that helps them see the truth of what we say and act appropriately. We want them to care that the adults close to them will be pleased by their good behaviour. We want them to want to please. If they feel we care, the guilt they feel about letting us down will be far more effective than the fear of a meaningless consequence.

If the attentive caring conversation is followed up by a detention, I suggest the default response is often to wonder how authentic the conversation and the advice was, to take another small chip away at the connection the student feels with the school, and to increase the chance of the not caring about the consequences of the next infraction.

This is why I talk about feeling the school code rather than understanding it. It is an emotional response rather than an intellectual one that will cause young people to change their behaviour. This is because it is an emotional response that causes their behaviour in the first place. Young people will transgress against the expected norms if they feel no emotional connection with the society. Note that I wrote “feel no connection” rather than think they have no connection. If it was a merely intellectual understanding, then a logical argument will explain to them why it is wrong and so they will understand and never repeat it. We know this does not happen. The logical explanation is never enough. They need to feel the meaning, feel the guilt and feel the need to change.

Any suffered consequence that takes away or reduces the guilt that they have been wrong, lessens the chance of the lesson being learned. If the student can possibly feel that the response is unjust, unfeeling or unfair, then they will grasp this straw as a lifeline to reduce their own guilt and replace it by the much more comfortable feeling of being a victim. Once we have got to that point, we have lost.

We are all victims of our own past. We are created by our own experiences. So, my views on detention and punishment in school are framed by my school experiences. I had numerous detentions, and I don’t think I ever thought that any of them made sense. It reinforced my opinion that the adults in the school were, with few exceptions, people who did not care for me. I really couldn’t care less if I upset them because they couldn’t care if they upset me. I was not an easy student and confrontation followed confrontation. I am now adult enough to see my responsibility in my actions. What has happened is that I have emotional distance and can see through the fog of feelings to apply my intellect. I also feel that the detentions did the opposite of help my feelings. If only someone had made me feel guilty rather than wrong, I might have been a better person.