The Limits of Skill.

Daniel Kahneman’s book Thinking, Fast and Slow seems to be often-cited by those in education who advocate macro-scale ‘evidence’ over individual experience.

The book provides some challenging insights from an educator’s perspective, but it seems to me that this is another case of confirmation bias on the part of those with a particular axe to grind. Kahneman’s work has been (mis)appropriated for ideological purposes in much the same way as has happened, I suspect, to that of Carol Dweck.

The early chapters do indeed appear to dismiss the ability of the human mind to make rational sense of the world around it, when compared with the power of statistical analysis. A key foundation of Kahneman’s work is the dominance of the instinctive mind over the rational part, whereby much of our understanding of the world is constructed from a series of assumptions that may – or may not – resemble a more objective reality.  But this is certainly not the main thrust of the book, which is concerned with the failure of (economic) rationalism to explain observed human behaviour. Kahneman is, after all, the father of behavioural economics.

It is easy to see how this argument could shift decision-makers towards the use of algorithms, as seems to have happened in everything from job interviews to teaching methods, to ways of assessing educational outcomes. I will naively assume that its tendency also to shift decision-making away from front-line practitioners towards managers and policy-makers – The Few who have access to supposedly more sophisticated aggregate data – is merely coincidence…

But Kahneman’s central theme is more sophisticated than that: he argues that the world is so complex that a key, often-ignored determinant is Chance, coupled with the tendency of phenomena to revert to the mean (in other words, the norm). He proceeds to show how so-called expert analysis is often no more reliable than a random guess.

He also distinguishes between two sorts of human intuition: that which operates in ‘low validity’ circumstances, where predictions are based on hunches against essentially random events – and that which operates in regular circumstances, whereby the practitioner has had the ability to learn from experience in a controlled and repeating environment. A stock market trader is an example of the former; a clinical practitioner the latter.

The former he shows to be little more than guesswork – whereas the latter is in fact the act of experienced individuals subconsciously recognising and responding to patterns that they have encountered before, and modifying them as needed. This is close to my experience of teaching over many years, and I think it is a hugely significant distinction that has been ignored by those who use this book to argue for a technical approach.

Kahneman does not address the educational world directly, but one can debate whether the classroom constitutes a regular, learnable environment more than an information-noisy, random-guess situation. Despite the short-term unpredictability of individual lessons, I would argue that in the long term, it does. And if this is so, then Kahneman’s suggestion that considered experience is about as good as most algorithms in making sound decisions can apply to teaching. This is not to suggest that teachers never resort to hunches – but to say that they should not, aiming instead for deeply-considered judgements. But equally, they should be wary of the pseudo-rational claims of statistics.

However, this is still not Kahneman’s over-riding point; in fact, he argues, none of the approaches available to us is likely to make anywhere near as much impact on outcomes as we like to believe.

People have some ability to intuit short-term outcomes with success, based on a reading of circumstances, but this gives the illusion of greater control than actually exists, thereby creating the false sensation of expertise.

The illusion of coherence and the over-confidence that it breeds may indeed make for bad intuitive judgements, and we should beware this.  Hindsight creates the further illusion that events are linear, predictable and controllable, in way that is far less true than faith in human competence demands. Therefore we should also beware of claiming that human skill is able to influence outcomes to anything like the extent that most professions – and their masters – require

It is entirely understandable that professionals being held to close account will crave and claim the power to make things happen in an orderly and predictable way – but if Kahneman is to be believed, it is just not true. It is difficult to argue that anyone should act in a professional capacity simply on the grounds that “I felt like it at the time” – but much easier to acknowledge that long experience is a valid basis for decisions. The higher the degree of confidence, the more we should be suspicious of the claim; it is better to advance cautiously from a position of low certainty – but it takes far greater maturity to concede the limits of one’s powers than to ignore them.

One of the book’s key claims that will stick in my mind is that truly competent experts are those who know and accept the limits of their expertise.

The second half of the book is an exploration of why human behaviour is much better explained by recourse to psychology than the rationalism of classical economics suggests. Given that education deals almost exclusively with the domain of human behaviour, it seems extraordinary that some within it seem intent on finding a rationality equivalent to classical economics, even though that is increasingly shown not to explain the world adequately. One is left wondering whether such people have even read the second half of Kahneman’s moderately-demanding book.

If we accept his findings, as teachers we have little choice but to resort to a skilled heuristic approach. It would be much better to invest our effort in developing this to the highest possible level – while always acknowledging its limits – rather than deceiving ourselves (and everyone else) that education is controllable in the way a production line is.

The flaw with much current educational debate is that it requires knowable endpoints and finely controllable situations that simply cannot be guaranteed; it then holds people unrealistically accountable for delivering them. Moreover, the data-based approach presupposes outcomes whose criteria are little more than arbitrary benchmarks which may have little to do with the real world as experienced through individual lives or that reflect the workings of real causality.

Despite his faith in logic, Kahneman shows why a rationalist world-view is inadequate for explaining the human condition – and in the end, it is surely more use to apply theory to human need than it is to manipulate the latter simply to obey the laws of logic. Just as with economics, education should serve people, not the other way round.

9 thoughts on “The Limits of Skill.

  1. Great post. Nothing at all to add except to ask if you know of Jack Marwood’s work? His blog is called “Icing on the Cake” where he discusses his thesis that school adds a relatively small percentage onto what a child brings to school. He’s got a background in statistics and is talking back at people who are asking teachers and students to serve the numbers (I’d say “unconsciously asking” because, like you, I don’t think this is being done to schools on purpose… it’s just that some misguided people seem to be quite important).

    • Thanks Leah. I do know that blog but maybe I need to take a closer look. Kahneman’s work is quite salutory – suddenly facing the possibility that we don’t know as much about, or make as much difference to the world as we think.

  2. Super post TP – I think the Kahneman thesis has huge implications for modifying our understanding of huge issues in educational theory – ‘evidence-based approaches’, ‘professional judgement’, performance related pay etc. etc.

      • Thanks Leah – I’ve had a bit of a mini ‘sabbatical’ this half-term – trying to nail the nuts and bolts of the day job! Need to focus on an assignment for my Masters for a couple of weeks but some promising blogging should come out of what I’m focusing on.

    • Thanks Chris – I agree. And the more I delve into work like tgis, the more important I conclude it is for educators to appreciate it, rather than worship production-line edu-economics…

      • Yes me too… 😉 I read Thinking Fast and Slow 3 years ago, but at the time was pondering it from the ‘big’ overall viewpoint of how we as a race go about life. I wondered whether I could envisage teaching any of its messages via whatever means at a primary school level and felt pretty overwhelmed and depressed!

        Thinking about it again now through the lens of how we evaluate the practice of teaching and things I’ve written about in recent months, I had a sudden flash of inspiration regarding how it could relate to the areas I mentioned above and perhaps work with other things I’ve been thinking. I need to work out some careful details though.

        I was also given an audio version of the book, so over the summer I’m going to channel it into me through headphones while doing domestic chores, and reflect on it all again from the perspective of how we think about teaching – the nature of ‘professionalism’ and how we develop and evaluate it etc. Perhaps I’ll be in a position to write something useful later in the year (if not the holiday)!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s