The Limits of Skill.

Daniel Kahneman’s book Thinking, Fast and Slow seems to be often-cited by those in education who advocate macro-scale ‘evidence’ over individual experience.

The book provides some challenging insights from an educator’s perspective, but it seems to me that this is another case of confirmation bias on the part of those with a particular axe to grind. Kahneman’s work has been (mis)appropriated for ideological purposes in much the same way as has happened, I suspect, to that of Carol Dweck.

The early chapters do indeed appear to dismiss the ability of the human mind to make rational sense of the world around it, when compared with the power of statistical analysis. A key foundation of Kahneman’s work is the dominance of the instinctive mind over the rational part, whereby much of our understanding of the world is constructed from a series of assumptions that may – or may not – resemble a more objective reality.  But this is certainly not the main thrust of the book, which is concerned with the failure of (economic) rationalism to explain observed human behaviour. Kahneman is, after all, the father of behavioural economics.

It is easy to see how this argument could shift decision-makers towards the use of algorithms, as seems to have happened in everything from job interviews to teaching methods, to ways of assessing educational outcomes. I will naively assume that its tendency also to shift decision-making away from front-line practitioners towards managers and policy-makers – The Few who have access to supposedly more sophisticated aggregate data – is merely coincidence…

But Kahneman’s central theme is more sophisticated than that: he argues that the world is so complex that a key, often-ignored determinant is Chance, coupled with the tendency of phenomena to revert to the mean (in other words, the norm). He proceeds to show how so-called expert analysis is often no more reliable than a random guess.

He also distinguishes between two sorts of human intuition: that which operates in ‘low validity’ circumstances, where predictions are based on hunches against essentially random events – and that which operates in regular circumstances, whereby the practitioner has had the ability to learn from experience in a controlled and repeating environment. A stock market trader is an example of the former; a clinical practitioner the latter.

The former he shows to be little more than guesswork – whereas the latter is in fact the act of experienced individuals subconsciously recognising and responding to patterns that they have encountered before, and modifying them as needed. This is close to my experience of teaching over many years, and I think it is a hugely significant distinction that has been ignored by those who use this book to argue for a technical approach.

Kahneman does not address the educational world directly, but one can debate whether the classroom constitutes a regular, learnable environment more than an information-noisy, random-guess situation. Despite the short-term unpredictability of individual lessons, I would argue that in the long term, it does. And if this is so, then Kahneman’s suggestion that considered experience is about as good as most algorithms in making sound decisions can apply to teaching. This is not to suggest that teachers never resort to hunches – but to say that they should not, aiming instead for deeply-considered judgements. But equally, they should be wary of the pseudo-rational claims of statistics.

However, this is still not Kahneman’s over-riding point; in fact, he argues, none of the approaches available to us is likely to make anywhere near as much impact on outcomes as we like to believe.

People have some ability to intuit short-term outcomes with success, based on a reading of circumstances, but this gives the illusion of greater control than actually exists, thereby creating the false sensation of expertise.

The illusion of coherence and the over-confidence that it breeds may indeed make for bad intuitive judgements, and we should beware this.  Hindsight creates the further illusion that events are linear, predictable and controllable, in way that is far less true than faith in human competence demands. Therefore we should also beware of claiming that human skill is able to influence outcomes to anything like the extent that most professions – and their masters – require

It is entirely understandable that professionals being held to close account will crave and claim the power to make things happen in an orderly and predictable way – but if Kahneman is to be believed, it is just not true. It is difficult to argue that anyone should act in a professional capacity simply on the grounds that “I felt like it at the time” – but much easier to acknowledge that long experience is a valid basis for decisions. The higher the degree of confidence, the more we should be suspicious of the claim; it is better to advance cautiously from a position of low certainty – but it takes far greater maturity to concede the limits of one’s powers than to ignore them.

One of the book’s key claims that will stick in my mind is that truly competent experts are those who know and accept the limits of their expertise.

The second half of the book is an exploration of why human behaviour is much better explained by recourse to psychology than the rationalism of classical economics suggests. Given that education deals almost exclusively with the domain of human behaviour, it seems extraordinary that some within it seem intent on finding a rationality equivalent to classical economics, even though that is increasingly shown not to explain the world adequately. One is left wondering whether such people have even read the second half of Kahneman’s moderately-demanding book.

If we accept his findings, as teachers we have little choice but to resort to a skilled heuristic approach. It would be much better to invest our effort in developing this to the highest possible level – while always acknowledging its limits – rather than deceiving ourselves (and everyone else) that education is controllable in the way a production line is.

The flaw with much current educational debate is that it requires knowable endpoints and finely controllable situations that simply cannot be guaranteed; it then holds people unrealistically accountable for delivering them. Moreover, the data-based approach presupposes outcomes whose criteria are little more than arbitrary benchmarks which may have little to do with the real world as experienced through individual lives or that reflect the workings of real causality.

Despite his faith in logic, Kahneman shows why a rationalist world-view is inadequate for explaining the human condition – and in the end, it is surely more use to apply theory to human need than it is to manipulate the latter simply to obey the laws of logic. Just as with economics, education should serve people, not the other way round.

Locked in my own head

When the facts change, I change my mind. What do you do sir,? J.M. Keynes

In my last post, I observed that if we are expecting the blogosphere (one day) to deliver answers by way of improving education, I think we are going to be disappointed. In many ways, it is a fantastic device for professional discourse, and it has enriched my own professional experience very greatly, causing me to consider many issues and outlooks that I might not otherwise have encountered.

But the operative word is ‘consider’ – for ultimately the benchmark by which I evaluate what I encounter is always my own prior understanding of learning. Nothing that I have yet seen has carried sufficient objective robustness to demolish those preconceptions to the extent that a total rethink has proved necessary.

There are those who would probably argue that this is just evidence of my fixed mindset – or at least inflexibility. Those of a scientific bent might well argue that my own preconceptions simply should not come into it: what we should be looking for are scientific truths about what works in education, and if the weight of evidence is against me, then I should change my views. All well and good – but just where is the evidence that does carry that level of confidence? I’m still waiting…

I am not going to say that people are wrong to seek greater rigour in education – but I think that the attempt to build a generalised definition of good practice risks diverting us from the place we really need to be looking. Even the most dispassionate of scientists amongst us cannot escape the realities of the educational world: when we deal with (young) people, we are operating in the social and human domain, for which scientific approaches may just prove too technically precise; those soppy soft skills may actually be better. As my afternoon lesson today showed all too well, there are times when trying to be efficiently scientific in the way one deals with a situation simply do not produce the expected results….

While it may be perfectly possible to share good practice when it comes to class management, this is not at all the same as the cognitive process that is Learning. They may be related in the school setting but the former is a logistical exercise in people-management that comes about simply because society chooses to educate its young in groups where their collective behavioural dynamic needs to be steered. The latter, by contrast, is an invisible cerebral process that takes place solely within the brain of the individual, even though it is of course capable of outward expression. Much of the discussion of what constitutes good or successful teaching risks conflating these two elements.

It is a simple existential truth that at one level, each of us is locked inside our own head. For all that we can communicate with each other, each person’s experience of this world is entirely unique to them and not directly sharable with any other. Therefore the only kind of learning  of which we can gain any direct experience is our own. Even that is not easy, because the process can elude even the individual concerned, with the fact that learning did take place (possibly in a highly convoluted way) only becoming apparent in hindsight. So how much more difficult is it to make confident assertions about what or how others have learned, when we have not the slightest ability to know what really goes on in their heads?

I suspect that this is why educational discussion largely remains mired in a mass of anecdote, subjectivity and emotional (over?) reaction  – mine included. The inescapable fact is that the reference base-line is our own knowledge and experience of what it means to learn, as experienced by us and us alone. And why not? This is the only reasonably tangible experience we actually have to go on. Therefore, when we read something that brings new information, the prime reality-check is whether it fits with our own experiences or not; it may also explain the depth of feeling that debates on these issues often generate. A discussion of teaching and learning cannot help but be a discussion of our own core values and life-experiences – and who is going to dare to stand up and judge who is ‘right’ and who not on matters such as those?

I think this issue does, however, have more than philosophical implications. It means that if we want to understand the nature of learning, we perhaps need to spend less time in conferences and webinars and more time looking inwards at our own experiences as successful learners. The best thing to do to ensure that you do not go stale as a teacher is simply to keep learning yourself – and then taking the time to reflect as rationally as possible on what that process means.

To my mind, learning is not some kind of rarefied and elusive thing – it is the most natural, even common thing that the brain can do – arguably its raison d’etre. A life spent playing computer games or at the roulette table will not be devoid of learning; nor will a life spent in the company of criminals. This is not to say, of course, that all types of learning are equally useful or socially desirable – and I would suggest that some forms of activity are more learning-dense than others. But be that as it may, one of the most important things we can do as teachers is to reflect on our own experiences, and then to replicate them (possibly in modified form) in a way that will help others to follow the same path.

The implication of this is that one’s classroom style is probably inseparable from one’s own psyche and outlook; it may have consequences for the success that different teachers have with different pupils, and perhaps the way that such matches are made. But I would suggest that it is simply impossible to separate any individual from the way their own brain works. It is impossible fundamentally to think in any way other than your own – though this need not mean that one’s thinking is incapable of evolution over time. The trick is to understand how that happens.

The logical conclusion is that we need to accept that teachers are what they are when it comes to their teaching style; while we may reasonably make judgments about their effectiveness with respect to the various proxies we use for learning, people can only really teach in the way they do. It also means that, in the interests of fairness and accuracy, we need to be very wary of the limitations of those proxies in judging the real effect of a teacher on his or her pupils.

It means that any attempt to define ‘better’ and worse’ ways of teaching is a futile exercise, partly because there are just too many to categorise usefully – and partly because learning itself is so diverse and personal a process that it is impossible to judge what works and what doesn’t in any universal sense.

Finally, this might also allow us to achieve a degree of peace in the ever-churning cauldron of conscience regarding what we ’ought to do’, and how we justify it. There will be as many acceptable answers as there are reflective practitioners. My conclusions that traditional methods work best (for me – and hopefully my pupils) derive purely from my own experience, and there is little that can match that experience when it comes to individual decision-making.

In my case, my continuous learning has been in the fields of my musical and model-making activities (and much reading), all of which are learning-dense and where I constantly put myself in the shoes of the learner. For example, when I started learning the violin several years ago, I took online lessons which I followed roughly weekly – with associated practice. While I did have some useful prior experience (the mandolin shares a lot with the violin), there is no doubt in my  mind that progress was most rapid and focused when I was being shown precisely what to do, and given exercises to allow me repeatedly to imitate and develop. There were aspects of technique that I simply would have not discovered – at least nowhere near as quickly and accurately as by being shown them by an expert.

In recent years, I have stopped the lessons, and have allowed my ‘learning’ to be more serendipitous and self-guided. The result is that I have made nowhere near as much progress as I was doing under tuition; I have fallen into some bad habits and have probably not developed the degree of further proficiency that might have been expected. While learning through ‘idle play’ is less pressurised and in some ways more pleasurable, it is simply not delivering the results of a more formal programme – despite the fact that I do now have a degree of proficiency that I can bring to bear on the matter.

I am certainly not going to claim that this is cast-iron proof that direct instruction always works best, although I have had a couple of other recent experiences where ‘guide on the side’ style teaching has simply left me feeling I had never got to grips with the nuts-and-bolts of the topic we were supposedly studying.

According to my own logic, there will be as many answers to this question as there are people asking it – but I would suggest that I really don’t need much more evidence or justification for why I hold the views I do, and why I teach the way I do. It is also why attempts to create consensus about what is great teaching (whether online or elsewhere) stand about as much chance of success as those of defining great Art.

While we do have a responsibility to consider the individual needs of our students, I think it is perhaps unrealistic to expect teachers to operate in ways that make no personal sense to them, any more than we do artists. And we need to accept that this is all the justification that can, or needs to be given.

Take the log from your own eye…

“You hypocrite, first take the log from your own eye, and then you will see clearly to take the speck from your brother’s eye.” Matthew 7:5.

In her book Smile or Die, Barbara Ehrenreich dismantles the cult of positive thinking. In modern society, it is just not done to be critical – one is expected to be unremittingly positive, upbeat and optimistic; if you’re not, it’s because you’re bitter or a loser. Ehrenreich says that this is foolish because it causes people to ignore the nature and possible seriousness of underlying problems. She should know, having written the book while being treated for breast cancer.

The education world suffers from the same syndrome. I suppose that’s not surprising – after all, its raw material is the eternal promise of young people’s futures. But we also seem to suffer from an inability to divest educational practice of the social and political happy-agendas that only constrain thinking. We too are required to be relentlessly on-message, whatever our misgivings. To differ from the official line is merely to confirm one’s unsuitability, and this is becoming progressively worse the more forcefully officialdom imposes its monolithic, top-down view of what is educationally ‘correct’. The current Secretary of State, Michael Gove, has reportedly said that if you don’t agree with him, you’re an “enemy of promise”. The educational gurus, management fixers and unions have a different, but equally entrenched position and could probably use the same strap-line with equanimity.

It’s not easy, swimming against such a strong tide. The consequences of being a nay-sayer are too great for many, be that in terms of the general effort required, the disapprobation of the peer-group or more pragmatically the risks to one’s career progression. And the higher one goes, the greater the risk.

From time to time, I am given to wondering whether it really is me who has it all wrong. After all, 10,000 lemmings can’t be wrong – can they? When one spends the larger part of one’s working life in an environment where the basic assumptions being made seem to conflict with one’s own, it is only logical to wonder where the error actually lies. All the more so when one acknowledges one’s own latent Eyeore tendencies.

This is where a book like Rolf Dobelli’s can be helpful. I have known about cognitive biases for some time through my teaching of Critical Thinking, but this work examines them more thoroughly than I hitherto had. It helps one to do what I believe any thoughtful professional should, namely to question one’s assumptions. For example, the confirmation bias makes it likely that one will always interpret new information in a way that confirms one’s existing preconceptions. Couple this with the clustering illusion, and it is easy to create a coherent narrative from what are actually random occurrences. The over-confidence effect means that one likely to over-estimate one’s ability to be right, while outcome bias means that one is likely to be taken in by self-fulfilling prophesies, thus misidentifying any real casuality. I could go on.

You risk ending up feeling intellectually paralysed – not a good situation, one might think. But while one should of course treat cognitive biases themselves with care, this process reminds one just how difficult it is really to understand the true nature of reality, and no amount of ignoring the fact will change that.

In my case, the brick wall that I come up against stems from my considered belief that education should be an end in itself. It is a process whereby one attempts to develop people’s faculties for understanding the world around them, and responding to it in a wise, considered and hopefully compassionate way. It is about encouraging people to engage in thinking for its own sake, both for the rewards that can bring and as a transmission mechanism for our cultural and intellectual capital. It is therefore a serendipitous, unique and open-ended process that tends to defy constraining laws and systems. That for me is a timeless truth.

The educational system in which I work sees education as a socio-economic tool, which exists in order to induct people into a societal game, and which is therefore infinitely transmutable depending on the nature of the game it is serving. In that sense, education is simply a means to an end, and one that, through its emphasis on employability is increasingly defined in terms of wealth and power. This is why it has no difficulty couching ‘success’ in terms of hoops jumped through and targets met. It has no problem with closed thinking, as it wants people to arrive at pre-ordained answers rather than original solutions.  It is not especially concerned about the impact on the individual so long as the rules of the game are thoroughly instilled.

In my view, what results is Training, not Education.

I find this view intolerable; it is dehumanising and it diminishes people’s ability to make their own way through life.  It narrows the scope of education and the methods that might be used to deliver it, and it arguably demonises those who dare to differ. There is good evidence that the values it instils are psychologically and even physically harmful to those they touch. I certainly see plenty of signs that young people going through this process aren’t ending up better able to think for themselves or to know the world around them – or even to want to know. Curiosity is one of the first casualties of education-by-targets.

A couple of years ago, I sent a book manuscript on this theme to several publishers. None accepted it. Eventually some feedback was forthcoming: it was not that the book was badly written, or poorly researched, nor that the case wasn’t plausible. The view was, “This is not what the Education world wants to hear at the moment”. A more unprofessional, uneducated response I cannot conceive of.

Now, cognitive biases could suggest that I might well have this all wrong, and that modern target-based education actually delivers far better life-experiences than my more holistic, bottom-up alternative. Maybe I am indeed bitter or a loser, but my own well-reflected-upon experience of life begs to differ. But if nothing else, the educational establishment would both validate and develop itself more successfully if it bothered to answer its critics rather than just criticise or ignore them, while continuing to spout whichever guru’s pet theory happens to be fashionable this week.

I would have more confidence in the establishment’s view if I saw it practising the same kind of rigorous self-scrutiny that I do; I wonder how widely known concepts such as cognitive biases are known in the education world. I also wonder what approach the erstwhile National College of School Leadership was taking, that we now have such homogenous school management policies across the sector – or is it just the fear of Ofsted?

At present, the bulk of the profession seems content meekly to do what it is told. This is not a climate guaranteed to retain intelligent, reflective individuals in teaching – they tend to be independent-thinkers, not corporate doormats. That doesn’t mean that everyone actually agrees (see the dangers of group-think), but several colleagues have advised me to give up my case: “You won’t win, so why bother?” is the usual argument.

Apart from the fact that this is not a matter of ‘winning’, the more this mentality prevails, the less likely grass-roots teachers are to be heard, and the less likely we are to have a genuine, open and thoughtful debate about what precisely education is for and how it works, free from the dogma of Michael Gove – and the happy-clappy educational establishment that just as mindlessly opposes him.