Utter Rubbish – a postscript

Saturday’s post (Utter Rubbish) went straight to the top of my all-time-most-read. I’m not sure whether that means there a lot of people in agreement, or whether lots are thinking what an inveterate fool T.P. made of himself. Either way, here is a short clarification:

Andrew Sabisky commented that Lleras-Muney’s number is the result of large-scale research, and that it is possible using natural experiment (I like that) to obtain plausible figures. However, subsequent (peer?) review apparently contested the result. Thanks to Andrew for the link.

But my title did not principally refer to the research itself, even though its provenance may be questionable and I stand by my comments over why it cannot be applied to the individual. It is possible that this is not what Lleras-Muney intended. It is widely accepted that increasing educational opportunity can have significant aggregate benefits for populations. It is particularly noticeable in developing countries where general life expectancy is low, and where certain groups (notably women) gain access to education for the first time.

What I think is U.R. is the way that such data are misrepresented to support partisan(?) agendas, particularly in relation to specific classroom practice. I don’t know whether Dylan Wiliam was just being careless in his use of language, but to my eyes he is saying that one year in ‘school’ will add 1.7 years to one’s life. As a justification for formative assessment, I think he presumes too rather too much!

It exasperates me when discussion of classroom practice ignores the reality of teaching specific groups of young people. Extending their longevity is not normally foremost in my mind, even when embedding formative assessment!

While it is true that educating a population is achieved by educating many individual people, the process is oblique and it is not straightforward to make direct links between the specific process and the general effect. Likewise, I have long criticised the commodification of ‘achievement’ as something distinct from the process of actual learning. Neither is quantifiable in any absolute way, and such approaches fail to capture the real life-benefits that ‘being educated’ can bring, which are mostly not objectively experienced at all. I am not sure how one would go about quantifying ‘achievement’ in relation, for example, to a growing appreciation of Literature or competence in a foreign language, nor how this contributes to the effects that Wiliam suggests.

One might further argue that the marginal quantity of life is not as important as its quality, and even in economic terms, increased longevity is as likely to add to the cost burden on society as reduce it.

The promotion of this utilitarian, economised view of education by those with significant clout has diminished a more individual, cultural and humane appreciation of its effects. This is the outlook of those who know the cost of everything and the value of nothing, and it has done untold damage to the wider perception of education in Britain.

That is where the Utter Rubbish really comes in.


8 thoughts on “Utter Rubbish – a postscript

    • Well, if you put it like that, then yes I suppose so.

      But I think that years of accumulated experience in a sense constitute ‘research’ too – not that anyone would credit it with much rigour. I wouldn’t make any claims for the applicability of that ‘research’ beyond the realms of an individual’s own practice, and I offer observations simply for the consideration of others. But I would like to think that it is at least an *educated* guess, if not more, and at least it is grounded at the scale where the resultant decisions are actually being taken.

      I entirely sympathise with the desire to understand the educative process more deeply, and research in the standard sense may be one part of that. It can perhaps tell us aggregate truths – but it struggles to explain phenomena at the scale of the individual interaction – and that is where education actually happens. Conflating the two is where the problems begin…

      Thanks for your comment.

  1. The other place where problems begin is where research bumps up against advocacy. As long as the running of schools is part of a democratic process, that’s going to happen; and I’m a fan of the idea that democracy is awful until you compare it to the alternatives. But the overlap can lead to bad ideas- and I think you’re right on the nail that Improving Educational Outcomes is good to do because it Makes People Live Longer is a bad idea. Partly because (humanely) education is an intrinsic good, but also because, even in numerical terms, the calculations don’t necessarily stack up.

    Prof. Lleras-Muney has a preprint of her paper online (http://www.econ.ucla.edu/alleras/research/papers/mortalityrevision2.pdf if you’re interested), and it looks like a neat bit of data analysis. I’m sure her conclusion stands up. But it’s based on the USA between the World Wars, when the number of years of compulsory education was sometimes zero, and mostly in single figures. Put to a UK scale, it’s mainly about whether KS3 was compulsory. It’s not obvious that that measurement is transferable to a western world where education to 18 is all but compulsory. (My guess is that extra years of education show pretty rapid diminishing effects on health).

    I’m reminded of Darrell Huff’s idea of a “semiattached number”. It feels like “1.7 years extra life” ought to be a good motivation to teach better, especially from a inspiring presenter, but if you poke the number a bit, it’s not got much relevance at all.

    [Actually the whole book “How to lie with statistics” is brilliant. I stumbled across it as a geeky teenager. It’s short, amusing, non-technical, and it’s scary how many of the ruses described are par for the course in modern edu-data]

    • Many thanks for that R.I. You made me feel better about myself despite having rather shot my mouth off! If you’re right, then perhaps my instinct was reasonable. It’s not that I particularly question the research, so much as the interpretation (spin?) that’s put on it – especially when it seems to be trying to tell me how to a job for which I have many more meaningful sources of information and inspiration.

      • p.s. that inter-war data would seem to support the comment that the effect is larger in countries at earlier stages of development, as the U.S. effectively was then.

  2. If I understand your post correctly, you critique the way education research attributes a longer life to better education? I for one, see quite clearly how a better standard of education in health education, increasing employment oppurtunities and social skills developed during schooling can lead to a longer lifespan. I understand that quality of life is much more important, but I think Dylan Wiliam clearly attributes the extra lifespan to a greater quality of life ensuring better health.

    Whilst experience, good and bad, can lead to some staff developing, the unbiased quest to find out what works in education has to be the best method of improving practise. There might be other sources which appear to allign with our own ideologies and anecdotal evience, but wouldn’t you much rather chose to receive medical treatment from a newly trained doctor than a homeopathic practitioner with 40 years of experience?

    • Thanks for your comment. My original was perhaps not as clear as it could have been, being rather rushed. I don’t question that education can lead to those things. What I am less happy about is making them the explicit goals of education. In fact, I don’t think they can be. For many entrepreneurs, wealth is simply the by-product of excellence, and directly aiming to ‘get rich’ tends not to work. In the same way then the benefits that Wiliam promotes can only be the by-product of lives well lived and minds well cultivated. Implying that the effect can be manufactured and then identified at the level of the individual is just ridiculous.

      The ‘unbiased quest to find out what works’ sounds admirable, but I don’t see much progress. I am not persuaded that research has yielded very much of genuine use to the classroom practitioner, and I am in fact not convinced that it is not actually barking up entirely the wrong tree.

      Education is a culturally-defined phenomenon and as such is therefore not scientifically knowable, even if the workings of the brain are. Being educated is not at all the same as what someone ‘can do’. Defining ‘achievement’ or ‘learning’ is difficult enough, and pinning down educational ‘success’ virtually impossible, except in the sense that you know it when you see it. Many more knowledgeable people than I accept that learning is invisible. So how are we even to know what ‘what works’ means?

      The problem is that research findings are very poor at informing on-the-ground practice. Try running a marriage or friendship solely according to what research says works!

      And as for medical treatment, I might well resort to a homeopath if more ‘scientific’ means had failed, as they sometimes do – and I would almost certainly prefer to be treated by a long-experienced doctor than someone straight out of college.

      Unfortunately there seem to be many people out there who cannot understand that a successful education is more than the sum of its parts.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s