A document from the Joint Information Systems Committee (JISC) was recently brought to my attention. The author is Niall Sclater and it concerns codes of practice for learning analytics, largely with reference to the tertiary sector. However, there is one section that I believe may be of interest to the school teaching community, and as it is licensed under Creative Commons Attribution 4.0 I offer a few sections from it here. (It can be found on pages 26 and 27 of the original). I will leave most conclusions to readers; suffice it to say that I feel it supports some of my recurrent arguments about the severe shortcomings of use of statistics to inform the educational process.
The original document can be downloaded here:
“A working party at Charles Sturt University (2014) notes that “learning is a complex social activity and that technical methods do not fully capture the scope and nuanced nature of learning”. Reducing the complexity of student behaviour to a number or a traffic light is pointed out by Campbell et al (2007) to result potentially in oversimplified or insensitive conclusions. Any algorithm or method will be reductive in that it attempts to create a manageable set of metrics which do not necessarily reflect reality (Greller & Draschler 2012). No prediction can take into consideration all possible factors such as problems at home or financial difficulties…”
“Slade & Prinsloo (2013) point out that as much data related to learning is held in systems outside the control of the institution…it is impossible to obtain a holistic picture of student life. Moreover the data is itself temporal and may only afford a view of an individual at a specific place and time, not allowing for the changing and multiple identities of learners as they progress through their studies.”
“A number cannot represent the personal growth or development of relationships that arise from attending an educational establishment. Johnson (2014) worries that data mining can treat a subject as a collection of attributes rather than an individual. He discusses course recommendation systems where people are recommended to do what people like them have done before…For Johnson it appears that learners are being thought of as mere collections of skills to be matched to an outcome, rather than individuals. He thinks that such systems undermine student autonomy…denying students the opportunity to make their own decisions.”
“Tertiary education can be an “individual artisanal craft” where the standardised metrics and interventions of learning analytics ay not fit easily (Contact North 2012)