Pioneering new measurement

Measuring the outcomes of alternative learning programs has yielded extraordinary data, thanks to an Australian Research Council (ARC) project grant, writes DR RICHARD O’DONOVAN.

The outcome was a dataset of over 6,000 records: about 5,000 from non-HOL students and 1,000 from HOL participants.
And the results are amazing.

Dr Richard O’Donovan, Executive Director – Operations, Hands On Learning Australia

FOR YEARS, Hands On Learning Australia has collected data about the dramatic impact the Hands On Learning (HOL) method has on students’ attendance and retention. However, while important, attendance and retention tell us very little about the quality of student experience.
Attendance data alone cannot distinguish between a kid who is depressed and miserable but turning up regularly, and another kid whose attitude has shifted from one of suffering, to hope. Through the lens of attendance these two children look the same, but their lived experiences are incredibly different.
We needed feedback that better reflects the way HOL works with kids, that captures the kinds of impact HOL is intended to have; this is far beyond whether they simply keep turning up or not. We wanted a measurement tool that could provide data to help us tell the HOL story, to collect data on things that we know matter at least as much, or arguably even more, than mere attendance.


The ARC project is particularly significant for us as it captures the idea of HOL as an enabling space for those kids who aren’t coping with school. It made it possible for us to use the framework of Connections, Capacities, and Meanings (CCM) to develop a set of questions to put to all secondary school students to establish a baseline to then compare with the responses of kids in HOL.
For us, the CCM framework represented three overlapping ideas, and we came up with questions for each part as shown in the diagram below: