Tracking rich learning experiences – where the LMS falls short
The way learning activity is recorded by learning management systems is about to undergo a shake up. This shake up is long overdue. If you use a learning management system (LMS) as either a student, a teacher, an administrator, or even as an LMS provider, you no doubt already have some misgivings about the way in which learning activity is tracked by learner management systems: your LMS is probably too rigid to track all the results at the level of granularity you want; it is likely to be a painful process to conform your learning content to work with the way outcomes are tracked; and almost definitely your LMS is unable to track informal or non-curricular learning activity at all. You are not alone in your dissatisfaction. Fortunately, inroads are being made into redefining the way in learning experiences can be tracked and recorded.
S.C.O.R.M.
At the heart of the matter lies the S.C.O.R.M. standard. S.C.O.R.M. has been at the center of LMS and e-learning technology for nearly two decades. It is a set of standards designed to record training results for modular online learning, originally created for the US Department of Defense Office with subsequent revisions. It was widely, but not universally, embraced as a solution for recording learner results and activity within an LMS. It has served its purpose but there is mounting pressure to overhaul the specification due to its limitations.
S.C.O.R.M. is very hierarchical and assumes a sequential learning experience. You take a unit or assessment, then another, then another and so on. There is little room to record what happens elsewhere outside the LMS, either in the classroom or in what else the student does with the web browser or on their smartphone and so on. Put simply, S.C.O.R.M. lacks the flexibility to encompass a rich range of learning experiences.
Tin Can API – a successor to S.C.O.R.M.
There are a number of proposed means for overcoming the limitations of S.C.O.R.M. I want to highlight just one here as an example of a pending shift in approach. It’s called the Tin Can API. The Tin Can API offers capabilities similar to S.C.O.R.M. but with a greatly expanded capacity to record all kinds of learning activity and interaction. Consider some of the activities that might compose an ELT course. The learning experience might consist of some of the following:
- classroom task-based learning activities or project tasks
- watching Youtube videos
- playing language learning games on a smartphone
- web quests
- online workbooks
- in-class and online assessment
- blogging, video-blogging, preparing podcasts
- wide reading tasks / intensive reading
The limitations of S.C.O.R.M.
At present under the S.C.O.R.M. standards, an LMS would only meaningfully track formal learning activities, i.e. the results of online workbook study, assessment results etc. While those are important for tracking student progress, they give only a limited picture of the student’s engagement. Implementation of the same capabilities under S.C.O.R.M., on the other hand, while theoretically possible, is tricky and requires a lot of planning and custom programming. Moreover, the custom programming would set out to do what S.C.O.R.M. is not really designed to do.
How Tin Can could help
Tin Can, in comparison to S.C.O.R.M., is able to record all of the activities in the list above and is still able to pass the data over to not only an LMS, but also a social media service, a smartphone app, or any number of platforms or devices which are able parse commonly used web programming languages. Even without a very technical discussion on how the Tin Can API is implemented, we can appreciate how differently it attempts to capture learning experiences. Most of us are familiar with the kinds of statement that Facebook generates about our interaction with the Facebook site:
Aaron commented on Brigit’s album.
Carol likes Duncan’s link.
Ellen and Fiona are now friends.
Gary was at Heathrow Airport.
This is also the method that the Tin Can API, along with many other software services, uses to describe user interactions. The linguists among you will quickly recognize a simple grammatical pattern to these statements: actor + verb + object. Despite its apparent simplicity, the statement formula allows a surprisingly wide range of formal and informal learning activities to be described. Consider our list of ELT activities above. Translating it into quasi “Tin Can speak” would yield:
Aaron and Brigit made a Youtube video.
Carol watched Fast Food Nation.
Duncan read The Great Gatsby Chapter 1.
Ellen added a new blog entry.
Fiona scored 98% on the vocabulary test.
Gary reached level 12 playing Grammar Goblins (on his iPhone).
Helen completed Online Workbook 3.
And so on. The jargon for these kinds of lists is “Activity Streams”. Activity streams in a learning context would be equivalent to a rich learning profile. If you are interested in seeing a practical implementation of the Tin Can API that uses Activity Streams, take a look at the free app, Tappestry. It allows users to record a extremely wide variety of learning activities and share those results in a social way.
If a learning activity can be described, then it can be recorded and stored. Structured reporting queries can then mine the records to provide wide or narrow windows into the data. A traditional LMS, for example, might only need to report formal learning outcomes but a teacher might want to see a much wider picture of students’ informal learning activity. This offers much greater flexibility than S.C.O.R.M. which limits tracking of activity to what is a fairly basic hierarchical model of ‘completion / progress’. At the same time, Tin Can is envisaged as being backwards compatible, meaning that the way it captures data can be retranslated back into a format that S.C.O.R.M. can interpret. This is clearly good news for institutions that are potentially worried about their expensive legacy S.C.O.R.M.-based LMS.
So, what does this mean for learning?
The decoupling and liberation of learning reporting systems from the ‘assessment — result’ formula will arguably have some interesting side effects on pedagogy. Will the increasingly sophisticated grammar that software uses to describe rich learning experiences become a functional driver for leading administrative tools such as LMS (and therefore hopefully course administrators) away from a heavy emphasis on formal learning outcomes? Will the informal learning experiences and communicative interactions, both online and offline, that we, as either language learners or teachers, know to be just as crucial to language acquisition as formal exam-oriented learning, be able to take “center stage” in curriculum design, assessment and delivery? The future seems to be highly oriented towards building “learning profiles” summarizing all a learner’s key learning experiences, which can then be shared across a wide variety of platforms and track across a learner’s experience from institution to institution, even into professional life.
Benefits for research into language acquisition
Looking even further ahead, the ability to capture rich learning activity will create an invaluable resource for researchers. We know there is value in data, especially ‘big’ data. Google and Facebook have decisively demonstrated this in commercial terms. We are already seeing the sharp shift towards big data analysis in other areas such as hard sciences, political sciences and economics. We can expect similar shifts to take place in other, more specialized sectors such as education. The capture of data is functionally independent in most cases from its analysis and the real value of capturing a rich range of data on learning activity might only be realized further down the track when researchers start to correlate it to learning processes. The shift towards capturing activity streams, for example, might in the future enable analysts to cast light onto the role of informal learning activity in language acquisition backed by a larger statistical data set drawn from all over the planet. Although privacy remains a constant concern, if we collectively get our data policy right to preserve anonymity and protect privacy but share essential data, there is the potential for real advances to be made in quantitative and qualitative research findings.
Where next?
It is very early days for new the new breed of e-learning solutions such as the Tin Can API. One of the main challenges new standards will face is achieving sufficient uptake within e-learning solutions so as to be able to replace older, entrenched standards such as S.C.O.R.M. To be fair to the S.C.O.R.M. standards, two decades ago no one could have reasonably envisaged the future reality of smartphones, web and mobile platform integration, or the sophistication of interactive experiences the web is currently able to deliver. That reality is here, however, and e-learning technology has very little choice other than try desperately to keep pace with it. The genie is out of the can, so to speak.
To learn more about how SCORM and Tin Can developed, read Sean’s 2015 update.
LSMs are dead whatever the norm is. To succeed don’t focus on technology or noms. Focus instead on content and usages and put the teacher at the center. Never Forget: KISS (Keep it simple and stupid) for your users. LMSs are just the opposite of that!
Thank for this Sean, it’s fascinating reading and is obviously the way to go. For teachers monitoring student activity, this is so much more meaningful than what registers in Scorm compliant courses. The challenge will be presenting this sort of information in a useful, manageable and meaningful way to training managers who (if we’re honest) are just interested in ROI. How can this much more useful information be converted into some sort of format which will satisfy the entities that pay for the courses? This, I think, will be the challenge!
Thanks, Fiona. I think that is a critical point. I anticipate that if methods such as TinCan start to be more widely adopted, there will be a transitional period where the LMS reports will continue to mine out from the TinCan records only the traditional data about test scores, student progress rates through the course material etc. However, I hope eventually it will be hard for learning institutions to ignore the possibilities that activity streams open up for more sophisticated monitoring and evaluating of learning progress. I suspect that driving adoption into the mainstream will probably take some high profile, research-driven projects that highlight the alternative forms metrics could take and link them back decisively to student performance. If it could be demonstrated, for example, that monitoring a broader spectrum of learning activity results in greater student satisfaction, better teaching performance and improved learning outcomes, then this might start the ball rolling on convincing decision-makers that the often significant financial investment needed to upgrade legacy systems is worthwhile.