Not only did you fail to learn something; you had a horrible time trying.
ELTjam’s working definition of bad LX (learner experience)
Photo Credit: Chris Blakeley via Compfight cc
One of the things I enjoyed most about teaching was the immediacy of feedback. Learning how to harness and manage a feedback loop is one of the great takeaway skills of teaching: you create a plan, you put it into action, you observe, you adjust and tweak based on its effects, and you (hopefully) get the results you wanted. Job done.
My first unobserved class was not a massive success in this respect. I’d planned a lesson from Cutting Edge Upper Intermediate – one of those really gnarly, task-based ones that no one knew how to teach properly, especially three days off a CELTA – and it was clear within ten minutes that I was crashing and burning. But without the experience or confidence to do anything else, I soldiered on, refusing (or unable) to deviate from the plan. It was a horrible 90 minutes, for me as much as for my students. I’m surprised no one walked out.
The feelings present in class that day – frustration, some anger, confusion, boredom, repetition – are all hallmarks of bad user experience (UX). Actually, in this context, let’s call it something else; let’s call it bad learner experience (LX). Bad LX could be defined in a number of ways, but at its most basic it’s this: not only did you fail to learn something; you had a horrible time trying.
My first week of teaching was a masterclass in delivering terrible LX. But as I grew in confidence and experience, I got better at what I did. I learned how to improvise and adapt. I loosened up a bit. I started reacting to feedback. And, importantly, I started to have fun, which meant my students did too. By Christmas, I pretty much knew what I was doing. Had I not improved, however, my DOS would have been well within her rights to fire me. I was delivering a terrible customer experience, and my almost complete lack of efficacy – not to mention the joyless atmosphere I managed to create in class – would have been totally unacceptable in the long term.
Regardless of whether you think it’s a good thing or a bad thing, more and more language learning will inevitably move online. Call it disruptive innovation, call it the neoliberal takeover in sheep’s clothing, call it the death of language teaching as we know it, whatever you call it there’s no denying that ELT is in a state of flux, and that what eventually emerges will look quite different to what we have now. And that poses a problem, because right now most digital language-learning products deliver absolutely terrible LX. In many cases, it’s frankly unacceptable. It’s the equivalent of my first week of teaching. It’s bad enough to get you fired.
As more learning begins to take place outside the classroom, any person or organisation involved in the creation of learning products has a responsibility to deliver something great, just as a teacher does every time they step into a classroom. Learning English is a high-stakes endeavour for most people, so at ELTjam we take this responsibility extremely seriously. That’s why this year we want to find answers to four big questions that we think need addressing as more ELT moves slowly but surely online. We hope that by doing so, we might even sow the seeds of a sorely needed LX revolution in digital ELT.
1 How are we going to get to grips with digital pedagogy?
‘The criteria for evaluating the worth of any language learning software must include some assessment of its fitness for purpose. That is to say, does it facilitate learning?’
–Scott Thornbury, How could SLA research inform EdTech?, ELTjam, 16 June 2014
‘There’s an independent study that shows that if you use Duolingo for 34 hours you learn the same as you would in one university semester of language learning.’
–Luis Von Ahn, founder of Duolingo, speaking to The Guardian, 27 August 2014
‘A penguin reads our letters.’
– Actual sentence users of Duolingo were asked to translate from Catalan, according to http://wtfduolingo.tumblr.com/
Photo Credit: Baer Tierkel via Compfight cc
It seems no one is capable of discussing Duolingo without referencing random animal sentences – it’s a meme for language learning in the digital age. And, let’s be honest, most of us feel at some level that no one could ever truly learn a language using it. But how many of us could explain why? The jokes around Duolingo’s pet-centric content hide a widespread lack of understanding within ELT about the pedagogy of digital learning. Whether right or wrong, at least some form of consensus has built up about the pedagogy of language classrooms – broadly communicative, built around a grammar syllabus, a focus on skills as well as systems. That this consensus only holds for some of the myriad contexts in which English is taught is a topic for another post, but at least there’s a ‘there’ there. Digital language learning is more like the floor of the Chicago Mercantile Exchange – shout loud enough and wave your hands in the air and you can sell your method to anyone, no matter how flawed it is. We need a deeper understanding of what works and what doesn’t work in digital learning. We need to start drawing on what we know from SLA theory and applying it. And we need to have serious, difficult conversations about the role of teachers as more and more language learning moves online.
2 How are we going to solve our content problem?
‘Despite what some people claim, content is, actually, rather important when it comes to language learning.’
– Philip Kerr writing on Adaptive Learning in ELT, October 26, 2015
ELT has a content problem, and it’s not just that much of it isn’t very good. As demand for English increases, and as new technologies such as adaptive learning emerge, we need to find ways of producing vast amounts of high quality, effective, engaging, authentic learning content at a fraction of what it currently costs. ELT as a whole is entering what might be called ‘the margin wars’: as low-cost ways to learn English flood the market – whether free self-study apps or low-cost Skype lessons – the pressure at every level is to reduce costs in order to keep margins up. ELT content has always been incredibly expensive to produce, partly due to inefficiencies in the editorial and production processes, partly due to the royalties model, partly due to entrenched beliefs about what constitutes quality. But reducing the number of skilled in-house editors by freelancing more stuff out and offshoring to countries with lower labour costs isn’t the way to bring those costs down; focusing on the kind of content that will truly excite and engage learners is, as is admitting that we can and should automate the creation of some of the more mundane but necessary stuff. We need a radical reimagining of how we design content and distribute it to learners. We need to make bold moves to leverage new technology and techniques to create it. And we need to stop putting books on screen – it just isn’t going to cut it any more.
3 How are we going to improve the user experience of digital language learning?
When the features of a system, product or service combine to enable a user to achieve a goal frictionlessly, with accompanying delight.
ELTjam’s working definition of great user experience (UX)
At the classroom level, ELT can have pretty great UX. That’s because good ELT classrooms are communicative, learner-centred, social spaces, all of which combine to create a positive environment for learning a language. That’s not to say that classroom-based ELT is without its problems – see the account of my own early teaching days above! – but good teachers recognise these problems and figure out how to solve them, just as I eventually did.
Looking deeper, one of the reasons that good teachers are able to create a good user experience is because they employ, subconsciously or not, elements of user-centred design in their teaching. User-centred design is how good UX is created. At a very high level, it consists of three simple elements:
- Understand your users and their context deeply.
- Figure out what they want to achieve and why.
- Map a journey from where they are now to where they want to be, eliminating as much friction as possible.
Many teachers will recognise those three steps from their own teaching. We might not consciously carry out those exact steps in that order, but over the duration of a course, many teachers will use the first-hand knowledge they gain from their students to design something that’s tailored to their needs, expectations, interests and beliefs.
Why is it then, that the minute we create a digital ELT product, all of that goes out of the window? I’m embarrassed to say that at several points in my career, I’ve worked on products with budgets in the hundreds of thousands of pounds and timelines measured in years and not months that are released into the market without ever once being put in front of an actual user. That’s astounding. How is it that the feedback loop that so many teachers rely on in class is considered a luxury we can do without when it comes to moving that learning out of class?
We’ve talked at length in the past about what we’ve termed the Ed–Tech Disconnect – the phenomenon whereby digital products created by EdTech companies tend to be weak on content and pedagogy but strong on UX, and products made by incumbent ELT companies – usually publishers or language school chains – have the opposite problem: decent content and approach but lousy UX. If we’re going to stay competitive as more learning moves online and goes mobile, then we need to develop a deep understanding of user experience design. We need to understand more about the intersections of UX with content and pedagogy. And we need to start designing around the humans who are going to be using our products as opposed to the content that we happen to have or the platform that we happen to have built.
Which leads us to …
4 How are we going to humanise digital language learning?
Learning is primarily a social experience – a fact that is arguably even more true when a language is being learned.
– Blog reader Nikw211 commenting on ELTjam, October 23, 2015
If language learning is fundamentally a social act, then why do the majority of digital language learning products, with a few notable exceptions, focus on the individual rather than the collective, promoting independence and personalisation over collaboration and dialogue? Why is human–computer interaction given primacy over human–human interaction? Why is interactivity a byword for clicking or dragging and not communication? Why does digital learning for many people mean self-study? In an age of unbridled human communication, where social media and messaging platforms create communities measured in hundreds of millions, why are we struggling to create active language-learning communities of equal size? In order to reimagine language learning for a the digital age, we need to find ways to enable to kind of dialogic processes that characterise the best language classrooms. We need to find ways of having meaningful, non-scripted conversations. And we need to find ways to personalise learning without desocialising it.
Where might the answers lie?
How are we going to solve our content problem? How are we going to get to grips with digital pedagogy? How are we going to improve the user experience of digital language learning? How are we going to humanise it? When you look at each of those questions in isolation, they look like massive challenges. But if you take them together, if you start to look at how they intersect and overlap, they start to look like something else; they start to look like a way of creating amazing LX.
What would happen if you applied the very human-centred principles of UX design to content creation? Would it help us to deliver content that was more relevant, interesting and dynamic? What if you did the same for pedagogy? Would it help facilitate learning? What if you designed a digital learning product around human–human interaction and not around human–content interaction? Would it help learners to co-construct the knowledge and skills they needed? Would it lead to more meaningful interactivity?
Content, pedagogy, UX and human interaction – if we combined those four things, would they become greater than the sum of their parts, and would that make for better language learning in the digital age? We think it would.
At ELTjam, we’re constantly trying to figure out a way of improving the quality of digital ELT products – to find a better methodology for product development that addresses the complexity of language education. We think one way would be to change how those products are conceptualised and built. We’re calling that new way Learner Experience Design (LXD) for ELT. We’ve been running LXD training sessions in London, and plan to continue doing so. We’re also developing a Learner Experience Design online course, which is designed to help ELT professionals get to grips with the LX concept and, more importantly, to learn how to put them into practice. Find out more about that here.
Nice article, but how does this new-found love of human interaction and pedagogy square with eltjam’s often glowing reviews of edtech products which feature neither? How does one flirt with the analytics bods and espouse human interaction? Where does Flovoco fit into all this? And the new product you’re working on with Knewton?
I can’t be the only one who is confused, surely? Is this some kind of marketing gimmick which you don’t actually believe in?
These are genuine questions, by the way – I’m open to persuasion…
John
Hi John,
Many thanks for the comment, and the very valid questions. My own personal interest in the importance of the more human elements of digital learning have developed recently out of conversations that arose off the back of my ‘Content is no longer king’ post last year. I was particularly struck by the comments from Nik and Scott on this topic (hence quoting Nik above). In fact, I was lucky enough to hear Lindsay Clandfield speaking on the topic of interactivity at the IH Barcelona conference this last weekend, and he had some very interesting things to say about the hijacking of the term by EdTech and the need to focus on what he called ‘strong’ interaction (i.e. the more human kind) rather than ‘weak’ interaction (the more human–machine kind).
With regards to Flovoco, I would love to see more social elements being introduced to the product, and looking into that is certainly a long-term goal. Do bear in mind that Flovoco is still at a very early stage of its development, and there are other elements that we need to focus on first. For example, I think that while we’ve built a product with very good UX and a sound pedagogical underpinning, we need to do some more work on the content and the social elements. That’s one reason why I think that the LX framework is quite useful for developing product roadmaps.
And finally, to address your point about Knewton. How incredible would it be to find a way to harness data and analytics like that without sacrificing the social aspects of learning. I’m not saying that it’d be easy (or possible), but it’s certainly something to shoot for, I think.
Where might the answers lie? I think one clue is in a simple contrast: you used the word ‘product(s)’ 13 times in this post, whereas you only used the word ‘process(es)’ once (in the phrase ‘dialogic processes’). The emphasis on ‘product’ over ‘process’ in language teaching, in my experience, has led over the decades to a lot of minimally effective pedagogy.
The most obvious example to me is in the ‘comprehension approach’ to teaching listening: it focuses on students producing answers to comprehension questions and provides little or no help to students in the various processes which lead to comprehension. As such, it is largely an unrewarding experience for all concerned. Suggestions for alternative approaches have existed in the TESOL literature for a long time but, until more recently perhaps, haven’t been taken up widely.
As long as you and other designers agonise over ‘improving products’, ‘designing experiences’, ‘creating content’, etc., you and your users/learners are likely to be disappointed with the outcome. You’re eager ‘to find ways to enable to kind of dialogic processes that characterise the best language classrooms’ but are you over-complicating things? What if the technology to do this already exists, is owned by Microsoft, and is called Skype?
Regarding ‘content’, it’s not actually clear to me what you mean by it. In your ‘Content is no longer king post’, you wrote: “A huge amount of what we’ve traditionally called ‘content’ in learning is actually instructional design: the pacing and variety of activities, the flow, the challenge of zeroing in on the best and most enjoyable way to help someone master a particular skill.” Is that what you mean by ‘content’ in this post? It doesn’t come across that way to me but if it is, you and your fellow travelers around the world are at risk of chasing a ‘disruptive innovation’ dream, putting in a lot of blood, sweat, tears and money to bring to market an over-engineered white elephant.
Whatever you mean by it, “producing vast amounts of high quality, effective, engaging, authentic learning content at a fraction of what it currently costs” seems like a fantasy to me and one which appears at odds with your desire to ‘humanise digital language learning’. However, there is one way to achieve it: driving wages down significantly. The more complex the work, though, the lower the wages will have to be in order to meet your productivity imperative. The good news is that there are a lot of very wealthy and powerful technocrats, corporations, governments, and think tanks working around the clock as we speak to make that a reality!
Thanks for the great comment, Kyle. (By the way, for any ELTjam readers who don’t follow Kyle’s blog (https://pedagogablog.wordpress.com/) or his Twitter account (@ElkySmith), I’d highly recommend them; it’s some of the best writing on EdTech and its associated problems out there right now.)
Excellent thoughts on the ‘product’ vs. ‘process’ distinction, and we’re in complete agreement over the comprehension approach to listening in ELT!
Again, no disagreement! It’s clear that Skype’s potentially an amazing tool for online language teaching, but the existence of the tool itself isn’t enough, as I’m sure you’d agree; it’s how that tool fits into a wider process for language education that I’m interested in.
Regarding the productivity question, I find this a very interesting area at the moment (I noticed you raise this topic on Twitter the other day, and it actually got me thinking a lot.) Much of the current rhetoric around productivity and the automation of labour is highly problematic, and this is clearly going to be a massive issue in education over the next few years. The question I’m interested in is how we do we handle this in ELT. For example, right now jobs are being lost in ELT publishing, and it’s not because of automation; it’s because it’s believed that the same quality of work can be produced for lower cost elsewhere. What I wonder is whether it’s the nature of the work that we should examine. There is clearly huge creative endeavour required in the creation of good learning materials, but that’s not where most of the effort goes (I say this after spending over ten years in ELT materials development). You can’t automate creativity, thankfully, but I wonder what we could automate to allow more time and effort to be spent on creative work in materials creation. These are the kinds of questions we’re interested in right now.
Thanks for your response, Nick, and for the very kind words – I appreciate both!
I agree that the existence of a specific technology isn’t enough. However, I’d argue that we currently have all the understanding and basic underlying technology necessary to achieve the vision of ‘digital learning’ you’ve sketched here; if that’s true, we have to ask: why does that vision still seem so far off? This question has plagued futurists and technologists for centuries.
Two articles I’ve read today illustrate this: ‘President Obama Discovers Coding – Yippee!’ by Gary Stager (http://stager.tv/blog/?p=3596) and ‘The STEM Crisis is a Myth’ by Robert N. Charette (http://spectrum.ieee.org/at-work/education/the-stem-crisis-is-a-myth). Both of them were linked to by Audrey Watters in an excellent post last week, which helped crystallise a few things for me (which I think you witnessed on Twitter!): http://hackeducation.com/2016/02/05/computer-science-for-lol
Stager writes that “we continuously invent that which already exists, each time with diminished expectations” and that the “first thing we need to do is find a cure for amnesia” and that those “armed with “solutions” or prescriptions for “reforming” education do not lack for chutzpah. A sense of perspective and awareness of history are their greatest deficits.” The Charette article includes a collection of quotes from 1934 to 2013 all expressing panic about an impending STEM skills gap for which there is no real evidence.
I think this ‘amnesia’ is, to a large extent, selective and willful. Being honest about technology – who develops it, how and for what purposes – means bringing to the surface a whole lot of ‘messy’ social, economic and political realities that few people really want to talk about much, let alone try to address. Technology has always been and always will be primarily a means of increasing productivity – getting more output from less input or, as Larry Cuban puts it (https://larrycuban.wordpress.com/2016/02/05/musings-about-technology-in-work-and-life/), ‘doing more with less’.
Bringing this back to the question of why a utopian vision of digital learning hasn’t yet arrived, I think the answer may be that, for those with the resources to see it realised, the “object is always to satisfy needs with the greatest possible flow of commodities, to produce these with the techniques which permit of the greatest profit and, lastly, to accord prime importance to those needs which are most profitably satisfied” (Andre Gorz, 2013, Capitalism, Socialism, Ecology, Chapter 1, section 6). And, because paying wages to human workers is so expensive, anyone who aims to make a profit out of education (whether online or face-to-face) or ‘do more with less’ has little choice in the long term but to take the path of least humanism.
Hence, the current intensification of an old obsession with AI. Why else would IBM, Pearson, Google, Knewton, Uber, Apple, Facebook, Microsoft, etc. be buying up every bit of AI expertise and kit they can get their hands on if they weren’t in relentless pursuit of ever greater productivity (commonly referred to as ‘scalability’ or ‘economy of scale’)?
I’ve already taken up too much space on this page so I’ll end now. Thanks once again for the dialogue!
I think this is a really interesting post and how to come up with the perfect digital language learning resource is really the holy grail we’re all in search of, as educators in all kinds of contexts these days. There are many other tangential questions that spring to mind, but to stay focused on Nick’s post, I have some brief comments/questions below:
1. You talk about responding to feedback in the classroom, in other words, teaching and learning as a process, not a product. How can you build the process element into your digital product? What is the product? The learning gained by the user? The package you sell?
2. Connectivism as a methodological approach could provide the learner with the real world communicative practice/goal that is so rewarding and motivating.
3. How could a designer package that? It needs to be more than a concept, it can’t be just a set of promises- you need to have that network as part of your product, integral to the product, but it needs to be dynamic.
4. Somehow, that dynamism needs to be built into the product too.
Thanks, Denise! I’ll do my best to answer your questions, although it’s more likely I’ll provide unformed musings at this stage! Let’s to reflect on.
It’s interesting that this is the second time the ‘product’ vs. ‘process’ distinction has come up (see Kyle’s comment above). ‘Product’ is the generic term used in most digital ‘product’ development these days, even if the product is actually a service or process. So Facebook would describe itself as a product, even though it provides a service (or doesn’t, depending on your point of view!). So a learning ‘product’ could absolutely be the facilitation of a process. Hmm, I’m confusing even myself …
I think the question ‘What is the product?’ is a highly pertinent one, though, and one not always given enough through while ideas are being conceptualised. For me, a product is something that helps you do something you want or need to do. Not exactly a sophisticated definition, but one that works, I think.
I was actually just looking into connectivism the other day in the context of MOOCs, as I hadn’t realised that a distinction is made between cMOOCs (which are based around connectivist principles) and xMOOCs (which are what most of us think of as MOOCS – Coursera, etc.). I’m going to be looking into this in more depth.
Yes, absolutely agree! The ‘how’ is the tricky bit …
Yes, again agree.
I’m really greateful for this article and what is hopefully to become a fruitful debate on the ELT content.
Since these are my initial thoughts freshly after reading the article, I will limit myself to just one contribution, or to what I think is of critical importance in order to bring all 4 particles of the Amazing LX: Content, Human Interaction, Pedagogy and UX a bit closer.
My point is: questions. The quality of questions students receive. I am not going to elaborate on their importance, that it is questions that actually spark interaction, put learners first etc. It’s plain to see.
Or is it? Why hasn’t anyone in the industry recognized their role and importance? That they are to a large extent responsible for the quality of education people receive? What we are dealing right now with are questions of surprisingly low quality – mostly introspective and uncreative. I see a great room for improving the general LX in providing any learning platform with a set of great questions to think, and – possibly – talk about (though talking costs :).
We – teachers – may be wiped out by the technology, but our souls can still survive in the form of questions. After all a good educative question is already a teacher. If we are to lose, let’s at least lose against something of superior content, not wimpy turtles sending letters.
Great comment regarding questions. Making room for a wide variety of questions is always a wise strategy.
Yes, agree – thanks for taking the time, Jedrek.
I would take issue with your graphic above. I believe that AMAZING LX is the intersection of good UX, research supported pedagogy, highly engaging content, and rule governed, but free spirited interaction. Amazing LX goes beyond checking off boxes on a list (as the graphic suggests). The human heart and spirit must play a key part as well. In the end it is the marriage of both adjectives and nouns that determines quality.
Thanks, Mike! I very much concur (and I like the flourish that you’ve added to the necessarily curt wording in the diagram). And I don’t in any way see the diagram as a box-ticking exercise; rather, what it brings to mind for me is four lenses, each of which might be adjusted depending on what you’re trying (or have failed) to achieve. I’ll be talking through that idea a little more during the webinar; glad to see that you can join us.
A thought-provoking blog post, Nick, and the first one I’ve read on eltjam.
In reading the comments, Kyle’s mention of Skype jumped out at me. I’m currently thinking about how to re-design an outdated online course, and I keep coming back VoiP as the only way to effectively interact with students and for them to interact with one another. Surely, anything else is just a cop out.
Thanks for taking the time to comment, Craig, and very glad that you’ve discovered the site; I hope that you find it useful.
If you’re interested in finding out some more about possible option for synchronous interaction on your course, you might want to check out a series of recent posts from Jaime Miller about teaching online. In the first, here, she looks at platforms.
Very interesting/useful. It’s really hard to understand where the lines lie nowadays. You put a UX designer, an ELT/SLA expert and an instructional designer together and there’s suddenly a lot of overlap between their areas of expertise. Getting to an end product which takes into account good practices from all three disciplines is a bit of holy grail right now, so I’m very interested in what your methodology is going to look like 🙂 .
Cheers for commenting, Jonny! Glad you’ll be coming along. I’ll try not to disappoint …