Geoff Jordan vs. Duolingo

More than a million people a day connect to Duolingo, an app which claims to help people learn a foreign language by guiding them through a series of translation exercises. Here are a few examples:

duo4

duo2

duo3

duo1

The program is an artful way of getting people to translate texts for free. Clients upload texts for translation to Duolingo and the text is then presented to students who translate it as part of their study activities. When the document is fully translated, Duolingo returns it to the client who pays for the translation. Very clever, but is this a good way to learn a foreign language? First, let’s look at the clamorous case against Duolingo.

The Case Against Duolingo: 1. Tutorial CALL is OUT

Computer Assisted Language Learning (CALL) theorists make a distinction between ‘computers as tutors’ (the computer controls and evaluates the learning process) and ‘computers as tools’ (the computer assists teachers and learners). The Rosetta Stone is tutorial CALL, while concordancers, web browsers and email programs exemplify the tool role. Duolingo obviously takes a tutorial role and thus offends against every tenet of contemporary communicative language teaching.

Even though it’s a thoroughly modern smart phone app, it’s seen by angry hordes of Communicative Language Teaching (CLT) adherents as a throwback to ‘drill and kill’, and an affront to modern educational principles. It is, they say, based on a discredited behaviouristic theory of learning where positive reinforcement through repetition ignores the fact that language learning is an exploratory cognitive process of discovery. It confounds the distinction between explicit and implicit language knowledge, offending even the strong interface position.

It contradicts convincing SLA research evidence that the route of interlanguage development is impervious to the presentation and practice of pre-determined sequences of discrete-point grammar and vocabulary items. It relies on decontextualised, impoverished input and allows only unnatural and degenerate output. It provides no opportunity for the negotiation of meaning. Wolff (1999, p.126) speaks for most CALL experts when he says:

On the basis of our present knowledge of language learning, I believe that the only defensible role for the computer in language learning is that of tool. In learner autonomy, the key concept in modern language learning and in all humanistically oriented pedagogy, the computer as tutor cannot have a place.

2. Translation is OUT

Not only does Duolingo adopt the discredited tutorial role, but its prime vehicle for learning is translation, a pedagogical method confined to the dustbin of history by most contemporary ELT methodologists. The usual objections to using translation are that it encourages a sense of false equivalence between the two languages, and that it impedes automatic and fluent language use. As Guy Cook (2006) says, translation is so out of fashion in ELT that it is rarely discussed, either as a means or an end.

It is simply assumed to be wrong, and has attracted all the usual insults. It is boring, artificial, the last refuge of the incompetent teacher. It has been regularly and explicitly forbidden in materials, curricula and instructions to teachers – sometimes even upon pain of dismissal.

Just for good measure, Krashen (2011) is said to have dismantled the claims made by Vesselinov and Grego (2010) in their glowing ‘Effectiveness Study’ of Duolingo.

The Case for Duolingo. 1. Tutorial CALL ≠ Behaviourism

But now let’s look at the other side of the story. To begin with, in response to the criticism that tutorial CALL is behaviourist, some SLA research (see Lightbown & Spada, 1999, for a good summary) indicates that tutorial programs can help in the acquisition of vocabulary and grammatical morphemes, in phonological development, and in the acquisition of idioms and fixed phrases. More significantly, many tutorial programs adhere closely to the cognitive/constructivist perspective outlined in Warschauer and Kern (2000). Duolingo involves the conscious manipulation of language forms based on abstract rules, the antithesis of behaviourism; it includes reading and listening comprehension and it provides meaning aids such as the ‘Discuss Sentence’ feature, glossed words and transcripts, which are forms of meaning negotiation.

2. What’s wrong with Drills?

Furthermore, describing Duolingo as ‘drill and kill’ is unfair. The practice offered involves the conscious, reflective manipulation of language rules and forms, designed to increase linguistic knowledge, accuracy, and fluency. The translation exercises, many directly or indirectly involving the application of an abstract rule or rules, constitute practice but not drill. And anyway, what’s so wrong with repetition and rote learning? They’re artificial. So what? They’re boring and purposeless. Really? Then how come we all take delight in repeating rhymes, jokes and songs a thousand times? As Guy Cook (2006) says ‘In a wide range of genres (from prayers, plays and poems, to sports chants and jokes) repetition and rote learning typify the most enjoyable and valued human language activities.’ More empirically, SLA research shows that repetition and rote learning give learners very positive feelings of progress and that they are effective learning strategies which help turn input into intake.

3. Learners Have Some Control

Thirdly, the criticism that the computer is always in charge in tutorial CALL cannot be levelled at Duolingo, where learners have considerable control over the theme for an activity, and within that theme, various activity types. They can exercise various options such as skipping items, using help screens, discussing correct and incorrect answers with other learners and varying the speed of audio recordings. Generally, the program allows learners to influence the sequence in which they do certain activities and the manner in which they do them.

Most CALL theorists agree with Hubbard and Bradin (2004) that the most promising uses for tutorial CALL are:

  1. Acquiring conscious knowledge of the L2 (language forms, patterns, and collocations).
  2. Improving reading and listening skills. Listening programs are particularly successful. They give instantaneous control of pause and repeat/rewind to aid comprehension; provide meaning aids including slowed versions, text and glossary support, and translations; and add visual support through static or animated graphics, photos, or video.
  3. Improving pronunciation. Modern programs provide multimedia models with audio, technical description and graphic visualisation of sounds including waveforms, pitch contours and spectrograms. Speech recognition engines, used to grade the accuracy of the pronunciation relative to a given norm, are gaining in sophistication.

These three uses gel perfectly with the way Duolingo can best work. While presently concentrating on the first use, there’s nothing to stop it more fully incorporating the second and third.

4. Translation is IN

As for Duolingo’s use of translation, I recommend Guy Cook’s (2010) book Translation in Language Teaching, which does a brilliant job of reconsidering the place of translation in ELT. Cook points out that translating is a needed skill: ‘real-world’ foreign language use is full of translation. Both socially and at work, successful language learners need to mediate between speakers of their own language and speakers of the language they have learned. In short, there is absolutely nothing artificial about it. As a teaching tool for both grammar, where it compels students to use structures that might otherwise be avoided, and for vocabulary, it has enormous potential and is particularly suited to the realm of tutorial CALL.

While there are inevitably a few questionable translations among the texts, there is no evidence that Duolingo’s translations seek to preserve the grammatical construction of the first language or that they’re misled by ‘false friends’. As for the claim that translation acts as an impediment to automaticity, Cook (2006) comments ‘this is just assertion without evidence – belied by the millions of successful language learners who, having begun the study of a language through translation, go on to become fluent speakers without recourse to an interlingual equivalent’.

Cook concludes ‘The outlawing of translation not only reflects the monolingual mind-set of the English-speaking world, it has also – in a more sinister way – been to its political and commercial advantage. Monolingual native-speaker teachers have been privileged, and the status of ‘local’ experts undermined… Publishers have marketed monolingual textbooks globally. The classroom has become, in microcosm, a world where English is the only language, rather than one integrated with others.’ Amen to that.

 

5. The Study is OK

Finally, while Krashen (2011) certainly highlights weaknesses in the data presented in Vesselinov and Grego’s (2010) report, the basic claims made by the report seem completely credible to me and I wouldn’t be surprised to see replication studies support their findings.

Conclusion

So, in my opinion, Duolingo works well as a supplementary way of learning vocabulary and grammar. There’s lots of room for improvement in both content and exercise type, and I think more use should be made of longer texts where the pedagogical value of translation and dictation can be used to maximum effect.

What is especially good is the ‘Discuss Sentence’ feature, where learners discuss answers with other learners. In my first Duolingo session I had to listen to a sentence and type what I heard: ‘Mañana no es jueves.’ I was told that my answer was correct and invited to join the discussion of this answer, which already had 18 contributions. Somebody wrote ‘Shouldn’t jueves be plural? If not what is the plural form?’ and somebody else wrote a page long answer which was absolutely marvellous – a million times better than anything Murphy’s Spanish cousin might have written. This kind of collaborative learning makes my cynical old heart sing!

 

For more ELTjam posts on Duolingo here are some of our initial thoughts, when Duolingo  won Apple’s top app of 2013 , and on its voice recognition capabilities circa 2013.

References

Cook, G. (2006) Breaking Taboos. In Sökmen, A. (2006) Readings in Methodology. (Accessed 12/01/15)

Cook, G. (2010) Translation in Language Teaching. OUP.

Hubbard, P. and Bradin, C. (2004) Another look at tutorial CALL. ReCALL, Vol.16, Issue 2, pp448 – 461.

Krashen, S. (2011) Does Duolingo “Trump” University-Level Language Learning? (Accessed 12/01/15)

Lightbown, P. and Spada, N. (1999) How Languages are Learned: Revised edition. Oxford: Oxford University Press.

Vesselinov, R. and Grego, J. (2010) Duolingo Effectiveness Study: Final Report. City University of New York, USA.

Warschauer, M. and Kern, R. (eds.) (2000) Network-Based Language Teaching: Concepts and Practice. New York: Cambridge University Press.

Wolff, D. (1999) Review of ‘Levy, M., CALL: Context and Conceptualization.’ System, 27:119–132.

Geoff Jordan, Ph.D, is semi-retired. He works from home as an associate tutor in the Distance Learning MA in Applied Linguistics and TESOL programme at Leicester University. He has numerous academic publications to his name and writes from his own blog.

24 thoughts on “Geoff Jordan vs. Duolingo”

  1. What is the evidence, if it exists, of Duolingo improving speaking skills? It sounds like a great way to learn to read and write. It also occurred to me that its popularity could in some part be attributed to its compatibility with how most languages are still taught. Read and write first. Create grammatically accurate sentences. So, I’d be interested to know if extended use of Duolingo improved speaking or, possibly, impaired it.

    Reply
    • Hi Jason,

      The Vesselinov and Grego (2010) Duolingo study used the Spanish WebCAPE Language Placement Exam, which as its name suggests is a “Computerized Adaptive Placement Exam”. It consists of multiple-choice questions and has no spoken component, so there’s no data on speaking skills. But that’s fine since, in its present format, Duolinguo doesn’t offer oral fluency practice; rightly so, IMHO, because despite numerous attempts over the years, tutorial CALL hasn’t done anything very good to help in teaching conversation. While computers can help in certain limited aspects of speaking (such as improving pronunciation by providing model bits of oral text which can be recorded by the user and compared to the original), until a computer passes the Turing test, I think we humans are right to feel that computers just don’t make good conversational partners.

      Reply
  2. Hi Thomas

    Thanks for your kind comment.

    As to whether rote learning pushes explicit or implicit language knowledge, since it’s a conscious activity, I’d say it mostly addresses explicit knowledge, particularly when it’s used to practice grammar, evidently. There’s the subsequent question of whether explicit language knowledge can become implicit, and, Krashen notwithstanding, I think it can.

    Reply
  3. Hi Geoff
    I think that things have changed since 2006 when Guy Cook wrote that ‘translation is so out of fashion in ELT that it is rarely discussed, either as a means or an end’. But even then, I think that Guy overstated his case. A more nuanced view would be that, as far as the US / UK based world of ELT journals, magazines, etc. was concerned, it was an elephant in the room. This was much less the case in the journals and teacher magazines in other parts of the world (the Gulf, Iran, China, for example). Closer to your home and mine, Wolfgang Butzkamm in Germany had been writing on the subject since the 1970s. In Spain, Maria Gonzalez Davies published ‘Multiple Voices in the Translation Classroom’ in 2004. This body of work was influential and talked about in many local contexts. It didn’t, however, penetrate far into the ‘core’ world of UK / US-based ELT until Guy started publishing on the subject. There were probably two reasons for this. First, some of this research was not written in English. Second, the research was mostly concerned with primary and secondary state school contexts, and not the adult contexts which UK / US – based ELT usually takes as a default position.

    Even if we do restrict our scope to UK-centred stuff, translation or own-language use was not ‘a pedagogical method confined to the dustbin of history by most contemporary ELT methodologists’. True, a majority of methodologists didn’t write about it, but there were plenty who did (Bolitho, Prodromou, Maley, Rinvolucri, Widdowson, Ellis, V. Cook, Nation, Medgyes, Stern, Edge, Rivers, Jane Willis to name a few). Those who didn’t write about it did not, however, criticise it. It’s very, very hard to find anyone confining this to the dustbin of methodological history. And when pushed, everyone said, yes, nothing wrong with it at all in certain contexts for certain purposes. Around the time of the publication of Guy’s book (which was now about 5 years ago), many others made their positions more public. Since then, I have been able to find only one person arguing for the banning of translation / own-language, but I won’t embarrass him by naming names.
    This brings me, finally, to Duolingo. The problem with this app is not that it uses translation: it’s how it uses translation. You say, with considerable generosity, that ‘there are inevitably a few questionable translations among the texts’. My view is that there are a large number of questionable translations, and an equally large number of translations that are simply wrong. The number and seriousness of the errors depend on the particular combination of target language and own language. English – Spanish doesn’t work too badly, but I’d still say it was unacceptable. Some other language combinations are far, far worse. On a regular basis, perfectly acceptable translations are marked as wrong, the ‘corrections’ of perfectly acceptable translations are themselves wrong, and incorrect translations are marked as right. How bad does something have to be before we consign it to the dustbin of crap apps?
    Duolingo gets its translations wrong because it’s based on statistical correlations between parallel texts, aided (and sometimes the opposite!) by crowd-sourced ‘augmentations’. Among many other problems, it struggles to disambiguate homonyms (which are often high-frequency), to understand phrasal verbs (especially when the particle is not adjacent to the verb), and it has problems with lower -frequency word senses of high frequency words. The software is powerful, but relatively crude. Because sentences are presented in isolation, it can’t use surrounding textual information to help the translation process. It needs more sophisticated parsing tools, more sophisticated AI and it needs NLP. I’ve no doubt that they are, or soon will be, investing in this technology and, when they do, they might get their accuracy rate up to a more acceptable 95%. But they won’t get there if they stick to decontextualised sentences. They need longer texts, not just because of the reasons you give, but also because better automated translation requires them. They won’t get there, either, unless they have decent bilingualised dictionaries in their database.
    Perhaps, though, they don’t care. They’re not doing so badly as it is in terms of users and, at the lowest levels, where most of their learners are, the accuracy is higher. It’s not so bad as a supplementary tool for learning vocabulary and grammar at A1 and A2 levels with some language combinations, but at any higher levels and with other language combinations, I think it’s very poor.
    Cheers, Philip

    Reply
  4. Hi – really interesting article. However I’m curious about the assertion that Duolingo makes its money by getting students to translate texts bit by bit that have been provided by clients. Surely when they’re inputting the app content, the creators have to also input the correct translation so it can be marked as right or wrong? So how does this end up with the users doing the translation for the client? Surely it’s the Duolingo team themselves who are providing that service?

    Or is it a separate activity type with chunks of text that I haven’t seen yet?

    Thanks

    Amy

    Reply
    • Hi Phillip,

      Your point about my and Guy Cook’s exaggerated case against translation in ELT is very well made, and well taken.

      My writing this post will force me to get up to speed with computer-aided translation, but for now let me just say that your comments seem well argued and very reasonable. While some critics seem to just pounce on any mistake, I think the specific types of problems you highlight do indeed weaken the program’s ability to translate adequately. On the other hand, while I was “testing” the Spanish Duolingo, I didn’t find any cases of the translation actually leading me astray (I did the low-level exercises where I was confident I knew the right answers).

      I agree that the program “needs more sophisticated parsing tools, more sophisticated AI and it needs NLP”. And I completely agree with you about the need for more context. They must move beyond the present format to longer texts ASAP. My impression (and I’m not known for my sunny optimism)is that the Duolingo team do care, and that will work to improve their program.

      Reply
  5. Hi Amy,

    The part they use is under the heading “Real World Practice”. Here you see a text and you’re invited to “Click a sentence to help translate it”. But I don’t know how they use the students’ efforts to produce a good translation. If you watch this highly entertaining presentation by Luis von Ahn, the inventor of Duolingo: https://www.youtube.com/watch?v=cQl6jUjFjp4 , you’ll see, at around the 14th minute, that he says that they “play a trick”: they “combine the translations of multiple beginners to get the quality of a single professional translation”. He goes on to say that with 1 million users they could translate the entire on-line Wikipedia into Spanish in 80 hours! Maybe a technology-savy member of the ELT Jam team can explain how multiple translations are combined in this way.

    Reply
    • “Maybe a technology-savy member of the ELT Jam team can explain how multiple translations are combined in this way.”

      I wish I knew for sure, Geoff! I imagine that it works in the same way as Captcha (http://www.captcha.net/), which is by the same company. I believe — and don’t quote me on this! — that it works by making a best guess whether a translation is correct or not based purely on a vast quantity of data; that is, if enough people think that this word means X in English, it probably does.

      I’m sure someone else can give a better explanation than this, but that’s a first attempt 🙂

      Reply
      • Hi Nick,

        Thanks for your help! I guess the algorithm is basically “look at all the data for the same chunk and choose the most frequent one”. Phillip Kerr says “it’s based on statistical correlations between parallel texts” which I don’t entirely get, but it sounds sort of similar. Phillip, are you there?

        Reply
  6. Hi geoff,

    Enjoyed your article. I think you might want to perhaps reexamine the notional links between Behaviorlaism and ‘drill and kill’. I know you’re a bit fan of Noam but you might find CASTAGNARO 2006 enlightening in this regard. 😉

    Reply
    • Hi Russ,

      I don’t know what you think I think the “notional links between behaviorism and ‘drill and kill’” are, but I hope I made it clear that while I totally reject the behaviourist theory of learning, I don’t reject the use of drills and rote learning in ELT.

      As a result of reading Chomsky’s famous rebultal of Skinner*, and of reading the work of other scholars who adopt a cognitivist view (e.g. Fodor, Pinker, Casti, Hacking, Popper, Searle, Socal, Gross, Levitt), I’m persuaded that the behaviourist thesis that behavior can be explained without reference to non-behavioral mental activity is mistaken. The view that human behavior (including language learning) can be fully explained just by reference to its “functional” relation to the environment and to our history of environmental interaction is, IMHO, plain wrong.

      As for Castagnaro (2006), thanks for pointing out this gem, which I somehow missed. I think it is remarkable for the total mess it makes of defending both audio-lingualism and behaviourism.

      Castagnaro, P. (2006) Audiolingual Method and Behaviorism: From Misunderstanding to Myth. Applied Linguistics, 27/3: 519–526.

      Chomsky, N. (1959) A Review of B. F. Skinner’s Verbal Behavior. You can read an interesting re-write Chomsky did in 1967 here: http://www.chomsky.info/articles/1967—-.htm

      Reply
  7. In my recent co-authored work (Jarvis and Krashen, 2014) we argue that CALL is obsolete, and prior to this work I present a co-authored study (Jarvis and Achilleos, 2013) which suggest that we are in a “post-CALL era of Mobile Assisted Language Use” or MALU. My most recent work (Jarvis, 2014) asks what non-native students of English use mobile devices for, and what the implications are for #TESOL #ELT. When student have free choice they rarely go for tutorial CALL! All publications are #openaccess available from http://www.tesolacademic.org/huwjarviseditor.htm

    Reply

Leave a Comment

Get our newsletter

Learning Included is our newsletter for learning professionals who want to tap into the latest in learning design innovation, research and best practice.

Evidence-informed, inclusive learning design ideas every two weeks.


Not sure? Check it out first

Learning Included Newsletter