Developing an ELT product based on machine learning: Write & Improve

We believe that artificial intelligence (AI), machine learning and natural language processing are going to have a massive impact on ELT, and probably more rapidly than many might expect. A fascinating example of this is a new product from Cambridge called Write & Improve, which aims to provide automated help with writing. Diane Nicholls is one of the team behind the product, and we asked her to tell us more about how it. In this in-depth interview Diane talks about how the system works and, perhaps even more interestingly, how it was developed and what was learned in the process.

We think it encapsulates a lot of where ELT is heading – both in what the product itself is trying to do, but also in the way the project has brought together the worlds of ELT, academic research and technology in a way we haven’t seen before. This isn’t a review or endorsement of the product (you can try it out for yourself and see what you think), more a case study in how a product like this comes to exist and the thinking that goes into it.

Can you give us a brief overview of what Write and Improve is?

It’s a free online automated English language writing assessment and feedback tool that exploits advances in computational linguistics and machine learning to provide writing practice and feedback for EFL learners in an intuitive, engaging and easily interpretable way. Importantly, it’s a pedagogical tool and a practice environment, not a text editing facility.

The Write & Improve workbook

Who is it designed for, and how are you hoping it will help them?

It’s designed to be a supportive, encouraging environment for learners of English at all levels to practise their writing in a low-stakes, pressure-free environment, either in the classroom or for self-study, on mobile, tablet or desktop.
It’s based on the simple belief that the very best way for a learner to improve their writing, is through practice and feedback.
To provide the practice, it offers a range of essay prompts based on Cambridge English exams from KET to CPE. Learners choose the level they’re comfortable with or aiming for, select a prompt that appeals, and start writing. Alternatively, their teachers create Workbooks and tailor-made tasks for their students and assign them by sharing an invitation code or sending an email. Learners can also use the same Workbook function to set their own tasks, so that they can write about anything they like, or do their homework.
The main thing is to get learners writing. Through this writing practice, they gain confidence and, where relevant, familiarity with what might be expected from them in an exam.
When the learner submits their draft, the system provides targeted feedback of three types – summative, formative and indirect semi-corrective – in about 15 seconds. They can then make edits and resubmit as many times as they like. The aim is to gradually reduce the amount of shading and the number of feedback tips in their essay and, hopefully, raise their estimated score.

Feedback on writing

Because the system is non-judgmental and anonymous, they learn to ‘have a go’ and, because the feedback is immediate, they’re encouraged to keep trying.
It’s hoped users will learn the benefits and habit of reviewing their own writing, gain an awareness of their common and fossilized errors and the ability to spot and fix them, and become aware of other areas of frequent confusion to study further or take up with a teacher. Ultimately, we hope learners will start to enjoy writing!

There’s a lot of concern at the moment that AI is seeking to replace or sideline the role of the teacher. Is there a risk that Write & Improve could be seen as a threat?

To the teaching of writing by human teachers? Not at all! I doubt I need to list for your readers what a qualified and dedicated teacher brings to the teaching of EFL writing. Write & Improve is already miles better than no teacher. But even with the sort of scaffolding and other automated support we have planned for the near future, it is just a tool. What it can do is train motivated learners to identify and eliminate their common and repeated errors so that the version the teacher sees is free from those and the teacher is freed up to focus on the things only they can help with – discourse organisation, argumentation, nuance, and much more. Despite all the hype about AI, I can’t imagine an intelligent tutoring system ever being able to help with those things, because it can’t understand or infer communicative intent from ill-formed text, as a human can. Being sensitive to and anticipating the needs of a learner and finding just the right way to encourage and motivate them? Connecting with a learner and inspiring them? Being a language-learning role model? These are also not something anyone needs to worry about.

How exactly does it work?

It works by supervised machine learning based on an algorithm which is fed training data from the 30-million-word error-annotated Cambridge Learner Corpus and, from that data, ‘learns’ to spot the same errors and patterns of error in any future L2 data that is fed into it – a continuous improvement process. Basically, it ‘speaks’ learner English, exclusively – in fact, perfect English, purple prose and made-up nonsense confuse it enormously, because it’s so different to the training data. As it receives more data from users and our annotation team annotate that data and feed it back into the pipeline, it learns more and gives more and more accurate feedback. It was out in Beta, collecting data and learning from it for more than three years before launch in September, so it’s already very accurate. But, like teachers (and us!), it’s always learning.

Because it’s a pedagogical tool and because it can be used with or without a teacher, the algorithm is calibrated to be extremely cautious.

It only flags up possible errors when it’s more than 90% certain it’s right, as research is clear* that when it comes to feedback in pedagogy, an error left unmarked is less damaging than a correct use marked wrong. Precision is key here. This is all the more important in the absence of a teacher.

The system also refrains from making suggestions if it’s unable to be sufficiently certain what was intended. Instead, it highlights the whole sentence as dubious and needing attention. The learner is then, at least, pointed to areas where they could usefully focus their attention.

The algorithm’s also carefully calibrated not to give too much feedback in one go. We all know that too much red pen is ultimately demotivating and counter-productive. Instead, it returns initial feedback on common errors after first submission and then, as the learner edits those errors, surrounding errors become marked on next view, where possible and appropriate. This is designed to keep the user motivated and engaged.

There are three types of feedback:

  • Summative, overall assessment of writing competence, giving an approximate CEFR level, allowing learners to benchmark their progress against their peers. This score is based on their language production only.
  • Indirect, semi-corrective feedback on word-level issues, including spelling, grammar and vocabulary choice, plus a qualitative assessment of each individual sentence, making the learner aware of areas that might need attention. Crucially, with this feedback there is no “right answer” – don’t do that; do this. We only make suggestions on areas that need attention and, if clear, why. It encourages students to revise their work and assess the feedback to make their own decisions. There’s no spoon-feeding here!

Indirect feedback

  • Overview feedback, served with a heavy dollop of teacherly encouragement. It lets the learner know how they’re doing now – measuring themselves against their last performance, not anyone else’s. In this sense, it is personal, adaptive and formative. There are currently 268 different graded feedback bubbles to suit different achievement scenarios and levels.

Formative feedback

We also provide a personal progress graph so learners can visualise their progress on a task or across tasks:

Progress graph

This overview feedback performs three main functions:

  • It reassures learners that they’re not practising writing in a vacuum.
  • It takes the focus away from levels in cases where the level has not improved.
  • It emphasises that progress is incremental and not in six increments from A1 to C2, but little by little, within a level. That every little bit of progress counts!

Are there any other products that do something similar?

No. As far as we know, it’s completely unique. I think some users expect it to work as a spelling and grammar checker like the one Word offers, or a program like Grammarly or StyleWriter, for example. But that’s not what it is at all. It’s designed exclusively for EFL learners, and for teachers to use with their learners. It provides practice materials and a platform for that practice and, in its feedback, rather than correcting writing, it gives suggestions, leaving learners to reflect and make decisions themselves. It’s a pedagogical tool; a permanently free writing gym where learners are encouraged to practise, revise their work, then keep on practising, because, well, that’s how to improve.

What’s the story behind its development?

Like most things, it started with a conversation. With Write & Improve, it was between Ted Briscoe and Michael Milanovic. Ted Briscoe is Professor of Computational Linguistics at the University of Cambridge and co-founder and CEO of iLexIR Ltd, the company that spun out Swiftkey, maker of the world’s most popular predictive keyboard for smartphones. Mike Milanovic was CEO of Cambridge English Language Assessment (now retired), and had been working in English language teaching and assessment since 1977. They discussed the great things that could be achieved if the natural language processing and machine learning expertise at the Cambridge Computer Laboratory could meet the invaluable learner language data represented by the Cambridge Learner Corpus, a corpus of 30 million words of the written production of EFL learners which had been built, annotated and analysed over a period of 23 years by Cambridge English. That data, they agreed, would make excellent training data for pedagogical tools for learners.

As a result, English Language Intelligent Tutoring (ELíT) was created as a technology transfer business to develop the vehicle that would bring the data and the expertise together and drive the results out into the world. Paul Butcher then joined ELíT as third co-founder and CTO. Most recently, Paul was Chief Software Architect of SwiftKey, the #1 best-selling Android application worldwide for 3 years (SwiftKey was recently acquired by Microsoft). His role is to ensure that ELíT’s technology is robust and adaptable, and can scale to accommodate the demands of millions of learners worldwide.

The technology investment from the founder was complemented by funding from our Joint Venture partners at Cambridge University; Cambridge University Press and Cambridge Assessment.

Tim Parish and I had both worked with Ted on a variety of Natural Language Processing (NLProc) projects over the years and Tim, a software developer, joined to take care of the NLProc pipeline; the behind-the-scenes parts of Write & Improve which process learners’ submissions. I took on management of the data annotation project and recruited our team of 5 veteran EFL teachers to do the annotation. I had worked on the Cambridge Learner Corpus annotation for 20 years (part-time!), so I know my way around learner writing and the training data. And as I’ve worked in ELT materials development for just as long, I have a lot of input into the learner-accessibility and pedagogy of the content.

Meanwhile, Paul put together a crack team of developers, 4 of whom came to ELiT as a package deal after the demise of music-streaming service, MixRadio, and they work out of a converted shipping container in Bristol. Henry Garner, our data scientist and a Clojure expert, works from London and is responsible for the data capture and close analysis of learner behaviour.

Finally, Sara Garnham, joined us as General Manager in the spring of this year and is coordinating brand and marketing as well as relations and communications between the various stakeholders. She brings a lot of energy and business expertise to the company.

Did the proposition change or evolve during the course of development?

The big picture hasn’t changed, no, but as always, the devil is in the details.

What did you learn through developing the product?

Write & Improve was in Beta for more than 3 years and underwent rigorous testing with schools, and user and usage data were carefully analysed at the computer lab, among ourselves, and at the ALTA Institute (Automated Language Teaching and Assessment), which Ted directs. And so, that was a long and productive learning phase. One important lesson was that we needed to make the sign-up process much slicker and allow users to use the site without having to create a profile, at least initially. The leap from the Beta version to the current live version was enormous in terms of look and feel and technology. Crucially, we went from a traditional website model, to a Single Page App (SPA), which gives us much quicker response times. We were getting assessment times with the old Write & Improve of about 40 seconds, sometimes more, but now it’s around 15 seconds.

Since launch, of course, we’ve continued to learn. The product’s still under active development and we’re constantly assessing learner activity on the site to better understand how to make it more effective for them. We achieve this through a mixture of web analytics, data mining, and a variant of A/B testing called bandit testing. This is Henry’s department. A bandit test is similar to an A/B test, but, rather than assign variations equally amongst the users of Write & Improve during the test phase (as we would with an A/B test), a bandit test constantly analyses which variations are leading to positive outcomes and prioritises them in real time.

We’ve used this technique to prove that user-interface changes are actually yielding improvements in learner outcomes. For example, we log whether learners are interacting with the detailed feedback suggestions we provide – a strong indicator that the learner is attempting to engage with Write & Improve’s suggestions. Through bandit testing, we learned we can encourage learners to ‘click and reveal’ detailed feedback if we return the first detailed feedback pre-revealed. As data was collected on learner behaviour with this test in place, the site automatically adjusted itself to always present learners with their feedback this way.

Well, it’s also helped us prove that lots of things don’t make that much difference, but I don’t think that’s the most exciting message! For example, our pop-up messages encouraging users to create a profile are all bandit tested, and Henry still hasn’t uncovered any particularly statistically significant difference between our various messages (including the variant which just randomly chooses one). In a similar way, it helped by showing us that, surprisingly, conversion rates weren’t differentiated by whether we offered 1, 2 or 3 pre-signup tasks before asking users to sign up.
That last example led to a deeper scrutiny of what was actually going on, and the fact that most users weren’t reaching their pre-sign-up task entitlement. That came from ‘data mining’ – it’s the data mining, rather than the bandit testing, that’s showing the number of answers attempted by the average user each day is very gradually increasing for example. And how many and what sort of workbooks users are creating.

Luke, one of our Bristol team, managed to lure a number of EFL learners into the shipping container to do some live observation of them using the tool blind. This gave us a lot of interesting UX insights, many of which we’ve acted on.

So, based on these findings, we’re constantly trying different versions of functionality and optimising them in real time based upon real user behaviour.

What’s the business model?

The vision behind Write & Improve is to contribute to the democratisation of English language learning – easy access to a simple tool that can help anyone (with access to the internet), on any device, anywhere in the world, increase their confidence and improve their English, leading to greater opportunities in their life. To that end, we’re committed to keeping everything that Write & Improve currently offers free for all users. Going forward, we will make greater functionality available for paying customers as part of a freemium model. By the end of 2017 we hope students, teachers and institutions will all be customers of Write & Improve, and, looking ahead, will be able to use ELiT tools to support the other skills, not just writing. We are also working with our joint venture partners, and others, to incorporate this technology into their product ranges.

How would you like to see it developing in the future?

Write & Improve’s been built from the ground up to be a data-driven product. In practice, this means that future development will be guided by a close understanding of which features will improve learners’ experience the most, and help ensure that everyone using Write & Improve is motivated to achieve their goals.

We’re constantly working on Write & Improve, adding new features and adding to those already there, so it’s developing all the time. And we’re learning all the time, too, by studying how our users use the tool and responding to user feedback. This week, we added a History view, which will mean learners will be able to trace back through every iteration of their essays, from first draft to last, to see what they wrote, what feedback they got, what they did in response, how it improved their writing and their score etc. All their work is saved chronologically in a library they can revisit at any time. Of course, that means they can also share previous drafts with their teachers, for example.

Colleagues at the ALTA Institute at Cambridge University have been working on a prompt relevance function that will provide a score alongside the CEFR level for task achievement, gauging how relevant the writing is to the prompt. That’s all ready to go but we found it was slowing down the overall assessment, so it’s undergoing refinement. This feature, we hope, will help learners focus on *answering the question*, which is something we know they’re not generally good at.

We’ll also be starting to roll out our Premium offerings early in the new year, including a full teacher mode, so that teachers can see their students’ work and get diagnostic reports on progress etc.

And a trophy cabinet is coming soon where users will get ‘badges’, not just for achievement but for attendance, perseverance, frequency of sessions etc. Anything that will keep them engaged and keep them writing!

In the long-term, there are exciting plans for a Speak & Improve product, and much more besides …

And personally, I’d like to see Write & Improve becoming a regular habit with students and teachers as a way to practise their writing and learn to be better reviewers of their own work. Since launch in late September, learners in 180 different countries have had free writing practice and feedback with Write & Improve. I get a real kick out of that and I’d love to see that number get as close as politically and geographically possible to the full 195! One learner responding to the ‘tell us what you think’ prompt in Write & Improve said ‘I’m even starting to like writing!’ If we could get other learners to feel the same, that would be amazing, too.

Further information

Write and Improve will be involved in the first ever summer school in Machine Learning for Digital English Language Teaching (ELT), to be held 3-7 July 2017, in Chania, Crete, Greece, organised by the Automated Language Teaching and Assessment Institute, University of Cambridge. Expect to leave with a better understanding of various aspects of Machine Learning, Natural Language Processing and Psychometrics and how they can apply to your ELT context. Also expect to leave with a bit of a tan! For more information and tickets, check out the summer school website.

Write & Improve website:

Write & Improve promotional/overview video:

* Ryo Nagata and Kazuhide Nakatani. 2010. Evaluating performance of grammatical error detection to maximize learning effect. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, COLING ’10, pages 894-900, Stroudsburg, PA, USA. Association for Computational Linguistics.

A lot of the early background research and NLProc is in this paper, if you’re interested in looking into it in more depth:

You can also find Write & Improve on Twitter at @WriteandImprove


21 thoughts on “Developing an ELT product based on machine learning: Write & Improve”

  1. I agree that writing programs like this will be a great advantage to learners. As a former teacher I know that marking letters is extremely time consuming. Of course there is still a lot of development to be done in this area, but it is good to see that other publishers are making a start.

    I should mention that this product is not completely unique. We have had writing exercises that are semi-automatically corrected for a couple of years. Ours are part of a complete course and not available as a stand alone product.

  2. It’s great to see how far this ground-breaking service has come since beta. UX is now very attractive and intuitive.
    How well does it work? Well I tried it with a couple of real FCE answers, a high-scoring and low-scoring one. A tiny sample, but enough to get a first impression. Summative assessment looked spot-on to my eye, and always moved in the right direction with subsequent iterative submissions. For error detection, precision was pretty high (the errors flagged seemed to be correctly identified), while recall was low (lots of errors were not flagged) – but this is something Diane – and the literature – declare to be intentional. My feeling is that this was fine for the first submission of the low score answer, but I would have liked it to flag new errors with later iterations (maybe that does work, I just didn’t see it in my little test).
    Granularity of feedback is really impressive – 268 feedback types!
    And Diane says that the answer content relevance (how well the learner answered the question) , scaffolding and other goodies are all in the pipeline.
    This is certainly one to watch – it will be interesting to see how it is to be embedded in future products…

    • Dear Paul Ricketts,
      I am a phd student working on my doctoral dissertation research and my focus is Write&Improve. I am planning to test the platform by getting a teacher account and getting my students to engage in some writing tasks. My aim is to find out whether the platform contributes to the improvement of their writing skills by giving the participants pre and post tests. Another component of my research is to get opinions of teachers who have tried / practiced on the platform around the world. Those teachers’ invaluable ideas will be really helpful to see potential strengths and challenges related to the platform. Would you be so kind as to contribute to my research by answering a few open- ended questions which will not take more than 20 minutes of your time ? I am looking forward to hearing from you. Please find my e-mail address here: for your reply, comments or any further questions.

  3. Paul,

    I’m Paul Butcher, co-founder and CTO of English Language iTutoring, the company behind Write & Improve. Thank you so much for your kind comments, I’m delighted to hear that your experiments with it have been successful.

    We’re very excited about what we have in development, and have plans to improve and enhance the product that will keep us busy for several years. You mentioned that it would be interesting to see how it’s is to be embedded in future products – if you have any particular thoughts or ideas about that, we’d love to hear them.

    Thanks again,


    • Hi Paul,
      Well, I think it would be really interesting to see how teachers could interface with W&I, which does a great job at assessment but still leaves room for improvement in giving meaningful feedback. I’d like to see integration of the automated feedback and manual corrections and comments from the teacher, as well as acceptance/rejection of the assessment and suggested corrections.
      That way students will have been encouraged to reiterate and improve their tasks before submission, and teachers, given well-crafted tools and UI/UX, would be very happy to get help with the repetitive, boring task of marking in digital format for ELT, which is usually even more of a pain than on paper.
      Then you have a seamless service, from draft through submission to correction, feedback and gradebook, which you could plug into a LMS and would be an extremely attractive prospect for any ELT course.
      This would create a corpus of annotations on the automated output, so W&I could get a lot back too.

      • Thanks for the feedback, Paul.

        Happily we’re thinking along very similar lines. We’re currently working on a number of enhancements aimed specifically at teachers to help them get an overview of how their students are progressing. We expect to demonstrate it at IATEFL and release it shortly thereafter.

        It won’t provide everything you’re after at first release, but what you describe is definitely the end-game.

        Thanks again!

  4. I am an English teacher and I’m absolutely enthusiastic about this tool. I have created workbooks to make my students practice their writing skills, but I can’t find the way to look at their trials, nor the tasks I give them. Can you help me, please?

    • Dear Susanna Carra, I am a phd student working on my doctoral dissertation research and my focus is Write&Improve. I am planning to test the platform by getting a teacher account and getting my students to engage in some writing tasks. My aim is to find out whether the platform contributes to the improvement of their writing skills by giving the participants pre and post tests. Another component of my research is to get opinions of teachers who have tried / practiced on the platform around the world. Those teachers’ invaluable ideas will be really helpful to see potential strengths and challenges related to the platform. Would you be so kind as to contribute to my research by answering a few open- ended questions which will not take more than 20 minutes of your time ? I am looking forward to hearing from you. Please find my e-mail address here: for your reply, comments or any further questions.

  5. Hi Susanna, it’s great to get your feedback. Thank you so much for your enthusiasm! We will soon be releasing a full teacher dashboard so that teachers can see their students’ progress and look at their writing in detail. We’re working hard on it right now and it will be released in April as one of our Premium offerings. Until then, I’m afraid asking students to copy and paste to send their work to you, or looking at work on the student’s device is the best way to see their efforts. Thanks again! Diane

    • Dear Diane Nicholls,
      I am a Phd student at Anadolu University Institute of Social Sciences, Department of Distance Education and I am writing with regard to my doctoral dissertation research. I would like to use “Write & Improve” for a group of EFL learners (about 20 students) studying at open and distant learning programs at Anadolu University in Turkey. I am planning to purchase a teacher account and get my students to register my class on the platform. My aim is to give them a pretest followed by 6 different writing tasks and a post test in order to gain insights about the potential positive effects of the platform on the improvement of their English writing skills.I am planning to ask learners about their opinions regarding the platform reflecting on their experience. I also want to compare the scores given by the platform with those given by the language teachers. To conduct my research in accordance with ethical values, I kindly ask you to allow me to use this platform for my dissertation research. If you need any further information or have any questions, I would be more than happy to contact you. Here is my e-mail address:
      Ayşe Taşkıran
      Anadolu University
      School of Foreign Languages

      • Hi Ayşe,
        Thank you very much for your interest in Write & Improve. We would be more than happy to help you with your research, which sounds very interesting. Please contact to discuss the best way for us to help you.
        Best wishes

  6. This looks like an excellent idea. Many learners suffer from lack of feedback that guides them to think of the acceptable answer for their mistakes. Even more suffer from rather ill-informed but well-meaning comments from friends or self-styled gurus on the Internet. I look forward to seeing this concept in action and building on its own experience. As someone who has spent many years helping learners improve their writing skills and analyzing the common mistakes they make, I welcome your initiative.

  7. Hi there,

    I am an English trainer working in Xi’an. Like many of the others who have commented above, I was excited to test Write & Improve to see if it would be useful with my trainees, who are all college-educated Chinese with English proficiency levels ranging from very low (e.g., having difficulty with basic questions and answers) to fairly high (e.g., fluent and near fluent).

    While the design is rather slick and streamlined and the premise enticing, there are several things that have concerned me and made me unlikely to consider this ready for use in my training classes.

    I’ve tried answering some of the prompts myself, and I’ve also experimented with creating prompts and entering past student work, as well as using text from various sources on the internet. I’m sorry to say that my experience has largely been negative for the following reasons:

    1) Most of all, the system seems to either fail to identify, or neglect to point out, a wide variety of errors in style, grammar, spelling, word choice, etc., while also misidentifying correct words and sentences as “suspicious” or problematic (“There are some problems in this sentence” or “This sentence could maybe be improved”). You say that the system only flags something as a potential error if the system is 90% sure it is an error, but given the high number of false errors and incorrectly flagged words and sentences I have found in my small amount of testing so far, that is worrying. Many of these errors are caught by tools like Microsoft Office, and with better explanations, too.

    Oddly, your system also seems biased against certain words, like “toward” (I know Brits prefer “towards”, but both forms are grammatically correct) and “actual” (no idea why that one is disliked–I tried using it in a variety of different ways and the system nearly always flagged it as a suspicious word or problematic/potentially problematic sentence).

    I am not working with a corpus of 30 million words, but I have tried a fairly wide variety of actual writing from EFL learners in addition to my own writing and that of other sources. I did note that you say it is by design that not all errors are pointed out initially, so as to avoid discouraging writers with too much ‘red pen,’ but even after correcting all the ‘errors’ and ‘problem’ areas the system identified, simple errors in spelling, tense, subject-verb agreement, count, word usage, punctuation, etc. were not identified (I did notice that, sometimes, if typos and errors were corrected, the graph might reflect a small improvement, but not always, and sometimes good edits caused downward movement on the graph as well). If no errors or suggestions are given, shouldn’t basic mistakes be highlighted, or are these actually being missed?

    2) I applaud the idea of highlighting problem areas and asking learners to identify and correct mistakes for themselves, instead of relying on a machine to make corrections for them that they may not understand. However, the suggestions I encountered very rarely were helpful in indicating what kind of a mistake or problem might be present if suggestions were even offered at all (most of the time, for most of the writing samples I submitted, they weren’t, though occasionally if a wrong form of a word was used–e.g., “different” instead of “difference”–the correct word was suggested as a possibility). Much more guidance is needed. Again, I’ve found Word’s explanations (though less user-friendly to access) much more helpful in indicating what might be wrong and how it could be improved. This is especially frustrating when the system misidentifies a correct sentence as in need of improvement or possibly in need of improvement.

    2) There appears to be little to nothing in the way of helpful explanatory material on how to use the various features of the site. FAQs, guides and tutorials would be welcome–as would message boards for teachers and students. Just finding this page wasn’t easy.

    3) Lack of functionality for teachers. I understand that some of these features (as you and others have described above) are in development for a premium/freemium option for teachers, but given my experience so far I am very unlikely to pay money to test them out unless the core product is significantly improved.

    4) Lack of error reporting options.

    I know you say the system was only designed with materials from L2 English learners, such that “perfect English, purple prose and made-up nonsense confuse it enormously, because it’s so different to the training data,” so if it were only my own writing or writing from other educated, native speakers that was causing the problems I’ve described above, that would be understandable. But when I supplied various samples of actual writing from my trainees, at a variety of English levels, my results were largely the same. I know at least some others have commented on having much more positive experiences. I don’t know how to explain the difference.

    I would love to have something like this to use with my trainees. It could cut down on the time I have to spend marking and giving feedback, and allow me to focus more on higher level issues when I do give feedback, all while encouraging my trainees to write more and to edit and revise their work. That would be wonderful. But the issues I’ve described above really are too significant right for me to try using it with a class right now.

    • As a point of interest, running the text from this article, as well as text from the Write & Improve website, garners scores between A1 and C1 (from the segments I’ve tested so far, at least), including thinking that saying each of the ‘words’ “Write” “&” “Improve” (punctuation is always described as a ‘word’ in your feedback bubbles, when it’s even commented on at all) were all “suspicious,” and flagging several sentences as containing problems or “could maybe be improved.” Again, I know what you said about the system being used to L2 texts and not “perfect English” (etc.), but when it makes the same mistakes for L2 supplied content, that is worrisome. How does a learner know what really is good and isn’t, if the system is incapable of making the distinction itself?

      I ran everything under a self-created prompt for writing an essay about anything, with a suggested length between 25 and 600 words, testing blocks of text from this article and from the Write & Improve website (including the feedback bubbles). I get similar results when running L2 texts that my trainees have submitted for homework in previous terms.

    • Michael,

      As you know, we’ve discussed some of the points you raise over e-mail, but I wanted to summarise them here for anyone else visiting this page.

      Write & Improve is a young product, under continuous development. There are a lot of features that we would love to have in place, but haven’t yet had a chance to implement, of which the kind of “Report” or “Feedback” feature you mention is definitely one. For the time-being I’m afraid that we have to rely upon this kind of feedback We’re also acutely aware of the point you make there about a lack of FAQs, guides and tutorials. Expect to see some improvement in this area soon.

      Regarding your main point about the feedback, there are two points you should be aware of:

      The first is that the system is not trying to compete with systems like Microsoft Word’s grammar and spell check functionality, or Grammarly. Those are valuable products, but they’re not trying to solve the same problem as Write & Improve. They are useful for people who simply want their writing “fixed”. Write & Improve, by contrast, is a pedagogical tool intended for learners who want to improve their skills. To that end, it deliberately does not “bombard” the user with excessive feedback. Instead it is intended to highlight areas of the text that could do with additional work to allow students to focus their efforts most effectively, together with a few concrete suggestions. These suggestions have been deliberately tuned to avoid “bombarding” the user with excessive feedback, so you are quite right that there will be plenty of occasions where a suggestion could be made, but is not.

      The second is that the system is targeted at language learners, not fluent native speakers. The Write & Improve “engine” has been trained on the kind of writing created by learners, and will therefore only return valid results for that kind of text. It simply hasn’t seen enough text written by native speakers to be able to give sensible results. Correcting the kind of writing that you create as a fluent native speaker simply isn’t something that it’s capable of doing. The same would be true if you were, for example, to give it extracts from James Joyce or Shakespeare.

      I’m aware that you have tried with some of your students’ writing. My strong expectation is that you would see much more helpful and accurate results for students’ writing, although please bear in mind that the intention of the tool is to provide them with pedagogically sound feedback and pointers, not a simple set of suggestions that they should mechanically apply without thinking about them. If you have any examples where the feedback provided by Write & Improve for a student’s writing is not accurate or helpful, we would love you to bring it to our attention – the engine is continually learning and improving, and examples like this are a massive help.

    • Dear Michael, I am a phd student working on my doctoral dissertation research and my focus is Write&Improve. I am planning to test the platform by getting a teacher account and getting my students to engage in some writing tasks. My aim is to find out whether the platform contributes to the improvement of their writing skills by giving the participants pre and post tests. Another component of my research is to get opinions of teachers who have tried / practiced on the platform around the world. Those teachers’ invaluable ideas will be really helpful to see potential strengths and challenges related to the platform. Would you be so kind as to contribute to my research by answering a few open- ended questions which will not take more than 20 minutes of your time ? I am looking forward to hearing from you. Please find my e-mail address here: for your reply, comments or any further questions.

  8. More than one year after Michael’s criticisms of the software, I continue to run up against the same errors while using “Write & Improve.” I have been testing the program as an option in our bilingual school. However, the “feedback” is hardly feedback at all. It has a tendency to flag perfectly valid sentences as problematic without providing any information beyond “this sentence has problems.”

    After a few trial runs, it became clear that the system prefers British English wording as well as simple sentences. In addition, it is biased towards more formal and analytical writing. For example, I gave my advanced students the task of writing the inner thoughts of a character from our reading and nearly all of them came back with A1-A2 scores. When I asked them to rewrite it as though they were presenting an argument, the scores increased to B2 (which is still low compared to their ability level). As an experiment, I cut and paste the same text into a different but still relevant W&I Advanced task box and found that the progress tracker moved down rather than staying at the same level.

    I always attempt an assignment myself before giving it to students and, as a former Oxbridge post-graduate, I found it surprising that my own practice tests came back anywhere between B2-C2 level. I understand that native speakers are not the intended audience for this program, but it seems bizarre that it doesn’t recognize native writing when it sees it, especially as C2 level is supposed to indicate near to total native fluency in a language.

    Frankly, I’ve found Grammarly much more effective for collecting data and correcting mistakes. In addition to tracking common mistakes in grammar, punctuation, and style, Grammarly allows me to add vocabulary words to my dictionary when necessary instead of counting them as errors. It also comes with a plagiarism checker and allows for larger assignments that more adequately prepare my foreign students for rigorous academic work. Grammarly doesn’t give any indication of the CEFR level and that’s essentially the only thing “Write & Improve” has going for it–in my humble opinion. Unfortunately, the lack of reliability in assessing high-level work undermines its usefulness for students studying attempting the exams designed to prove proficiency.

    • Dear Ms. B., I am a phd student working on my doctoral dissertation research and my focus is Write&Improve. I am planning to test the platform by getting a teacher account and getting my students to engage in some writing tasks. My aim is to find out whether the platform contributes to the improvement of their writing skills by giving the participants pre and post tests. Another component of my research is to get opinions of teachers who have tried / practiced on the platform around the world. Those teachers’ invaluable ideas will be really helpful to see potential strengths and challenges related to the platform. Would you be so kind as to contribute to my research by answering a few open- ended questions which will not take more than 20 minutes of your time ? I am looking forward to hearing from you. Please find my e-mail address here: for your reply, comments or any further questions.


Leave a comment