One of the promises of digital publishing is the ability to improve and update a product on a continual basis. We recently posted about iterative publishing. In order for that to work, you need to know which parts of your course are working well and which are failing to meet their objectives. In part 3 of our interview series with Knewton, David Liu (Chief Operating Officer) and Sally Searby (Partnership Manager) explain how they believe their data analysis can provide the insights needed to refine and improve ELT courses, going beyond what even the most knowledgeable and experienced author/editor team could do.
Do you want to say a bit more about how [Knewton’s data analysis] will improve ELT products on an iterative basis?
With the data, each of the publishers gets analytics. And there are analytics at different levels. There are teacher analytics, which of course help the customer of the publisher: they can group students into optimal study groups, they can really do deep dives on students, and really understand what’s happening with each student. So that’s a very visible, marketable benefit.
We help publishers and content creators to evaluate content efficacy at the concept level and create more effective learning materials
But for the publishers themselves, we have the ability to provide content efficacy. We help publishers and content creators to evaluate content efficacy at the concept level and create more effective learning materials.
A good example is when we look at the knowledge graph of our partners, which is a map of how concepts relate to other concepts and prerequisites within their product. There may be two or three prerequisites identified in a knowledge graph that a student needs to learn in order to understand a next concept. And when we have hundreds of thousands of students progressing through a course, we begin to understand the efficacy of those said prerequisites, which quite frankly were made by an author or set of authors. In most cases they’re quite good because these authors are actually good in what they do. But in a lot of cases we may find that one of those prerequisites actually is not necessary, and not proven to be useful in achieving true learning or understanding of the current concept that you’re trying to learn. This is interesting information that can be brought back to the publisher as they do revisions, as they actually begin to look at the content as a whole. Is there a return on your investment in producing some of this content? It costs money to build the textbook, and even in a digital world it costs a lot of money. So, if we can provide some content efficacy back to the publisher on how not only that book, but how certain constructs and chapters are working, publishers can be smarter about how they invest in those areas.
We’re beginning to value both the bundled content, which is traditionally textbooks and even online textbooks, but also how to unbundle certain content to be able to provide subscriptions to certain areas. And, if we can do that, then you need to be more efficient with your investment.
So those are a couple of benefits. But I think overall, analytics and understanding what is actually happening with you customer, with students and teachers, is important. You can begin to provide value to schools and help teachers differentiate instruction or understand what concepts a class struggles with, or even help administrators at a more macro level.
And the ELT publishers are all talking about how to get closer to their customers through digital content and delivery and analytics – through measurable data in order to improve the learning experience through better content.
better insight changes the role of the publisher
Yeah, I think generally better insight changes the role of the publisher. You’re providing awesome content obviously, you’re providing a great user experience, but now there’s insight into what’s happening, and I think that, we hope, provides a bit of an evolution in the business model. Because it’s not about binding and printing and sales and distribution only.
I think one of the interesting challenges for publishers is actually going to be using those insights. How quickly and how effectively will they be able to take the insights that they can gather from this kind of data, and then put them into practice in the next iteration of the product? I think that’s going to be an interesting thing for publishers to grapple with once the data starts coming in.
Absolutely. In ELT I find it to be a fascinating issue, because it’s so big, but also what we’re learning in some cases is that ELT is very successful in what it does in an offline mode. As the world in education is moving online, how does ELT transition? We’re seeing a lot of different interesting models being tested out around that transition.
There’s a lot of opportunity to provide the same great content or even better, if you can show some efficacy and put the best content for the individual student in front of them. Again that’s another huge mindset change. It’s not just the best book overall – what does that mean? The best book for the average of a million people may not be the best book for me, or Sally, or Molly.
And so it’s not even about the book, it’s about that particular concept or that chapter or that module that you’re working on. I think it’s an incredible opportunity to keep the best of what has been working in ELT and publishing, and education in general, but then leverage these new tools and technology.
And I think the notion of programs lasting 15 weeks or 10 weeks – the time-based kind of paradigm that we’ve been in, or session-based paradigm, gets blown up. What happens then? What you know becomes more important and students can move at their own pace. We’re seeing a shift even in physical universities in the United States and abroad. The University of Southern New Hampshire and other schools are moving to competency-based programs.
It’s just an exciting time to be in education. And with ELT specifically, it’s such a global business that I think they can leverage adaptive learning technology more in some respects than others, because you’re talking about students learning similar things around the world. That’s a lot of data. The learnings that you can gain are tremendous if you’re an ELT provider.