Most institutions that buy an AI tutor assume the hard part is choosing the right one. One of the first large-scale controlled trials points somewhere else. A two-year randomised trial of Khanmigo in 18 Tennessee middle schools, published by the National Bureau of Economic Research in August 2026, found that 96 percent of students tried the tutor at least once, yet they messaged it in only 17 percent of the exercise sessions in which they made a mistake. The tutor was there, scheduled into the school day and configured to coach rather than give answers, and students mostly did not talk to it.
That is the AI tutor student engagement problem, and it changes what a college curriculum lead or a university programme director should ask of a pilot. If learners do not use the tool at the moments it is meant to help, no amount of model quality will show up in attainment, and a pilot that only measures outcomes will not be able to tell you why. This guide is written for that reader: someone who has to decide whether an AI tutor earns a place in a course, and who will be held to the results, not to the vendor's demo.
The scope is deliberately narrow. This piece is about measuring and improving engagement during a pilot. It does not compare products, which our Khanmigo comparison, the alternatives guide for UK buyers, and the AI teacher platform buyer's guide already do. It does not argue the case for Socratic design, which our Socratic AI tutor explainer covers, and it leaves the reading of vendor impact studies to our CENTURY Tech comparison, the alternatives guide for colleges.
This guide is shipped by the team behind Dilr Academy, an AI tutor that builds interactive, multilingual courses from an institution's curriculum and asks before it explains. Or see DATS, our five-stage system for placing AI inside an organisation's existing work.
What is the AI tutor student engagement problem?
The AI tutor student engagement problem is the gap between giving learners access to an AI tutor and learners actually using it at the moments it can help. A 2026 trial found that almost every student tried the AI tutor, but few turned to it when they were stuck, and the messages they do send are often bare answers rather than reasoning. Access is cheap; productive use is the hard part.
The distinction matters because the two are measured differently. Access is a licence count and a login rate, which is what a licence report shows and which says nothing about whether learning moved. Productive use is narrower: a learner hits a problem they cannot solve, turns to the tutor, explains what they tried, and works through the next step. That event is what an AI tutor is built for, and it is the event most pilots never count.
The word engagement also carries a second meaning that an institution should keep separate. In safeguarding terms, heavy or emotionally loaded use of a tutor is a risk to watch, not a target to maximise, and the AI safeguarding guide for schools and trusts covers that side. Throughout this guide, engagement means learning engagement at the point of difficulty, not minutes on screen. A course team that understands this will design its AI operating model around the moments that matter rather than around usage totals.
Why do students ignore AI tutors, according to the trials?
Trial evidence suggests students ignore AI tutors because using one is optional effort at the moment they are stuck. In the 2026 Khanmigo trial the median student messaged the tutor on only a third of the days they practised, and a 2026 Stanford-led trial found nearly half of the pupils in its control group never used an AI literacy platform. Availability did not create the habit.
The detail of the NBER study, Working Paper 35620 by Philip Oreopoulos and Nina Low, is worth reading closely because the conditions were favourable. Students used Khan Academy with Khanmigo during existing daily remedial mathematics sessions, so the time was already in the timetable. Assignment raised maths achievement by 1.3 national percentile ranks per term, about 0.06 to 0.08 standard deviations over a school year, and the authors note that these gains resemble those from Khan Academy practice without AI assistance. The median student messaged the tutor on only a third of the days they practised, and the messages students did send were mostly bare answers or clicks on suggested prompts. The authors conclude that the binding constraint appears to be engagement. Our buyer guides summarise the same trial for readers comparing products; this guide uses it for a different question.
The second piece of evidence comes from younger learners. In EdWorkingPaper 26-1451, Carly Robinson, David Gormley, Ana Trindade Ribeiro and Susanna Loeb report two randomised controlled trials with 355 students in grades 1 to 5, run in 2024 and 2025. Despite dedicated session time, nearly half of the students in the control group never used the AI literacy platform, and those who did averaged only 2 to 5 minutes per week. That is a primary-school reading context, not a college maths course, but the pattern matches the Tennessee result: time on the timetable does not turn into time on the tutor.
Put the two together and a plausible mechanism appears. Asking an AI tutor for help means admitting you are stuck, typing out what you tried, and reading a question back instead of an answer. A learner who can guess, skip, or wait for the teacher has three cheaper options. On that reading, a tool does not need to be disliked to go unused; it only has to lose to easier behaviour, which is why the AI placement diagnostic we run for institutions starts with the workflow a tool must fit into rather than the tool itself.
Does a tutor that asks before it tells solve the engagement problem?
A tutor that asks before it tells does not, on current evidence, solve the engagement problem on its own. The Khanmigo tutor in the 2026 NBER trial was already configured to coach rather than give answers, and students still rarely used it when they made mistakes. Socratic design shapes what happens inside a conversation; it does not make a learner start one.
This is an uncomfortable point for anyone selling or buying a Socratic tutor, including us. The case for questioning before explaining is about learning quality: a tutor that hands over solutions invites learners to offload the thinking, and our Socratic explainer, linked above, sets out the evidence for that. The engagement problem sits one step earlier. If the learner never opens the conversation, the quality of the questioning is irrelevant, and a coaching tutor may even raise the effort cost of starting, because the learner knows a question is coming back rather than an answer.
So the honest position for a programme director is that design and engagement are two separate requirements. A tutor needs a teaching pattern worth engaging with, and the course around it needs a reason for the learner to engage. Our guide to adaptive learning and the UK curriculum covers the first half; the rest of this guide covers the second, which is mostly an institutional job rather than a product feature.
Which engagement metrics should an institution track in an AI tutor pilot?
An institution piloting an AI tutor should track productive engagement at the point of difficulty, not total minutes. The most useful measures are the share of learners who never start, the share of active days, the share of mistake moments where the learner asks the tutor for help, and the quality of what the learner sends. Each one maps to a failure the 2026 trials actually observed.
The table below turns the trial findings into pilot metrics. None of them needs a research team; most can be drawn from the activity log a supplier should be able to export, and an institution should ask for that export before the pilot starts rather than after.
| Metric | What it catches | Where the trials saw it |
|---|---|---|
| Never-started rate | Learners who had access and did not use it once | Nearly half of the control group in EdWorkingPaper 26-1451 |
| Active-day share | Use that drops away between sessions | Median student messaged on a third of practice days, NBER 35620 |
| Help-at-mistake rate | Whether the tutor is used when a learner is stuck | 17 percent of exercise sessions with a mistake, NBER 35620 |
| Message quality | Bare answers and prompt clicks instead of reasoning | Mostly bare answers or clicks on suggested prompts, NBER 35620 |
| Weekly minutes | Context only, never a target on its own | 2 to 5 minutes per week among users, EdWorkingPaper 26-1451 |
The help-at-mistake rate is the one to build the pilot around. It needs the tutor or the surrounding platform to know when a learner has got something wrong, which a structured course with checked exercises can do and an open chat window cannot. Message quality is the second priority, because a learner who types only an answer is using the tutor as a checker, not a teacher.
Weekly minutes belong in the table only as context. The DfE product safety standards expect learner-facing products to report usage duration to teachers, and separately to include default time limits as a wellbeing safeguard, so duration is something to watch rather than a number to push up. Treat rising minutes as a question to investigate, not a win to report. A DATS execution office engagement can set these measures up as a standing review rather than a one-off report.
What actually raises AI tutor engagement in practice?
The evidence so far points to human structure around the tutor rather than features inside it. A 2026 Stanford-led trial, the only one here that measured engagement directly, found an in-person tutor focused on engagement raised AI platform use. A 2025 World Bank trial in Nigeria found learning gains when AI tutoring ran in teacher-supported sessions. Neither shows structure alone guarantees learning gains.
The Stanford-led result is the clearest test of a single lever. In EdWorkingPaper 26-1451, working with human tutors increased average weekly platform usage by 1 to 4 minutes and engagement by 71 to 80 percent. The same paper is candid about the limit: usage remained low, and the intervention did not improve reading achievement. Human support moved the engagement number without moving the outcome, which is exactly why a pilot needs both measures and should not stop at the first.
The Nigerian study shows what a well-structured programme can look like. World Bank Policy Research Working Paper 11125, by Martin De Simone and colleagues, evaluated a six-week after-school programme in which first-year senior secondary students used Microsoft Copilot for English learning. It reported an improvement of 0.31 standard deviations on a combined assessment of English, AI knowledge and digital skills, and 0.23 standard deviations on English, the main outcome. The authors' own cost-effectiveness framing puts the gain at 1.5 to 2 years of business-as-usual schooling. The World Bank's write-up of the programme adds that the more sessions students attended, the greater their gains, and stresses teacher support. That attendance pattern is observational rather than randomised, and the setting, a Nigerian after-school English course, is a long way from a UK college, so it is a design example, not a forecast.
Read together, the studies suggest four levers worth testing in a pilot. The first two draw directly on the trials; the last two are our own design suggestions that follow from them, not tested findings. Each is an institutional decision:
- Protected, supervised time. Both trials that found weak engagement had scheduled time, so time alone is not enough, but the Nigerian programme paired its sessions with teacher support.
- A human whose job is engagement. The Stanford-led trial isolated this and found a real effect on use, if not on reading scores.
- Tie tutor work to assessed work. A tutor that sits beside the course is optional; a tutor whose exercises feed a teacher's review becomes part of the course.
- Review the activity log weekly. Building that review into the course's AI operating model means teachers who can see who never started, and who only types answers, can intervene within days rather than at the end of term.
The same diagnostic logic underpins our AI solutions for institutions, which start from the job a team has to do and fit the tool to it, not the other way round. For a university, the governance side of that decision is covered in AI governance for UK education institutions.
How should a college or university run an AI tutor pilot that tests engagement?
A college or university should run an AI tutor pilot in four steps: agree the engagement measures before launch, schedule supervised tutor time inside an existing course, review the activity log weekly, and judge learning outcomes only after engagement clears an agreed bar. Measuring outcomes without engagement data leaves an institution unable to explain a weak result.
Agreeing the measures first sounds administrative, but it decides whether the pilot can be read at all. Write down the never-started rate, active-day share, help-at-mistake rate and message-quality check you will use, and ask the supplier to confirm the activity export can produce them. If it cannot, that is a finding in itself.
Scheduling the time inside an existing course is the step the trials suggest is necessary but not sufficient. Put the tutor into sessions a teacher already runs, with a named person whose job during those sessions is to get learners talking to it. Then review the log every week. The point is not surveillance; it is spotting the learners who never started in week one rather than week ten.
Judge outcomes last, and against something. The NBER estimate rests on randomly assigning students to use the tutor or not, which is what lets it attribute a gain to the tool. A single cohort that improves tells you little, because a cohort's results can move for many reasons besides the tool. If a full randomised design is out of reach, compare against the same course's previous cohort and be honest about the limits. Our AI placement diagnostic is built around this kind of measurement plan for any AI tool, not only tutors.
What do the DfE standards expect suppliers to report on usage?
The DfE generative AI product safety standards, updated on 19 January 2026, expect learner-facing products to monitor and report data to teachers, including the rate of requests for cognitive offloading and the duration of usage by each individual learner. They are expectations on developers and suppliers in England, not statutory duties on schools, colleges or universities.
That reporting expectation is useful to a pilot even outside the schools sector it was written for. A university is not the audience for the DfE product safety standards, but it can use them as a benchmark for what a supplier's activity data should cover. The same document is direct about claims: "Suppliers should not exaggerate the impact or capabilities of their tools." It goes on to ask that claims be supported by transparent evidence. In our reading, that points to published trials with comparison groups rather than testimonials.
Two cautions follow. First, the responsibility for meeting these expectations sits with the supplier, so the institution's job is to ask for evidence and read it, not to certify the product. Second, the standards treat time limits and emotional engagement as safety matters, which reinforces the earlier point: duration is reported for teachers to review, not to maximise, and a pilot should use the reporting to find learners who are not being helped, not to celebrate rising usage. The full standard-by-standard review sits in our safeguarding guide for schools and trusts.
What is the best AI tutor for student engagement in 2026?
The best AI tutor for student engagement in 2026 is the one whose evidence and reporting let an institution see, within weeks, whether learners use it when they are stuck. We have not found a product with published UK randomised evidence of high engagement, so the verdict rests on criteria: exportable activity data, help-at-mistake visibility, honest outcome claims, and fit with the course timetable.
On those criteria the choice depends on the buyer. Khanmigo has been studied in one of the first large-scale randomised trials, two years across 18 schools, and a US district that values that evidence base has a reasonable case for it even with the engagement result. CENTURY Tech, Sparx and Third Space Learning are other common shortlist names, and our comparison guides set out where each one fits: see the alternatives for maths homework practice and the alternatives for online maths tutoring. Where a product already sits inside a well-run course with a teacher reviewing it, switching for a newer tool rarely fixes engagement on its own.
Dilr Academy fits a narrower case: an institution that wants an AI tutor to teach its own curriculum, in the learners' languages, rather than a fixed library. That is a reason to shortlist it, not a claim about engagement. The AI teacher platform buyer's guide sets the products side by side, and the overview of AI teacher platforms in the UK maps the wider market.
Where does Dilr Academy fit in an engagement-led pilot?
Dilr Academy fits an engagement-led pilot as the tutor inside a course an institution already runs. It builds courses from the institution's curriculum, asks a question before it explains, and uses knowledge tracing to set the next step. As our buyer guide states, no published trial has measured Dilr Academy's effect on learning, so a pilot should test it the same way as any other tutor.
What Dilr Academy offers a course team is structure. Its courses are a real syllabus with modules, topics and stated outcomes, and the tutor works through each idea with a live diagram, a guided question and a check on the reasoning before it moves on. Learners can answer in voice or text, and the whole course follows the learner's language, including right-to-left scripts. Those features make a course easier to build around, which matters for the levers above, but they do not make a learner open the tutor on a bad day.
To be plain about the boundary, the Dilr Academy page does not state per-learner usage reporting to teachers, cognitive offloading detection or session time limits, and we do not claim them here. An institution piloting Dilr Academy should send us the same measures and data-export questions it sends every supplier, and judge the answers the same way. The product overview linked below explains it in more detail, and a scoping call is the fastest way to plan a pilot.
Frequently asked questions
Do students actually use AI tutors?
Students try AI tutors readily but use them far less than access suggests. In the 2026 NBER trial of Khanmigo, 96 percent of students tried the tutor at least once, yet the median student messaged it on only a third of practice days. In a 2026 Stanford-led trial, nearly half of pupils in the control group never used the AI platform at all.
Is there UK randomised evidence on AI tutor engagement?
We have not found a large UK randomised trial of AI tutor engagement. The main trials ran in Tennessee middle schools, Nigerian secondary schools and two school districts in a Stanford-led study. In June 2026 the Education Endowment Foundation opened a fund of up to 2.5 million pounds to research how generative AI affects learning and cognition, with first findings expected in 2027.
The EEF's research programme announcement sets out the scope. Until it reports, a UK institution is effectively running its own evidence base, which is one more reason to measure engagement properly in every pilot, and to write the pilot plan for an AI teacher platform around it.
Should a pilot set a target for minutes on the tutor?
A pilot should not set a minutes target for an AI tutor. Minutes measure exposure, not learning, and the DfE product safety standards expect learner-facing products to include default time limits. A better target is the help-at-mistake rate: the share of moments where a learner who got something wrong asked the tutor for help and explained their thinking.
Want to see this in a real course? Read what Dilr Academy is, compare it in our AI education tool guide, see our DATS methodology, or read about our approach to placing AI inside existing work. More on strategy sits in the strategy articles.
Pilot an AI tutor you can actually read.
30-min scoping call · No deck · Confidential. See Dilr Academy teach one of your own topics, and talk through how you would measure a pilot of it.
Written by the Dilr.ai engineering team, practitioners who ship enterprise AI in production. Follow us on LinkedIn for shipping notes, or subscribe via the RSS feed.