Studies on student engagement, including research reviews published in 2018 in the journal Social Behavior and Personality and in 2020 by the Australian Journal of Teacher Education, have shown its importance for academic success. Although AI has many benefits, Nitta says, “it’s not very good at motivating you to keep doing something you’re not really interested in.”
“The elephant in the room with all these chatbots is: how long is someone going to interact with them? he said. He claims that during Watson’s trials, students ignored his attempts to engage with them.
In a TED talk in spring 2023, shortly after Khanmigo launched, Khan Academy founder and CEO Sal Khan highlighted that tutoring has brought some of the greatest impacts on student performance among educational interventions studied. But there aren’t enough tutors available or enough money to pay them, especially in the wake of pandemic-induced learning loss.
Kristen DiCerbo, Khan Academy’s chief learning officer, was vice president of research and learning design for educational publisher Pearson in 2016, when the publisher partnered with IBM on Watson, which she describes as “a different technology” that relied heavily on scripted responses, as opposed to the unscripted interactions students can have with generative AI.
Khanmigo does not answer students’ questions directly, but begins with questions of his own, such as asking if the student has any ideas on how to find an answer. Then he guides them toward a solution, step by step, with advice and encouragement.
Despite Khan’s expansive vision of “amazing” personal tutors for every student on the planet, DiCerbo assigns Khanmigo a more limited teaching role. When students are working independently on a skill or concept but find themselves stuck or stuck in a cognitive rut, she says, “we want to help students get out of that impasse.”
Some 100,000 students and teachers piloted Khanmigo over the past school year in schools across the country, helping detect the robot’s hallucinations and providing tons of conversations between students and robots for DiCerbo and his team to analyze.
“For example, we try to summarize, give advice and encourage,” she explains.
It is not yet clear to what extent Khanmigo has closed the AI engagement gap. Khan Academy plans to release summary data on student-robot interactions later this summer, according to DiCerbo. Projects to use third-party researchers to evaluate the impact of the tutor on learning will take more time.
AI feedback works both ways
Since 2021, the non-profit organization Saga Education has also been experimenting with AI feedback to help tutors better engage and motivate students. Working with researchers at the University of Memphis and the University of Colorado, the Saga pilot team in 2023 fed transcripts of their math tutoring sessions into an AI model trained to recognize when the tutor prompted students to explain their reasoning, refine their answers or engage in more in-depth discussion. The AI analyzed how often each guardian took these steps.
Tracking some 2,300 tutoring sessions over several weeks, they found that tutors whose coaches used AI feedback peppered their sessions with many more of these prompts to encourage student engagement.
