Is ChatGPT Study Mode Good? What AI Tutor Studies Found
- No published experiment has tested ChatGPT study mode itself, as far as this note could find.
- In a close test, classes with an answer-giving AI scored 17% lower on a later exam without AI.
- Classes with a hint-only AI tutor lost nothing on that exam, but they did not gain either.
- The AI tutors that worked had a teacher's worked solution for every problem. Study mode has none.
- Give study mode your own notes and solutions, then recall what you learned the next day without AI.
No published experiment has tested ChatGPT study mode itself, as far as this note could find. But AI tutors built on the same idea, hints instead of answers, have been tested in two trials. They prevented the damage an answer-giving AI did, and a well-built one taught more than a strong classroom lesson. Both of those tutors carried a teacher's worked solution for every problem, and both measured learning only on the same day.
What does ChatGPT study mode do differently?
Study mode guides you with questions and hints instead of handing over the answer. OpenAI launched it on July 29, 2025.
According to OpenAI's announcement, study mode runs on custom system instructions written with teachers, scientists, and pedagogy experts. It uses Socratic questions and hints, breaks material into steps, checks your level, and quizzes you to confirm understanding. You can switch it on and off within a conversation.
OpenAI also named a limit: because it runs on instructions, its behavior can be inconsistent across conversations and it can make mistakes. The announcement reports no measurement of what study mode does to learning or memory. So the best evidence comes from trials of AI tutors that work the same way.
Why is an answer-giving AI a problem for learning?
It raised practice scores, then lowered scores on an exam taken without AI.
Bastani and colleagues (2025) ran a field experiment in a Turkish high school with about 50 math classes and nearly 1,000 students. Each class did practice problems with textbooks only, with a standard ChatGPT-style AI (GPT Base), or with an AI told to give hints rather than answers (GPT Tutor). Then, in the same session, everyone took a closed-book exam on similar problems.
The answer-giving AI raised practice scores but lowered exam scores without AI
- Textbook only
- Answer-giving AI
- Hint-only AI
See the numbers
| Textbook only | Answer-giving AI | Hint-only AI | |
|---|---|---|---|
| Practice with AI | 28.4 | 42.1 | 64.5 |
| Exam without AI | 32.1 | 26.7 | 31.7 |
With the answer-giving AI, practice scores rose 48%, but exam scores came out 17% lower than in the textbook classes. Students used it as a crutch, asking for and copying solutions, and they did not notice that they had learned less. With the hint-only AI, practice scores rose 127% and exam scores were no different from the textbook classes.
Two things follow. The hint-style tutor prevented the harm. It did not add learning on the exam. Yet those students believed they had done better. Feeling that study mode is working is not the same as learning more. More on this study is in Is Using ChatGPT to Study Bad?
Does AI tutoring actually work?
A carefully built AI tutor worked well. In a Harvard physics course, students learned more than twice as much from it as from an active-learning class.
Kestin and colleagues (2025) ran a crossover trial with 194 students in an introductory physics course in fall 2023. For one week, students had an active-learning class with group problem solving. For the other week, they learned the same content at home with an AI tutor. They took a quiz before and after each lesson.
Quiz scores rose more after the AI tutor than after the active-learning class
See the numbers
| Median quiz score | |
|---|---|
| Baseline before | 2.75 |
| Active-learning class | 3.5 |
| AI tutor | 4.5 |
The AI group also spent less time: the class ran 60 minutes, and the AI group's median was 49 minutes. The authors estimated an effect size of 0.73 to 1.3 standard deviations. Older evidence points the same way. A review of tutoring studies found intelligent tutoring systems nearly as effective as human tutors.
Intelligent tutoring systems came out nearly as effective as human tutors
- Commonly believed
- Found in the review
See the numbers
| Commonly believed | Found in the review | |
|---|---|---|
| Human tutoring | 2 | 0.79 |
| Intelligent tutoring systems | 1 | 0.76 |
What did the AI tutors that worked have?
A teacher-written solution for every problem. Study mode has general instructions only, and no problem-specific solutions unless you supply them.
Both AI tutors that worked carried a worked solution, written by people, for every problem.
| Hint-only AI (Bastani et al.) | AI tutor (Kestin et al.) | ChatGPT study mode | |
|---|---|---|---|
| Problem solutions | Solution and suggested hints in the prompt | Step-by-step solutions in the prompt | None (you supply them) |
| How it guides | Hints, no direct answers | Walks through each part in order | Questions, hints, quizzes |
| What was measured | Exam in the same session | Quiz right after the lesson | No published results |
Bastani et al. (2025) noted that the standard AI often gave wrong answers, while the hint-only AI held the solution, so students could check their work against it. Kestin et al. (2025) added full solutions to avoid trusting answers the model invented. They also found that instructions alone could not walk students through multi-part problems in order, so the platform fed the parts one at a time. In our reading, giving study mode your textbook or answer key is the simplest way to close that gap.
Does what you learn in study mode last?
Nobody knows yet. Both trials measured learning on the same day.
The Bastani et al. (2025) exam came right after practice, in the same session. The Kestin et al. (2025) quiz came right after the lesson. Kestin et al. list adding retention strategies such as spacing as future work.
Tests right away and tests days later can disagree. In Roediger and Karpicke (2006), rereading beat recall practice on a test 5 minutes later, but on tests 2 days and 1 week later, students who had practiced recall remembered substantially more. The rereaders were also more confident. A smooth session in study mode can carry the same trap.
How should you use study mode?
Give it your materials, answer first, and recall the material the next day without AI. All of this works without study mode too.
- Hand over your notes and solutions. Paste lecture notes or the answer key and ask: "Use this solution, and give me hints only." That is what both successful tutors had.
- Write your own answer before asking. A wrong guess is fine. Take the hint after.
- Hold off once when you want the answer. Study mode can be switched off, so it is easy to grab the answer when stuck. If you do, close the screen and solve it again in your own words.
- Recall it the next day without AI. Pick three things from yesterday that you want to keep and explain them with nothing in front of you. Repeat a few days apart to make them last. Intervals are in Spaced repetition schedule.
In Aloom
If study mode helps with understanding on the day, Aloom takes over from the next day. You ask AI, keep only the answers worth keeping, and each one joins what you already know in your graph. Each day you recall 1, 3, or 5 of them from a question, before you see the answer. The more you recall, the darker the ink. It covers the days-later memory that the trials above did not measure. See how Aloom works
FAQ
Does ChatGPT study mode work?
No study has tested it directly. Trials of AI tutors built the same way found that hint-only guidance prevents the losses of answer-giving AI (Bastani et al., 2025), and that a tutor with expert-written solutions can beat a strong class on a same-day quiz (Kestin et al., 2025).
Is study mode different from asking ChatGPT for hints only?
In principle, no. Study mode also runs on system instructions. In the trials, what mattered was hint-first guidance and a worked solution behind it, not the name of a mode. Either way, give it your materials.
Can an AI tutor replace a human tutor?
The evidence is partial. In VanLehn (2011), intelligent tutoring systems (0.76) were nearly as effective as human tutors (0.79), but those were systems built before generative AI. Kestin et al. (2025) suggest using AI tutors to bring everyone up to speed before class, not to replace it.
References
- Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122.
- Kestin, G., Miller, K., Klales, A., Milbourne, T., & Ponti, G. (2025). AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting. Scientific Reports, 15, 17458.
- VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221.
- Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255.
- OpenAI. (2025). Introducing study mode.