한국어

How to Improve Metacognition: Judge Later, by Recall

At a glance
  • In studying, metacognition is judging what you know and what you don't, so you know what to study next.
  • Judgments made right after studying are often wrong. Made a little later, they become far more accurate.
  • Across many studies, judging later raised accuracy a lot (g = 0.93) while memory itself barely changed (g = 0.08).
  • Rereading raises confidence, yet a week later readers remembered less than students who practiced recall.
  • Close the material, recall from the question alone, then judge. Restudy what you could not recall first.

The surest way to improve metacognition is to change when and how you judge what you know. Instead of trusting the "got it" feeling right after studying, wait a little, close the material, try to recall the answer from the question alone, and judge from that. In study after study, simply judging a little later made those judgments far more accurate, while memory itself barely changed.

What is metacognition in learning?

In learning, metacognition is your judgment of what you know and what you don't. In a wider sense it covers planning, monitoring and evaluating your work. This note focuses on the judgment you make most often: should I go over this again, or move on?

When that judgment is wrong, study time goes to the wrong place. You review what you already know and skip what you only think you know. So researchers measure metacognition as how well your confidence matches what you later remember. The value used in this note is G (gamma). A G of 1 means the items you were more confident about were always the ones you remembered. A G of 0 means confidence and memory had nothing to do with each other.

Why are judgments right after studying so often wrong?

Because the answer you just saw is still in your head, so it feels known whether or not it will last. Judging by that feeling, you cannot tell what you will forget tomorrow from what will stay.

Nelson and Dunlosky (1991) had students study word pairs and, for each pair, rate how likely they were to recall the second word from the first on a later test. Some judgments were made immediately after studying a pair. Others were delayed until shortly after study and made with only the first word shown. Judgments like these had rarely predicted recall well, but the delayed ones were extremely accurate. The authors named it the delayed-JOL effect (JOL stands for judgment of learning).

This is not one lucky experiment. Rhodes and Tauber (2011) pooled the studies on the delayed-JOL effect in two meta-analyses. Across 4,554 participants and 112 effect sizes, delaying the judgment raised accuracy by a lot (g = 0.93). A second analysis, of 3,807 participants and 98 effect sizes, found that memory itself improved only a little (g = 0.08). The power of waiting lies less in remembering more than in knowing what you know.

Judging later raised accuracy a lot and memory only a little

00.250.50.751Effect of judging later (g) · Judgment accuracy · 0.930.93Judgment accuracyEffect of judging later (g) · Memory itself · 0.080.08Memory itselfMemory barely moved
Average effect of delaying the judgment (Hedges' g), pooled across studies. 0.2 is usually read as small and 0.8 as large. Accuracy is how well confidence matched later recall (112 effect sizes, 4,554 participants), and memory is how much was recalled (98 effect sizes, 3,807 participants).Source: Rhodes & Tauber (2011), abstract
See the numbers
Effect of judging later (g)
Judgment accuracy0.93
Memory itself0.08

Metacognition is not only a trait you are born with. The same person judges far more accurately depending on when they judge.

Why does rereading make you feel you know more?

Karpicke and colleagues (2009) suggest one reason: each reading makes the text easier to process, and students may mistake that ease for learning. Confidence goes up, but what lasts does not follow.

In the second experiment of Roediger and Karpicke (2006), 180 undergraduates studied a prose passage across four periods: 5 minutes for each reading, 10 minutes for each recall. One group spent all four periods reading it. One read in three periods and used the last to write down what they could recall. One read in the first period and spent the other three recalling it. At the end they rated, on a 7-point scale, how well they would remember it in a week.

The most confident group remembered the least a week later

0%20%40%60%80%100%Recall after 1 week · Read 4 times · 40%40%Read 4 timesRecall after 1 week · Read 3 times recalled once · 56%56%Read 3 timesrecalled onceRecall after 1 week · Read once recalled 3 times · 61%61%Read oncerecalled 3 timesConfidence was highestin the group that only read
Share of the passage's idea units recalled on the final test one week later (%). 30 students per condition.Source: Roediger & Karpicke (2006), Experiment 2, Final Tests section
See the numbers
Recall after 1 week
Read 4 times40%
Read 3 times recalled once56%
Read once recalled 3 times61%
How they studiedWill remember in a week (1-7)Recall after 5 minRecall after 1 week
Read 4 times4.883%40%
Read 3 times, recalled once4.278%56%
Read once, recalled 3 times4.071%61%

Five minutes later, the rereaders were indeed ahead. But the question they had answered was about a week later, and a week later they were behind. The feeling that you understand more deeply than you do, until you try to explain it, is also covered in Does the Feynman technique work?.

How do students actually study?

Most reread, and few test themselves. The most popular method is the one that feeds the illusion.

Karpicke and colleagues (2009) asked 177 college students to list the strategies they use when they study.

Eight in ten reread, and one in ten practiced recall

0%20%40%60%80%100%Students who listed it · Rereading · 83.6%83.6%RereadingStudents who listed it · Practice problems · 42.9%42.9%PracticeproblemsStudents who listed it · Flashcards · 40.1%40.1%FlashcardsStudents who listed it · Practice recall · 10.7%10.7%Practicerecall1.1% ranked recallas their top strategy
Share of students who listed each strategy (%). Students listed 2.9 strategies on average.Source: Karpicke, Butler & Roediger (2009), Table 1
See the numbers
Students who listed it
Rereading83.6%
Practice problems42.9%
Flashcards40.1%
Practice recall10.7%

Rereading was the top strategy for 54.8% of students. Asked whether, after reading a chapter once, they would rather restudy it, try to recall it, or do something else, 57% of the 101 students who got that question chose restudying and only 18% chose recall. The authors suggest students experience illusions of competence while studying, and that these illusions steer the strategies they pick.

Does better metacognition actually raise test scores?

One experiment found that more accurate judgments led students to restudy the right texts, and their scores went up. A judgment is not the goal. It is the tool that picks your next study.

Thiede and colleagues (2003) had 66 students read six expository texts taken from encyclopedias. One group wrote five keywords for each text after reading all six. One wrote keywords right after reading each text. One wrote none. Then everyone rated how well they understood each text, took a test, and chose which texts to reread.

GroupPicked less-learned texts to reread (G)Second test, repeated questions
Keywords later, after all six0.7969%
Keywords right after each0.3553%
No keywords0.3656%

On the first test, the three groups were about the same. The later-keyword group judged more accurately which texts they did not understand, chose those to reread, and pulled ahead only on the second test. New questions showed the same pattern: 66% against 51% and 55%. The key, again, is "later". The same keyword task done right after reading did not bring this advantage.

How to improve metacognition, starting today

Move your judgment from right after studying to a little later, and base it on what you can recall, not on how it feels. You need a notebook and a pen, not Aloom.

  1. Write questions as you study. After each chunk, write one question about it in the margin (for example, "Why does spacing make things last?"). Do not write the answer.
  2. Judge after studying something else. Do not tick "got it" on the spot. A few minutes later, or after the next section, look only at the questions and try to recall the answers.
  3. Mark the result, not the feeling. Recalled fully: a check. Partly: a triangle. Not at all: a cross. This is your record of what you know.
  4. Restudy the crosses and triangles first. Do not spend review time on the checks. Tomorrow, judge again with the same questions.

The same holds when you study with AI. The "oh, I get it" after reading a ChatGPT explanation is a judgment made right after studying. Do not move on there. The next day, with the chat closed, answer the same question. What comes out is what you know. It is also why you should not drop the flashcards you feel you have learned, as covered in Are flashcards effective?.

In Aloom

Aloom's review follows the same way of judging. You ask AI, keep only the answers worth keeping, and each one joins what you already know. From the next day you recall 1, 3 or 5 of them a day, whichever you chose, from the question first, before you see the answer. Whether you know something is decided by what you recall after time has passed, not by the feeling right after reading. The more you recall a node, the darker its ink, so your graph shows what you can bring back apart from what you have only read. See how Aloom works

FAQ

What is metacognition in learning?

It is stepping back to look at your own thinking and knowledge. In learning it mostly means judging what you know and what you don't, and using that judgment to decide what to study next.

How can I improve metacognition as an adult?

The studies here were run with college students, and none of them looked at age. But the method needs nothing special: judge later instead of right away, and judge by trying to recall from a cue rather than by how familiar the material feels. Then restudy what you could not recall. Adults studying alone can do this with a list of questions.

Is there a test for metacognition?

For studying, you can run one yourself. For each thing you learn, write down how likely you think you are to recall it later, then check a few days later by actually recalling it. If the items you were surer about are the ones you recalled, your judgments are accurate. Researchers measure that match with G.

References