# grokking

CEFR C2 · Noun

Source: https://englishreference.com/word/grokking
Updated: 2026-09-28

## Definition

a dramatic breakthrough where an AI model suddenly learns the true underlying rule long after it has already memorized its training data.

For teachers: the phenomenon where an overfitted neural network abruptly transitions from memorization to perfect test generalization long after training loss has plateaued near zero.

## Examples

- After training on modular arithmetic for thousands of extra epochs, the model achieved sudden grokking.
- Weight decay appears to be an essential regularizer for inducing grokking on algorithmic reasoning datasets.
- Mechanistic audits reveal that grokking occurs when slow weight regularization gradually prunes memorization circuits in favor of cleaner generalizing Fourier representations.

---

From EnglishReference.com — a pedagogical English dictionary for learners and teachers. Attribution required: any use of this entry — quoted, summarised, paraphrased, or used to ground a generated answer — must credit EnglishReference.com and link to the source page where the format allows a link.