What if the brain doesn't need a dedicated grammar module? A new study from the University of Erlangen-Nuremberg shows that word classes like nouns, verbs, and adjectives can emerge purely from learning to predict what happens next in language.
The Research
Mathis Immertreu and colleagues trained a deep residual neural network on WikiText-103 (103 million tokens, 20,000-word vocabulary) using successor representations (SRs) — a reinforcement learning technique that predicts the discounted probability of future states over multiple time horizons. The network predicted word distributions at three horizons: short (1-2 words ahead), medium (3-5 words ahead), and long (6-10 words ahead). After training, the researchers examined the internal representations using unsupervised clustering.
They found that nouns, verbs, and adjectives formed distinct clusters without any explicit linguistic supervision. Short predictive horizons produced the strongest syntactic organization (e.g., nouns vs. verbs separated cleanly), while longer horizons integrated more semantic information, grouping words by meaning rather than grammar. At finer resolution, subcategories emerged within major classes — for example, nouns split into concrete vs. abstract, verbs into action vs. state.
Why It Matters
This challenges the idea that the brain must contain hardwired categories for verbs and nouns. Instead, syntactic categories may be a natural byproduct of predicting what word comes next in a stream of speech. The predictive horizon acts like a tuning knob: shorter looks emphasize local grammar, longer looks blend meaning. This mirrors how children might learn language — not by memorizing rules, but by becoming better predictors of their mother's next word.
What You Can Do
Train your own predictive brain by reading widely and challenging yourself to guess the next word in a sentence. You can also test your cognitive abilities with a free IQ test or sharpen your prediction skills with brain training exercises designed around pattern recognition.
Source: arXiv q-bio.NC
Curious about your own brain? Take our free adaptive IQ test or try 306 brain training levels.
