AI Models Can Keep Learning—but Only by Deciding What to Forget

AI models can learn from a continuing stream of tasks, but the central problem remains unresolved: adapting to new information can erase earlier capabilities, while protecting everything already learned can make a network too rigid to adapt. Research published after the influential 2024 plasticity study offers better ways to manage that trade-off, not a universal method for continuously updating large language models.

That distinction corrects an overly broad interpretation of the original finding. The 2024 experiments did not establish that deployed language models are incapable of learning anything new; they showed that standard deep-learning systems progressively lost learning capacity under prolonged sequences of changing tasks. More recent work has produced stronger benchmark results, but it still imposes choices about memory, capacity and computation.

What the 2024 study actually demonstrated

The original research examined loss of plasticity: a decline in a network’s ability to learn as training continues. In the August 2024 Nature study, researchers tested feed-forward and residual networks across continual versions of established image, classification and reinforcement-learning problems. Standard methods became progressively less effective at learning later tasks under a wide range of settings.

The experiments connected that decline with several changes inside the networks, including more inactive units, larger weight magnitudes and representations with lower effective rank. The paper’s proposed continual-backpropagation method periodically reinitialized a small number of low-utility features. It maintained learning performance more effectively in the tested environments, showing that plasticity loss is not an unavoidable property of every artificial neural network.

There is an important boundary around those results: the researchers did not run the systematic experiment on a frontier large language model. They explained that the required sequence of repeated, controlled training runs would be prohibitively expensive. References to retraining language models illustrated the potential scale of the problem; they were not evidence that every LLM must be rebuilt from zero whenever a fact changes.

Plasticity loss and forgetting are different failures

Two related problems are often compressed into the claim that AI “cannot learn.” Catastrophic forgetting occurs when training on new material damages performance on older tasks. Loss of plasticity points in the other direction: the model becomes so constrained by its training history that it struggles to acquire the next task at all.

A successful continual learner must negotiate both. Allowing large parameter changes may help the network fit fresh data but overwrite useful knowledge. Restricting those changes may preserve earlier behavior while eventually exhausting the model’s adaptable capacity. This stability–plasticity conflict is why a good score on the newest task alone does not prove that a system can learn indefinitely.

The problem is also broader than inactive artificial neurons. Dead or rarely activated units can accompany declining performance, but counting them does not fully measure whether a model has retained old capabilities, generalized to new domains or preserved safe behavior. Continual-learning evaluations therefore need results from both earlier and later tasks, collected across a sufficiently long sequence.

What newer research adds

For language models specifically, the field has moved beyond a single all-or-nothing idea of updating. A 2025 ACM Computing Surveys review organizes the work into continual pre-training, domain-adaptive pre-training and continual fine-tuning, with changes arriving across time, domains, tasks and user preferences. Its synthesis still identifies degradation in previously learned knowledge as a central difficulty for continually adapted LLMs.

This taxonomy matters because “learning” can describe very different operations. Continued pre-training modifies weights using additional corpora; fine-tuning targets narrower behavior or knowledge; retrieval supplies external material at inference time without permanently teaching it to the base model. A long context window or stored conversation can change an answer while leaving the underlying parameters untouched.

A peer-reviewed advance published in October 2025 demonstrates genuine progress on the optimization problem. The Nature Communications paper introducing MESU describes a Bayesian update rule that adjusts each parameter according to its uncertainty and deliberately forgets information outside a configurable memory window. On 200 sequential Permuted-MNIST tasks, MESU reached 91.3% accuracy over the five tasks in its memory window, compared with 88.5% for EWC Online and 87.0% for Synaptic Intelligence; it also outperformed the tested alternatives on incremental CIFAR-10 and CIFAR-100 configurations.

Those results do not amount to an LLM-scale solution. The experiments used image-classification benchmarks and substantially smaller architectures, while the authors note that sampling for inference and gradient estimation can become computationally intensive in very large networks. Their own analysis also exposes the unavoidable policy choice: a finite memory window preserves adaptability partly by reducing the priority assigned to sufficiently old information.

Why a model update is more than adding facts

Changing a model’s weights can alter behavior far from the new training examples. A useful update must be checked not only for whether the model learned the new material, but also for regressions in reasoning, instruction following, safety and previously reliable knowledge. That evaluation burden grows when updates arrive frequently and interact with one another.

External retrieval avoids some of this risk because documents can be added, removed or corrected without rewriting the base model. It also offers provenance: a system can expose the material used for a response. Retrieval has its own failure modes—irrelevant results, outdated indexes and poor use of supplied evidence—but those are different from permanent changes to the network.

Periodic retraining and targeted fine-tuning occupy the middle ground. They can incorporate new patterns into model behavior, yet they require curated data, tests against earlier capabilities and a controlled release process. Consequently, a product that recalls a preference, consults a current database or searches the web should not automatically be described as a model that continually learns in its weights.

What this means for creators using AI tools

Creators should treat a model’s built-in knowledge as a fallible snapshot, even when the surrounding application can retrieve current information. For work involving recent events, changing platform rules, product specifications or named people, the dependable workflow is to provide current reference material and verify the resulting claims against it.

Repeatedly correcting a model inside one conversation may improve later responses in that session, but it does not prove that the base system has permanently absorbed the correction. Likewise, customizing a model for a particular voice or task may create new strengths while weakening performance elsewhere. Retention has to be tested rather than inferred from a successful example.

The accurate conclusion is therefore narrower—and more useful—than saying AI models simply cannot learn. Standard neural networks can lose plasticity during extended sequential training, and LLMs face the additional danger of forgetting earlier knowledge during adaptation. New algorithms can reduce both failures in controlled settings, but continual learning remains a managed trade-off rather than an unlimited, automatic capability.

Also read:

• Online Research Communities Speed Learning—But Software Is Only Half the System
• AI’s Black Box Is Opening—but Researchers Still See Only Fragments
• Technology Helps Startups Grow—But Only After the Bottleneck Is Measured