What is catastrophic forgetting in ai training

What Catastrophic Forgetting Actually Refers To

Catastrophic forgetting refers to an AI model losing previously learned capability when it undergoes additional training on new information, a phenomenon named for how suddenly and significantly this capability loss can occur relative to the seemingly modest additional training that caused it.

Why This Happens at a Technical Level

This happens because the same internal parameters a model uses to store its existing learned knowledge get adjusted during subsequent training on new information, and without careful management, this adjustment process can overwrite or degrade previously learned patterns rather than cleanly adding new capability alongside what already existed.

Why This Represents a Genuine Risk During Fine-Tuning Specifically

This risk becomes particularly relevant during fine-tuning, where a foundation model with broad general capability undergoes additional targeted training for a specific task, creating real tension between gaining strong performance on that new specific task and inadvertently degrading the model’s original broader general-purpose capability.

How Companies Actively Manage This Risk

Several mitigation techniques exist to manage this risk, including careful control over how aggressively a model’s parameters are adjusted during fine-tuning, and periodically testing a model’s broader general capability throughout the fine-tuning process to catch meaningful degradation before it becomes severe enough to matter.

Why This Remains a Genuine, Ongoing Consideration Rather Than a Fully Solved Problem

Despite these available mitigation techniques, catastrophic forgetting remains a genuine, ongoing consideration companies have to actively manage rather than a fully solved problem, since completely eliminating this tradeoff while still achieving meaningful specialization through fine-tuning remains a genuinely difficult balance to strike perfectly.

Bottom Line

Catastrophic forgetting describes an AI model losing previously learned capability when trained on new information, since the same parameters storing earlier knowledge get overwritten, a genuine risk companies actively manage during fine-tuning through careful training techniques rather than a fully solved, eliminated problem.

Look Up AI Terms

Search plain-English definitions of AI and machine learning terms in our free AI Glossary.

Go deeper

• Can You Fine-Tune an Open-Source AI Model Yourself, and What Does That Take?
• What does training a model actually mean at a basic level?
• What’s the Difference Between Prompting an AI and Fine-Tuning It?