Chat_O

Catastrophic Forgetting in LLMs: What happens when you retrain?

Catastrophic Forgetting in LLMs: What happens when you retrain?

Continuous learning in LLMs sounds like an obvious function: import new data, briefly retrain, update knowledge. This is exactly where the operational risk begins. Fine-tuning can improve performance on the new training data while degrading skills that were reliable before the update. This Catastrophic Forgetting does not arise from the Transformer architecture alone. It is…

Read More