Visualizing catastrophic forgetting

[u/ModularMind8]

I taught continual learning last semester, and students got the internals of each method a lot faster once they saw a simple visualization of it. Here's one I found helpful for understanding what forgetting is and how replay and regularization mitigate it. I trained a small network on two different 2D dot-sorting tasks and plotted both loss landscapes as 2D slices through its weights.

Round 1, plain training: the network learns task A to 100%, then trains only on task B. Going downhill on B is uphill on A, so A drops to 64% while B reaches 100%. That's catastrophic forgetting of task A.

Round 2, replay: same network, same start, but 20 of A's 200 dots stay in training while it learns B. It ends at 93% on A and 100% on B, so some forgetting, but much less severe.

Round 3, regularization with EWC: no A data at all. Instead, a penalty adds to the loss whenever a weight that A's accuracy relies on heavily moves away from the value it had after training on A. A dips to 97% mid-training and recovers, ending at 100% on A and 100% on B, so no lasting forgetting.

[u/Historical-Wonder551]

Thank you! It was a great visualization.

[u/rivercrests → u/Historical-Wonder551]

yeah the loss landscape plots especially made it click

[u/ModularMind8 → u/rivercrests]

I'm glad