Models suffer from "catastrophic forgetting" if you train them on new data.

People are working on this field, recent results suggest that continual learning can be possible by converting the input data to "LLMese"