LLM Memorization and What To Forget

LLM Memorization and What To Forget

Language models have a pronounced tendency to memorize long passages of particular texts in their training data. For example, one of the OpenAI APIs could potentially regurgitate most of a chapter of a Harry Potter book from just a paragraph if prompted. While this capability can be impressive, there are many contexts in which such memorization is non-desirable.

Understanding how memorization arises during the course of training, and identifying what influences which data points get memorized, is critical. This knowledge can help design better language models.

In the context of memorization, it's not that we want models to avoid memorizing altogether. Certainly, remembering facts like “what two plus two is” is useful. However, there are specific kinds of data that should not be memorized, such as personal and private information. Even true information could become problematic if frequently repeated without context.

Being able to understand which data points get memorized, and why, can hopefully enable us to design models that selectively memorize useful information while avoiding the repetition of undesirable data.

Keywords

• Language models
• Memorization
• Training data
• OpenAI API
• Contexts
• Personal information
• Private information
• Selective memorization

FAQ

Q: Why is memorization by language models an issue? A: Memorization by language models can lead to the unintended regurgitation of long passages from the training data, which can include sensitive or private information.

Q: What influences which data points get memorized by language models? A: The influences include the nature of the training data and the specifics of how the model was trained.

Q: Is it always undesirable for language models to memorize data? A: No, memorization can be useful for factual information, like basic arithmetic. The challenge is ensuring that only the appropriate data is memorized.

Q: How can we design better language models concerning memorization? A: By understanding how and why certain data points get memorized, we can improve the training process to ensure models selectively memorize useful information while avoiding undesirable data retention.