Model Collapse
A degradation phenomenon where AI models trained on AI-generated data progressively lose quality, diversity, and accuracy over successive generations. As more AI-generated content populates the internet, models trained on this synthetic data produce increasingly homogeneous and error-prone outputs. Model collapse is relevant for AI agents because agent-generated content (support responses, marketing copy, code) may eventually feed back into training data, creating quality feedback loops.
Example
A company uses AI to generate hundreds of product descriptions, which get indexed by search engines. Future AI models trained on web data learn from these AI-generated descriptions. Over time, product descriptions across the industry converge toward similar phrasing and style—reducing the distinctiveness that originally made them effective.
Frequently asked questions
- How do I prevent model collapse in my agent outputs?
- Ensure agent outputs are reviewed and edited by humans before being used as future training data. Maintain clean, human-curated datasets for any fine-tuning. Use diverse prompting strategies to avoid homogeneous outputs. For content generation, regularly audit output quality and diversity metrics. The risk is primarily for organizations that fine-tune models on their own agent outputs without human quality filters.