Musk: The AI industry has basically exhausted all available real-world training data

According to Techcrunch, Musk stated during a live conversation with Stagwell Chairman Mark Penn that the AI industry has basically exhausted all available real-world training data. He believes this change occurred last year and pointed out that future AI model training will rely on AI-generated synthetic data. Musk's viewpoint aligns with former OpenAI Chief Scientist Ilya Sutskever's remarks at the NeurIPS conference. Sutskever mentioned that the AI industry has reached a 'data peak,' and the data shortage will prompt the industry to change its model development approach. Currently, several companies, including Microsoft, Meta, and OpenAI, have begun widely adopting synthetic data for training. However, research also indicates that excessive reliance on synthetic data may lead to impaired model functionality or even 'model collapse.'