ChatGPT collapse: Know how quantum computing can save it

What Happens When AI Runs Out of Real Human Data? With less data to learn from, Large Language Models (LLMs), like ChatGPT, have become smarter by learning from huge amounts of real human-written content. But researchers now warn that if in the future, AI runs out of new useful data by 2026.
A 2024 report by watchdog group Epoch AI says that there may soon not be enough fresh human content left to train the next generation of AI models. Even Elon Musk has said AI may have already used up most of the internet’s quality knowledge.

AI models grow stronger by learning from diverse, high-quality content. But the question arises: if that content dries up, how will AI continue to improve?

Can AI train itself with fake data?

One idea is to use synthetic data. This means generating new, artificial content using existing AI. For example, Amazon used this method to train customer service tools, creating questions and answers with AI and checking them for quality.

In simple tasks or where real data is structured like spreadsheets, synthetic data can work well. But when it comes to complex or creative tasks, using AI to train AI can lead to serious problems.

Researchers warned of “model collapse”, where AI trained on fake data starts to repeat the same errors and loses creativity, and it will most likely to produce biased results. SAP’s Mohan Shekar calls this “model incest”; each new model becomes worse by copying its earlier mistakes.

Innovation in AI training:

While some researchers also believe that changing how AI is trained may matter more than how much data it uses. This includes new ideas like multimodal learning, where AI learns from not just text but also images, audio, and video.

For example, instead of recording thousands of people talking in the rain, engineers can combine voice data with sound effects of rain to teach AI how to listen in tough conditions.

Even more promising is quantum computing, which could help AI learn from large amounts of unstructured data, like raw images, video, or audio files that don’t have labels. That could unlock entirely new ways for AI to learn without needing perfect human-written text.

What does this mean for the future of AI?

If AI runs out of fresh, real data, its ability to stay accurate, unbiased, and most importantly, innovative would suffer. However, new training methods, better tools, and technologies like quantum computing are helping to solve this problem.