AI companies have long boasted about the size and capabilities of their massive products, but recently the focus has been on smaller, more efficient models, which companies say will help cut costs and power consumption.
Programs like ChatGPT are based on algorithms known as “large language models,” and the chatbot’s creators boasted last year that the GPT-4 model contains nearly two trillion “parameters,” the building blocks of these models.
The massive size of the GPT-4 model allows ChatGPT to answer queries in fields from astrophysics to zoology, but if a company needs a program that knows something specific, like crocodiles, the algorithm could be much smaller.
As Laurent Felix of consulting firm Ekimetrics puts it, “You don’t need to know the terms of the Treaty of Versailles to answer a question about a particular element of engineering.”
Moving towards smaller models
Companies like Google, Microsoft, Meta, and OpenAI have started offering smaller models, and Amazon also makes all sizes of models available on its cloud platform.
At a recent event in Paris, Amazon's sustainability chief, Kara Hurst, said the shift shows the tech industry is moving toward "moderation and frugality."
Energy requirements for small models
Small forms are better suited for simple tasks such as summarizing or indexing documents, or searching an internal database.
For example, the American pharmaceutical company Merck is developing a model in collaboration with Boston Consulting Group to understand how some diseases affect genes. “It will be a very small model, between hundreds of millions and a few billion parameters,” said Nicolas de Bellefond, head of artificial intelligence at Boston Consulting Group.
These models have several advantages over larger models, said Laurent Daudet, head of French startup LightOn, which specializes in small models. “They are often faster and can respond to more queries and users at the same time,” he said. They also consume less energy, making them more environmentally efficient, which is one of the main concerns associated with AI.
Large models require huge clusters of servers to “train” AI programs and process queries, and these servers, made of highly advanced chips, require huge amounts of electricity to operate and cool. Small models require far fewer chips, making them less expensive and more energy efficient.
Small models do not need data centers.
He also stresses that the small models can operate without the need for data centers, as they can be installed directly on devices. “This is one way to reduce the carbon footprint of our models,” said Arthur Mensch, head of the French startup Mistral AI.
Laurent Felix pointed out that using models directly on devices also means more “security and confidentiality of data,” as programs can be trained using private data without fear of it being leaked.
However, larger models still have an advantage in solving complex problems and accessing a wide range of data, and de Bellefon said the future will likely involve different models communicating with each other.
“There will be a small model that understands the question and then sends this information to several models of different sizes depending on the complexity of the question,” he added. “Otherwise, we will face solutions that are either very expensive or very slow, or both.”