The Economics of Bigger AI: How Much Scale Is Enough?

The AI industry continues to invest billions in larger models and computing infrastructure. At the same time, smaller models are becoming more capable, cheaper to operate and increasingly practical for specialized applications. These developments raise questions about where AI investment creates value, and who ultimately benefits from it.

In 2024, researchers at Epoch AI estimated that the hardware and energy costs of training frontier AI models had been growing by approximately 2.4 times annually since 2016. If that trajectory continued, they projected that the largest training runs could exceed $1 billion by 2027.

The figures are striking, although training represents only part of the investment required to develop and operate frontier AI systems. Research, data preparation, engineering, infrastructure and deployment add considerably to the bill.

The scale of investment reflects a conviction that has shaped AI development for much of the past decade: more computing power, more data and larger models will produce more capable systems. So far, that conviction has delivered impressive results. Models have progressed from predicting the next word in a sentence to writing software, analyzing complex documents, interpreting images and solving increasingly difficult problems.

Yet the economics of AI are beginning to change. Smaller models are achieving performance levels that once required substantially larger systems. Researchers are developing more efficient architectures, while the cost of using AI models has fallen dramatically.

For organizations deciding how to invest in AI, this creates a more interesting set of choices. The largest model may offer the greatest range of capabilities, but its additional performance comes at a cost. Whether that cost is justified depends on what the organization actually needs the technology to accomplish.

The economics of scaling

The relationship between computing power and AI performance has been studied extensively. Researchers have identified what are known as scaling laws: empirical relationships between the resources devoted to training a model and improvements in its performance. Three factors have been particularly important: the number of parameters in a model, the amount and quality of training data, and the computational resources used during training. Parameters are numerical values adjusted as a model learns from data. They determine how information is processed and contribute to the model's ability to represent complex patterns. Larger models generally have greater capacity, although size alone provides an incomplete picture of performance.

Research by OpenAI in 2020, followed by DeepMind's Chinchilla work in 2022, helped establish how model size, training data and computational resources interact. The Chinchilla findings were particularly important because they demonstrated that many large models had been trained with insufficient data relative to their size. A smaller model trained on more data could outperform a larger one using a comparable training budget. This finding has significant economic implications. The efficient use of computational resources matters as much as the quantity available.

Scaling laws also describe diminishing returns. Improvements in model performance generally require progressively larger increases in computational investment. Whether those improvements justify the expense depends on their commercial or practical value. A model that performs marginally better on a benchmark may require substantially more computing power. That improvement may be valuable for scientific research, complex software development or advanced reasoning. It may contribute very little to an organization that needs to classify documents, extract information or answer questions from a defined knowledge base. The distinction between technical capability and economic value deserves more attention in discussions about AI progress.

Why frontier AI is becoming an industrial concentration

The cost of training frontier models has implications for competition. Developing these systems requires access to specialized processors, large-scale data centers, reliable electricity, sophisticated engineering and highly skilled researchers. The investment extends beyond a single training run. Companies must conduct experiments, evaluate model performance, develop safety systems and maintain the infrastructure needed to serve customers. These requirements favor organizations with access to substantial capital and computing resources.

The consequences extend well beyond the companies developing the models. Businesses, governments and nonprofit organizations increasingly build their own applications on top of AI systems provided by a relatively small number of technology companies. These relationships can be commercially sensible. Accessing an advanced model through an API allows an organization to benefit from years of research and substantial infrastructure investment without having to reproduce either. But the arrangement also creates dependencies.

Providers determine the availability of their models, establish pricing, set technical conditions and decide when particular versions will be retired. Organizations that build important processes around these services must account for the possibility that costs, capabilities or contractual terms will change.

An organization can own the software it develops while remaining dependent on an external provider for the intelligence that powers it. This distinction matters increasingly as AI moves from individual productivity tools into core operational systems. A change in model pricing is inconvenient when employees use AI occasionally to draft documents. It becomes a significant operational consideration when hundreds of automated processes depend on that model every day. The economics of scaling therefore influence the distribution of technological control across the wider economy.

Smaller models are becoming more interesting

Stanford University's 2025 AI Index documented one of the clearest examples of improving model efficiency. In 2022, the smallest model achieving a score above 60% on the Massive Multitask Language Understanding benchmark, or MMLU, contained 540 billion parameters. By 2024, Microsoft's Phi-3 Mini reached the same threshold with just 3.8 billion parameters.

That represents a 142-fold reduction in model size. The comparison concerns a specific benchmark, and equivalent scores do not imply equivalent performance across every task. Nevertheless, the improvement demonstrates how quickly the relationship between model size and capability has evolved. Several technical developments help explain this progress.

Better training data can improve model performance without requiring proportional increases in size. Knowledge distillation allows smaller models to learn useful patterns from larger systems. Improvements in training methods and model architecture make more effective use of available computational resources. The result is a growing range of smaller models capable of performing specialized tasks at lower operating costs. This is particularly relevant for organizations working with relatively predictable processes or well-defined bodies of knowledge.

Consider an international development organization that wants to analyze project reports, identify recurring implementation challenges and retrieve evidence from previous programmes.

A large general-purpose model could perform these tasks. It might also provide sophisticated reasoning across unfamiliar material.

However, a smaller model combined with a well-designed document retrieval system could offer comparable performance on selected tasks at a lower operating cost. The organization could evaluate it against its own documents, adapt it to its terminology and establish rules governing the information it can access.

The specialized approach would still require development, evaluation and maintenance. Its performance might decline when confronted with unfamiliar tasks. The economic case would depend on the volume of work, required accuracy and costs of maintaining the system. But the organization would have a genuine choice of architectures, rather than treating access to the largest commercial model as the starting point for every application.

The price of intelligence is falling

Another development is changing the economics of AI adoption: the cost of inference.

Training is the process through which a model learns from data. Inference occurs when the trained model processes a request and produces an output. Every question submitted to an AI assistant, every document summarized and every automated workflow involving a model generates inference costs. Stanford's 2025 AI Index reported that the price of querying a model performing at approximately GPT-3.5's level on MMLU fell from $20 per million tokens in November 2022 to $0.07 by October 2024. That represents a reduction of more than 280 times in less than two years.

For organizations that previously considered AI prohibitively expensive, this changes what is financially possible. Yet the price per token tells only part of the story. A system that uses an inexpensive model may require multiple attempts to complete a task. An AI agent might call several models, retrieve documents, run code and revise its outputs before producing a final result. Each step consumes resources.

Conversely, a more expensive model might complete a difficult task accurately with fewer attempts and less human intervention. The relevant measure is the cost of completing a task to an acceptable standard, including the time required for supervision, verification and correction.

This is familiar territory for anyone who has managed an organizational budget. The cheapest input does not necessarily produce the lowest overall cost. AI procurement should be subject to the same scrutiny as any other investment in operational infrastructure.

The architecture matters as much as the model

The discussion about AI performance often focuses on comparisons between individual models. Yet an AI application involves considerably more than the model itself. A system may combine a language model with a document database, retrieval mechanisms, external tools, structured workflows and human review. The way these components interact can have a substantial effect on performance. Retrieval-augmented generation, commonly called RAG, provides a useful example. Rather than expecting a language model to contain all the information needed to answer a question, the system retrieves relevant material from an external knowledge source and supplies it to the model. This allows organizations to work with current or proprietary information without necessarily retraining a model. It can also improve the traceability of answers, since the system can identify which documents were retrieved. However, retrieval does not guarantee factual accuracy. The model can still misinterpret evidence, overlook important context or generate unsupported conclusions.

Other architectures use several models for different purposes. A smaller model might classify incoming requests or extract structured information, while a more capable model handles difficult reasoning tasks. Some systems use mixture-of-experts architectures, in which only selected parts of a large model are activated for a particular input. This allows substantial overall model capacity while reducing the amount of computation required for each operation.

These developments complicate the assumption that greater capability must always come from making every model larger. They also create opportunities for organizations to design systems around specific requirements for performance, cost, speed, reliability and control. The practical challenge is knowing which combination of technologies produces the most useful result.

The next phase of scaling

Continued investment in frontier AI remains economically understandable. Large general-purpose models offer capabilities that smaller specialized systems may struggle to reproduce. They can handle unfamiliar tasks, integrate information across domains and perform increasingly complex reasoning. Researchers are also exploring ways to improve performance by allocating more computation during inference.

This approach, often called test-time compute or inference-time scaling, allows models to devote additional computational effort to difficult problems. Rather than relying entirely on what was learned during training, a system can spend more resources generating, testing or refining possible solutions. The approach has produced improvements in certain reasoning tasks, although the additional computation increases costs and may introduce delays.

This development also changes the economics of AI scaling. Computing resources are increasingly allocated both to creating a model and to the reasoning it performs after deployment. At the same time, advances in frontier models can contribute to the development of smaller systems. Distillation and other techniques allow some capabilities developed through expensive research to be transferred into more efficient models. The two approaches are connected. Research at the frontier can expand what is technically possible, while work on efficiency can make those advances accessible to a wider range of users. The resulting AI market may become increasingly differentiated, with organizations choosing between general-purpose models, specialized systems and combinations of the two.

What this means for national AI strategies

The economics of scaling present governments with difficult investment decisions. Countries seeking to develop domestic AI capabilities must decide how to allocate resources among computing infrastructure, research institutions, model development, skills, data infrastructure and support for local companies. Building a competitive frontier model requires considerable financial and technical capacity. For countries with smaller economies or limited computing infrastructure, the opportunity costs can be substantial.

The alternative is to invest in capabilities that support a wider domestic AI ecosystem. Public computing facilities could provide researchers and smaller companies with access to resources they cannot afford independently. Investment in local-language models could support public services and commercial applications poorly served by existing systems. Shared datasets, evaluation infrastructure and open research could reduce barriers to entry.

These investments would still require technical expertise, sustained funding and institutional coordination. Their value would depend on whether they support applications that people and businesses actually use. There is also a distinction between technological sovereignty and technological self-sufficiency. A country can exercise meaningful control over its AI infrastructure while continuing to rely on international suppliers for certain components. The relevant questions concern the importance of those dependencies, the availability of alternatives and the country's ability to negotiate or change its technological arrangements. For governments, an AI industrial strategy should therefore examine which capabilities justify domestic investment and which can be accessed more efficiently through international cooperation. The objective is to develop an economy capable of making informed technological choices and sustaining the institutions required to implement them.

A different calculation for nonprofits

For nonprofit organizations, the same economic questions arise on a smaller scale.

AI offers opportunities to improve knowledge management, proposal development, research, administration and programme delivery. Yet many organizations approach adoption primarily through subscriptions to commercial tools.

These services provide a useful entry point. They also leave important decisions about infrastructure, data processing and long-term costs in the hands of external providers. As AI becomes more deeply embedded in organizational operations, those decisions deserve closer examination. An NGO developing an AI system to support agricultural extension workers, for example, might need to consider whether the model can operate with intermittent connectivity, process local languages, draw on verified agricultural information and remain affordable as the number of users increases. A large cloud-based model might offer the strongest general capabilities. A smaller model deployed locally or through regional infrastructure might provide better economics for a narrower application. The appropriate choice would depend on testing both approaches against the conditions in which the system will operate.

Ownership is another consideration.

An organization may own its application, data and workflows while using a proprietary model through an API. Alternatively, it may deploy an open-weight model on infrastructure it controls, gaining greater flexibility over customization and operation. Open-weight systems still involve licensing conditions, infrastructure costs and technical responsibilities. They do, however, create possibilities for institutional control that may be unavailable through exclusively proprietary services.

At Mission AI, this is an important part of how we approach AI development with nonprofit organizations. We focus on building solutions around institutional requirements, with the resulting products owned by the organizations themselves. The architecture must support that objective. Ownership of an application is meaningful, but the extent of control also depends on the models, infrastructure and external services on which it relies. A well-designed AI strategy makes these relationships explicit.

How much scale is enough?

The extraordinary investment in frontier AI has expanded the capabilities available to businesses, governments and individuals. Research into larger models continues to produce advances, particularly in areas requiring broad knowledge and complex reasoning. At the same time, improvements in smaller models and system architectures are changing the cost of putting those capabilities to work.

These developments suggest that AI investment will increasingly involve two related questions: how far can computational scale extend model capabilities, and how efficiently can those capabilities be delivered for particular purposes? For frontier developers, the economic case for scaling depends on whether additional performance generates sufficient commercial value.

For governments, it depends on the relationship between investment, domestic technological capacity and wider economic benefits.

For organizations adopting AI, the calculation is more immediate. What does the system need to accomplish? How reliably must it perform? What will it cost to develop, operate and maintain? And how much control does the organization need over the technology? These questions should shape the choice of AI model and architecture. The largest model may be the appropriate solution for some applications. Others may benefit from smaller systems designed around specific tasks, data and operating conditions. The economics of AI will ultimately be determined by how effectively computational capability can be translated into useful products and services. That is where the next phase of investment deserves serious attention.

Mission AI works with nonprofit organizations to develop AI solutions they can own, govern and adapt to their operational needs. Our approach combines AI strategy, product development and responsible AI

governance.

References

1. Epoch AI (2024). How Much Does It Cost to Train Frontier AI Models? https://epoch.ai/publications/how-much-does-it-cost-to-train-frontier-ai-models

2. Stanford HAI (2025). Artificial Intelligence Index Report 2025. https://hai.stanford.edu/ai-index/2025-ai-index-report

3. Hoffmann et al. (2022). Training Compute-Optimal Large Language Models. https://arxiv.org/abs/2203.15556

4. Kaplan et al. (2020). Scaling Laws for Neural Language Models. https://arxiv.org/abs/2001.08361

5. Gu et al. (2024). Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone. https://arxiv.org/abs/2404.14219