Modelo de reducción de la huella de carbono en la nube para servicios de inferencia de modelos de lenguaje de gran escala (LLMs)

Abstract

The rapid expansion of Large Language Models (LLMs) has revolutionized the field of artificial intelligence, although it has also generated a considerable environmental impact that continues to rise daily. While the training of these models often receives significant attention due to its high energy consumption, the inference phase, which runs continuously with millions of requests each day, can accumulate an environmental impact that surpasses that of training. This research focuses on reducing the carbon footprint of inference services for these models by designing, implementing, and evaluating a carbon-aware computing system operating on multi-cloud infrastructures. The main problem lies in the fact that current routing systems only consider aspects such as latency, availability, and cost, disregarding the spatial and temporal variability of carbon intensity in the electrical grid. This variability is significant: the same query can cause up to ten times more emissions depending on the region where it is processed and the time of day, due to differences in the local energy mix. This presents both a challenge and an opportunity: carbon-aware routing, which incorporates this data alongside traditional metrics, could achieve substantial reductions without affecting the inherent efficiency of the models or the hardware hosting the application or service. To address this challenge, the research employs the Design Science Research methodological approach, articulated in five stages: from a thorough literature review and architectural design to the implementation of a functional prototype, experimental evaluations, and the dissemination of results to various audiences. The designed model consists of two layers: the control layer and the inference layer. The control layer comprises four modules: a carbon intelligence and analysis layer that collects and normalizes data from multiple sources (ElectricityMaps and WattTime); a router that applies a multi-objective scoring algorithm balancing carbon intensity, latency, usage, and cost; a telemetry system for comprehensive observability; and a governance module that allows for the adaptation of priorities according to each organization's needs. The algorithm assigns a weighted score to each possible execution region, considering carbon intensity (prioritizing regions with low CI), latency, current utilization, and cost, with configurable weights to reflect different organizational priorities. Strict constraints are guaranteed to ensure maximum latency is never exceeded, even if this implies selecting a closer but more polluting region. Experiments were conducted across 3 AWS regions over one week, using synthetic loads based on real distributions of request types. Variables such as load type (batch or interactive), time window, and geographic location were studied. Emissions, percentile latencies, throughput, costs, and error rates were measured. The results confirm that the approach is viable and efficient, albeit under certain circumstances: an average reduction of 12.7% in emissions was achieved compared to conventional routing, with an 8.1% increase in P95 latency. However, averages conceal variations: for instance, in batch loads, the reduction reached 18.3% with only a 5.8% increase in latency, whereas in interactive inference with small models, the reduction was smaller and the latency increase more notable, requiring careful evaluation. It was also detected that operating during off-peak hours (at night) optimizes results, improving latencies while reducing emissions. Another significant finding is that cost is not a determining factor when implementing carbon-aware models. This is mainly due to the high homogeneity in pricing across different regions of major cloud providers, in this case, AWS. In conclusion, this research provides broad benchmarks, methodological protocols, conceptual principles, and theories to foresee conditions of effectiveness, serving as a foundation for adopting and advancing carbon-aware computing. Recommendations are also offered for organizations, researchers, and sustainability managers, pointing toward future strategic directions to accelerate the impact and scalability of these technologies. This work demonstrates that it is possible to reduce the environmental footprint of artificial intelligence significantly and practically. However, achieving real change requires overcoming technical, organizational, and cultural barriers, in addition to developing support ecosystems, appropriate regulations, and a shared vision of sustainability as a central axis.