Reducing Carbon Footprint for Generative AI

Información del proyecto

Green-GPT

Identificador del acuerdo de subvención: 101138056

DOI

10.3030/101138056

Proyecto cerrado

Fecha de la firma de la CE 4 Septiembre 2023

Fecha de inicio 1 Octubre 2023

Fecha de finalización 31 Marzo 2025

Financiado con arreglo a

• European Research Council (ERC)

Coste total

Sin datos

Aportación de la UE

€ 150 000,00

Coordinado por

THE HEBREW UNIVERSITY OF JERUSALEM
Israel

Periodic Reporting for period 1 - Green-GPT (Reducing Carbon Footprint for Generative AI)

Período documentado: 2023-10-01 hasta 2025-03-31

Resumen del contexto y de los objetivos generales del proyecto

Generative AI systems, such as chatGPT, recently passed the Turing test, forever transforming human-machine interaction. These systems provide giant productivity leaps across many sectors. However, their energy requirements increase nine-fold annually and their abundance grows at exponential rate. The resulting carbon footprint becomes significant.
IT giants such as Google, Nvidia, Microsoft, and Amazon, as well as many mid-sized companies, have committed to reduce their carbon footprint. The EU is strengthening regulation for emission reductions. But the new generative AI trend jeopardizes emission reduction commitments.
Most power consumption of generative AI is spent on matrix multiplication. Our novel solutions reduce energy consumption and carbon footprint by replacing current matrix multiplication algorithms with more efficient ones. These can be implemented on existing hardware and software stacks. Potential energy saving predicted at about 40-50%, while maintaining performance and accuracy.
The novel developments of Prof. Oded Schwartz and his strong team are based on years of research, and protected by patents. The funds are requested to pursue business opportunity.

Trabajo realizado desde el comienzo del proyecto hasta el final del período abarcado por el informe y los principales resultados hasta la fecha

We have performed energy use and power consumption experiments with several of the new algorithms, on various hardware settings, including CPU and GPU. Often-cases, the energy saving was proportional to the time saving, with larger percentage of energy saving compared to time saving.However in some cases no decrease in energy use was found despite significant time saving. Further research is being conducted.
We have implemented several algorithms on various hardware setting, including CPUs, GPUs, and other, including strong indications for large speedups on run-time programmable (FPGA-like) hardware.
We have extended and generalized the algorithmic method of [SPAA23, Patent-4], making it applicable to a wider range of settings.

Avances que van más allá del estado de la técnica e impacto potencial esperado (incluida la repercusión socioeconómica y las implicaciones sociales más amplias del proyecto hasta la fecha)

Potential impact of reducing energy costs and carbon footprint is of interest to governmental institutions, and to large IT companies.
Further research is required to demonstrate energy saving in a wider range of setting.
Most (but not all) solution tests were software-based. Further research is required on the impact of hardware-based implementations of our algorithms; particularly to demonstrate potential positive impact on energy use by replacing existing hardware accelerators with ones that utilize the new algorithms.

A new startup has been recently founded bases on the results of this project and others. A commercialization initial agreement has been signed with the TTO of HUJI. A final agreement is being processed. Pre-seed funding has been secured. Interaction with several potential design partners has been initialized. Several patents have been submitted or processed as part of this and related projects.