Reducing Carbon Footprint for Generative AI

Project Information

Green-GPT

Grant agreement ID: 101138056

DOI

10.3030/101138056

Project closed

EC signature date 4 September 2023

Start date 1 October 2023

End date 31 March 2025

Funded under

• European Research Council (ERC)

Total cost

No data

EU contribution

€ 150 000,00

Coordinated by

THE HEBREW UNIVERSITY OF JERUSALEM
Israel

Periodic Reporting for period 1 - Green-GPT (Reducing Carbon Footprint for Generative AI)

Reporting period: 2023-10-01 to 2025-03-31

Summary of the context and overall objectives of the project

Generative AI systems, such as chatGPT, recently passed the Turing test, forever transforming human-machine interaction. These systems provide giant productivity leaps across many sectors. However, their energy requirements increase nine-fold annually and their abundance grows at exponential rate. The resulting carbon footprint becomes significant.
IT giants such as Google, Nvidia, Microsoft, and Amazon, as well as many mid-sized companies, have committed to reduce their carbon footprint. The EU is strengthening regulation for emission reductions. But the new generative AI trend jeopardizes emission reduction commitments.
Most power consumption of generative AI is spent on matrix multiplication. Our novel solutions reduce energy consumption and carbon footprint by replacing current matrix multiplication algorithms with more efficient ones. These can be implemented on existing hardware and software stacks. Potential energy saving predicted at about 40-50%, while maintaining performance and accuracy.
The novel developments of Prof. Oded Schwartz and his strong team are based on years of research, and protected by patents. The funds are requested to pursue business opportunity.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

We have performed energy use and power consumption experiments with several of the new algorithms, on various hardware settings, including CPU and GPU. Often-cases, the energy saving was proportional to the time saving, with larger percentage of energy saving compared to time saving.However in some cases no decrease in energy use was found despite significant time saving. Further research is being conducted.
We have implemented several algorithms on various hardware setting, including CPUs, GPUs, and other, including strong indications for large speedups on run-time programmable (FPGA-like) hardware.
We have extended and generalized the algorithmic method of [SPAA23, Patent-4], making it applicable to a wider range of settings.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Potential impact of reducing energy costs and carbon footprint is of interest to governmental institutions, and to large IT companies.
Further research is required to demonstrate energy saving in a wider range of setting.
Most (but not all) solution tests were software-based. Further research is required on the impact of hardware-based implementations of our algorithms; particularly to demonstrate potential positive impact on energy use by replacing existing hardware accelerators with ones that utilize the new algorithms.

A new startup has been recently founded bases on the results of this project and others. A commercialization initial agreement has been signed with the TTO of HUJI. A final agreement is being processed. Pre-seed funding has been secured. Interaction with several potential design partners has been initialized. Several patents have been submitted or processed as part of this and related projects.