Reducing Carbon Footprint for Generative AI

Projektinformationen

Green-GPT

ID Finanzhilfevereinbarung: 101138056

DOI

10.3030/101138056

Projekt abgeschlossen

EK-Unterschriftsdatum 4 September 2023

Startdatum 1 Oktober 2023

Enddatum 31 März 2025

Finanziert unter

• European Research Council (ERC)

Gesamtkosten

Keine Daten

EU-Beitrag

€ 150 000,00

Koordiniert durch

THE HEBREW UNIVERSITY OF JERUSALEM
Israel

Periodic Reporting for period 1 - Green-GPT (Reducing Carbon Footprint for Generative AI)

Berichtszeitraum: 2023-10-01 bis 2025-03-31

Zusammenfassung vom Kontext und den Gesamtzielen des Projekts

Generative AI systems, such as chatGPT, recently passed the Turing test, forever transforming human-machine interaction. These systems provide giant productivity leaps across many sectors. However, their energy requirements increase nine-fold annually and their abundance grows at exponential rate. The resulting carbon footprint becomes significant.
IT giants such as Google, Nvidia, Microsoft, and Amazon, as well as many mid-sized companies, have committed to reduce their carbon footprint. The EU is strengthening regulation for emission reductions. But the new generative AI trend jeopardizes emission reduction commitments.
Most power consumption of generative AI is spent on matrix multiplication. Our novel solutions reduce energy consumption and carbon footprint by replacing current matrix multiplication algorithms with more efficient ones. These can be implemented on existing hardware and software stacks. Potential energy saving predicted at about 40-50%, while maintaining performance and accuracy.
The novel developments of Prof. Oded Schwartz and his strong team are based on years of research, and protected by patents. The funds are requested to pursue business opportunity.

Arbeit, die ab Beginn des Projekts bis zum Ende des durch den Bericht erfassten Berichtszeitraums geleistet wurde, und die wichtigsten bis dahin erzielten Ergebnisse

We have performed energy use and power consumption experiments with several of the new algorithms, on various hardware settings, including CPU and GPU. Often-cases, the energy saving was proportional to the time saving, with larger percentage of energy saving compared to time saving.However in some cases no decrease in energy use was found despite significant time saving. Further research is being conducted.
We have implemented several algorithms on various hardware setting, including CPUs, GPUs, and other, including strong indications for large speedups on run-time programmable (FPGA-like) hardware.
We have extended and generalized the algorithmic method of [SPAA23, Patent-4], making it applicable to a wider range of settings.

Fortschritte, die über den aktuellen Stand der Technik hinausgehen und voraussichtliche potenzielle Auswirkungen (einschließlich der bis dato erzielten sozioökonomischen Auswirkungen und weiter gefassten gesellschaftlichen Auswirkungen des Projekts)

Potential impact of reducing energy costs and carbon footprint is of interest to governmental institutions, and to large IT companies.
Further research is required to demonstrate energy saving in a wider range of setting.
Most (but not all) solution tests were software-based. Further research is required on the impact of hardware-based implementations of our algorithms; particularly to demonstrate potential positive impact on energy use by replacing existing hardware accelerators with ones that utilize the new algorithms.

A new startup has been recently founded bases on the results of this project and others. A commercialization initial agreement has been signed with the TTO of HUJI. A final agreement is being processed. Pre-seed funding has been secured. Interaction with several potential design partners has been initialized. Several patents have been submitted or processed as part of this and related projects.