Deep Reinforcement Learning-Based Battery Management System for Electric Vehicles

Informazioni relative al progetto

DeepBMS

ID dell’accordo di sovvenzione: 101064083

DOI

10.3030/101064083

Progetto chiuso

Data della firma CE 12 Luglio 2022

Data di avvio 15 Marzo 2023

Data di completamento 14 Marzo 2025

Finanziato da

• Marie Skłodowska-Curie Actions (MSCA)

Costo totale

Nessun dato

Contributo UE

€ 230 774,40

Coordinato da

AALBORG UNIVERSITET
Denmark

Periodic Reporting for period 1 - DeepBMS (Deep Reinforcement Learning-Based Battery Management System for Electric Vehicles)

Periodo di rendicontazione: 2023-03-15 al 2025-03-14

Sintesi del contesto e degli obiettivi generali del progetto

The project addresses key deficiencies in electric vehicle (EV) battery systems, particularly the challenges that arise as batteries age. Aging introduces uncertainties in battery behavior, which in turn increases the uncertainty of driving range, contributes to range anxiety, and can accelerate further battery degradation.

To address these issues, this project focuses on next-generation Battery Management Systems (BMSs), with a particular emphasis on their software components. Specifically, it targets State-of-X estimation algorithms, where X can represent either the State of Charge (SoC) or the State of Health (SoH). SoC indicates the remaining charge of the battery, while SoH reflects the battery’s health and provides insight into its remaining useful life. Both metrics are critical to the safety and performance of EVs.

The proposed system, DeepBMS, adopts a hybrid approach that combines model-based and data-driven methods using reinforcement learning. This aims to develop algorithms that are both accurate and data-efficient. Traditional model-based approaches often suffer from low fidelity due to dynamically changing battery operating conditions—such as temperature fluctuations, driving patterns, and aging effects. On the other hand, purely data-driven approaches are typically data-intensive, requiring extensive lab testing, and often lack interpretability.

DeepBMS uses reinforcement learning to augment physical battery models with a streamlined data-driven component that compensates for model uncertainties. This enables accurate SoC and SoH estimation with significantly reduced data requirements. Moreover, the system is designed to be adaptable, capable of learning from new battery conditions and incorporating this knowledge into the BMS software to maintain estimation accuracy, even as the battery ages. The work carried out in DeepBMS to develop these state estimators include the whole design cycle from battery cell testing and dataset generation, designing and training the AI agents, optimizing the AI, to deployment of the trained agents in embedded systems.

Lavoro eseguito dall’inizio del progetto fino alla fine del periodo coperto dalla relazione e principali risultati finora ottenuti

The scientific and technical parts was dedicated on the following aspects:

Algorithmic innovation: Two types of state algorithms are designed with distinct philosophies. In the first approach, a pure model-based algorithm based on an extended Kalman filter (EKF) for battery state estimation is proposed as baseline. A recursive least squares algorithm (RLS) is also augmented with the EKF to enable real-time update of the model parameters to account for dynamic operating conditions. In addition, to account for model uncertainties arising from aging, a reinforcement learning agent is trained to learn how to adjust the covariance matrices in the EKF algorithm to achieve minimum state estimation error as the battery age. The first approach yields a maximum SoC estimation error below 2% and a root mean square error (RMSE) below 1% when tested on a dynamic driving condition. The philosophy of second approach is based on introducing a neural network directly into the architecture of the EKF algorithm. This mechanism, called KalmanNet, learns to adjust the gain of the Kalman algorithm through a recurrent neural network (RNN) with input features designed directly from cell measurements and previous state estimates. KalmanNet similarly achieved promising state estimation results in particular for deeply aged cells very close to the end-of-life condition where state estimation error was maintained as low as ~1%.

Cell testing: Two types of battery cells were considered for development and testing of the algorithms. This includes lithium-ion batteries based on nickel manganese cobalt oxide (NMC) and Lithium Titanate Oxide (LTO). Different standard tests based on IEC62660 including capacity test, hybrid pulse power characteristic test (HPPC), OCV-SoC characteristic test, and dynamic test replicating real driving situations were fulfilled to collect the necessary data for algorithmic works.

In addition, a test plan was developed to explore embedded implementation of the trained AI agents considering low-cost DSP setups such as TMS320F28379 and the battery monitoring ICs BQ79616 both from TI families. For these implementations, discretization, simplification (network pruning), and quantization are fulfilled and effects on final integration SW are assessed. A second implementation strategy using a cloud-based digital twin considering ThingSpeak platform is implemented for hierarchical execution of trained agents where a heavy version loads on the cloud and a light agent is embedded onboard. This later solution offers a scalable and flexible tool for DeepBMS implementation.

Progressi oltre lo stato dell’arte e potenziale impatto previsto (incluso l’impatto socioeconomico e le implicazioni sociali più ampie del progetto fino ad ora)

Reinforcement learning (RL) is a relatively new AI paradigm that has been rarely explored in the context of electric vehicle (EV) battery management systems (BMSs). In DeepBMS, RL was leveraged to enhance various aspects of battery state estimation. While RL-tuned algorithms in DeepBMS showed only slight performance improvements over traditional algorithms for fresh cells, their advantages became evident when applied to aged cells. For deeply aged cells—those experiencing up to 70% of their lifecycle degradation—DeepBMS was able to achieve and maintain state estimation errors below 2%. Current research efforts, though promising, often do not evaluate performance under such extreme aging conditions. In contrast, DeepBMS explicitly tested this scenario and observed, for instance, that aging introduces shifts in the OCV–SOC characteristic curves—an effect that traditional BMSs typically do not account for or adapt to. By addressing these shifts, our approach maintained high accuracy under challenging conditions. Remarkably, the DeepBMS algorithm required approximately five times less training data than a fully black-box neural network approach. Our hybrid method, which integrates model-based reasoning with learning, not only improves data efficiency but also enhances interpretability—an essential feature for safety-critical applications like EV BMSs.