Deep Reinforcement Learning-Based Battery Management System for Electric Vehicles

Project Information

DeepBMS

Grant agreement ID: 101064083

DOI

10.3030/101064083

Project closed

EC signature date 12 July 2022

Start date 15 March 2023

End date 14 March 2025

Funded under

• Marie Skłodowska-Curie Actions (MSCA)

Total cost

No data

EU contribution

€ 230 774,40

Coordinated by

AALBORG UNIVERSITET
Denmark

Periodic Reporting for period 1 - DeepBMS (Deep Reinforcement Learning-Based Battery Management System for Electric Vehicles)

Reporting period: 2023-03-15 to 2025-03-14

Summary of the context and overall objectives of the project

The project addresses key deficiencies in electric vehicle (EV) battery systems, particularly the challenges that arise as batteries age. Aging introduces uncertainties in battery behavior, which in turn increases the uncertainty of driving range, contributes to range anxiety, and can accelerate further battery degradation.

To address these issues, this project focuses on next-generation Battery Management Systems (BMSs), with a particular emphasis on their software components. Specifically, it targets State-of-X estimation algorithms, where X can represent either the State of Charge (SoC) or the State of Health (SoH). SoC indicates the remaining charge of the battery, while SoH reflects the battery’s health and provides insight into its remaining useful life. Both metrics are critical to the safety and performance of EVs.

The proposed system, DeepBMS, adopts a hybrid approach that combines model-based and data-driven methods using reinforcement learning. This aims to develop algorithms that are both accurate and data-efficient. Traditional model-based approaches often suffer from low fidelity due to dynamically changing battery operating conditions—such as temperature fluctuations, driving patterns, and aging effects. On the other hand, purely data-driven approaches are typically data-intensive, requiring extensive lab testing, and often lack interpretability.

DeepBMS uses reinforcement learning to augment physical battery models with a streamlined data-driven component that compensates for model uncertainties. This enables accurate SoC and SoH estimation with significantly reduced data requirements. Moreover, the system is designed to be adaptable, capable of learning from new battery conditions and incorporating this knowledge into the BMS software to maintain estimation accuracy, even as the battery ages. The work carried out in DeepBMS to develop these state estimators include the whole design cycle from battery cell testing and dataset generation, designing and training the AI agents, optimizing the AI, to deployment of the trained agents in embedded systems.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The scientific and technical parts was dedicated on the following aspects:

Algorithmic innovation: Two types of state algorithms are designed with distinct philosophies. In the first approach, a pure model-based algorithm based on an extended Kalman filter (EKF) for battery state estimation is proposed as baseline. A recursive least squares algorithm (RLS) is also augmented with the EKF to enable real-time update of the model parameters to account for dynamic operating conditions. In addition, to account for model uncertainties arising from aging, a reinforcement learning agent is trained to learn how to adjust the covariance matrices in the EKF algorithm to achieve minimum state estimation error as the battery age. The first approach yields a maximum SoC estimation error below 2% and a root mean square error (RMSE) below 1% when tested on a dynamic driving condition. The philosophy of second approach is based on introducing a neural network directly into the architecture of the EKF algorithm. This mechanism, called KalmanNet, learns to adjust the gain of the Kalman algorithm through a recurrent neural network (RNN) with input features designed directly from cell measurements and previous state estimates. KalmanNet similarly achieved promising state estimation results in particular for deeply aged cells very close to the end-of-life condition where state estimation error was maintained as low as ~1%.

Cell testing: Two types of battery cells were considered for development and testing of the algorithms. This includes lithium-ion batteries based on nickel manganese cobalt oxide (NMC) and Lithium Titanate Oxide (LTO). Different standard tests based on IEC62660 including capacity test, hybrid pulse power characteristic test (HPPC), OCV-SoC characteristic test, and dynamic test replicating real driving situations were fulfilled to collect the necessary data for algorithmic works.

In addition, a test plan was developed to explore embedded implementation of the trained AI agents considering low-cost DSP setups such as TMS320F28379 and the battery monitoring ICs BQ79616 both from TI families. For these implementations, discretization, simplification (network pruning), and quantization are fulfilled and effects on final integration SW are assessed. A second implementation strategy using a cloud-based digital twin considering ThingSpeak platform is implemented for hierarchical execution of trained agents where a heavy version loads on the cloud and a light agent is embedded onboard. This later solution offers a scalable and flexible tool for DeepBMS implementation.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Reinforcement learning (RL) is a relatively new AI paradigm that has been rarely explored in the context of electric vehicle (EV) battery management systems (BMSs). In DeepBMS, RL was leveraged to enhance various aspects of battery state estimation. While RL-tuned algorithms in DeepBMS showed only slight performance improvements over traditional algorithms for fresh cells, their advantages became evident when applied to aged cells. For deeply aged cells—those experiencing up to 70% of their lifecycle degradation—DeepBMS was able to achieve and maintain state estimation errors below 2%. Current research efforts, though promising, often do not evaluate performance under such extreme aging conditions. In contrast, DeepBMS explicitly tested this scenario and observed, for instance, that aging introduces shifts in the OCV–SOC characteristic curves—an effect that traditional BMSs typically do not account for or adapt to. By addressing these shifts, our approach maintained high accuracy under challenging conditions. Remarkably, the DeepBMS algorithm required approximately five times less training data than a fully black-box neural network approach. Our hybrid method, which integrates model-based reasoning with learning, not only improves data efficiency but also enhances interpretability—an essential feature for safety-critical applications like EV BMSs.