Periodic Reporting for period 2 - Back to the Roots (Back to the roots of data-driven dynamical system identification)
Période du rapport: 2022-07-01 au 2023-12-31
System identification methods construct dynamic models from measured data. These mathematical models are used for simulation, prediction, monitoring, classification, or control tasks in applications for e.g. Industry 4.0 and eHealth. Most identification methods ‘solve’ an optimization problem, relying on some nonlinear iterative algorithm. Although effective in practice, those methods are suboptimal from a mathematical point of view. Undeniably, too many heuristics prevail: What do we mean by ‘solved’? Where did the algorithm converge to? Is the model globally optimal, unique and reproducible?
To tackle these scientific deficiencies, we design a framework to deal with inexact data. We solve a longstanding open problem of least squares optimality in system identification: for polynomial dynamical models, the optimal model derives from an eigenvalue problem. Hereto, we generalize notions from Algebraic Geometry (multivariate polynomials), Operator Theory (model spaces), System Theory (multidimensional realization) and Numerical Linear Algebra (matrix computations).
The first objective is to develop a mathematically rigorous realization approach that maps data onto new mathematical structures (multi-shift invariant subspaces).
The second objective is to conceive a misfit-latency framework to optimally map inexact data to these mathematical structures. We prove this to be a multiparameter eigenvalue problem. We expect breakthroughs in system theoretic characterizations of optimality, in the generalization to multiple input-output and multidimensional models and in finding the global optimum in the linear dynamic model reduction problem.
The third objective is to implement matrix computation algorithms for the results of the first two objectives, to root sets of multivariate polynomials, to solve multiparameter eigenvalue problems and to isolate only the minimizing roots. We focus on matrix aspects of large scale, sparsity and structure.
Deliverables are publications, software, graduate course material and science outreach initiatives.
To tackle these scientific deficiencies, we design a framework to deal with inexact data. We solve a longstanding open problem of least squares optimality in system identification: for polynomial dynamical models, the optimal model derives from an eigenvalue problem. Hereto, we generalize notions from Algebraic Geometry (multivariate polynomials), Operator Theory (model spaces), System Theory (multidimensional realization) and Numerical Linear Algebra (matrix computations).
The first objective is to develop a mathematically rigorous realization approach that maps data onto new mathematical structures (multi-shift invariant subspaces).
The second objective is to conceive a misfit-latency framework to optimally map inexact data to these mathematical structures. We prove this to be a multiparameter eigenvalue problem. We expect breakthroughs in system theoretic characterizations of optimality, in the generalization to multiple input-output and multidimensional models and in finding the global optimum in the linear dynamic model reduction problem.
The third objective is to implement matrix computation algorithms for the results of the first two objectives, to root sets of multivariate polynomials, to solve multiparameter eigenvalue problems and to isolate only the minimizing roots. We focus on matrix aspects of large scale, sparsity and structure.
Deliverables are publications, software, graduate course material and science outreach initiatives.
Objective 1: Quasi-Toeplitz matrices and their fundamental subspaces
We have described our new, mathematical unifying structure that encompasses scalar- or vector-, single- or multi-shift-invariant subspaces and the relation to their corresponding seed problems. For each of the four scenarios, we have found a compelling connection to system theory, which implies that solving the seed problems is essentially an application of realization theory for one-dimensional or multi-dimensional (mD) systems. This realization problem manifests itself in the right null space of the quasi-Toeplitz matrices.
By extending the Cayley-Hamilton theorem to a family of commuting matrices, we have established a connection between the row space of a Macaulay matrix and the difference equations that characterize the mD system considered in the right null space.
We have exploited the duality between the column space of the Macaulay matrix and its null space to design more efficient algorithms (Objective 3).
Objective 2: Misfit-latency data modeling
We have demonstrated how we can determine the globally optimal solution for the least-squares misfit identification problem across several model classes: we reformulate it as a multiparameter eigenvalue problem (MEP) and solve this through realization theory in the null space of the corresponding block Macaulay matrix. Furthermore, we have tackled problems involving more data points than before, by improving our algorithms (Objective 3) and moving to a high-performance computing environment.
The globally optimal solution of the model reduction problem in the H2-norm can also be found by solving an MEP.
Objective 3: Numerical linear algebra algorithms
We have translated our theoretical derivations into numerical linear algebra algorithms that yield reliable results. Moreover, we have increased efficiency by exploiting the inherent structure and avoiding the computation of the null space.
An important deliverable is the development of a toolbox that integrates the aforementioned algorithms. MacaulayLab (www.macaulaylab.net) a MATLAB toolbox, provides a user-friendly approach for addressing multivariate polynomial equations and MEPs. This toolbox includes a database of test problems.
The (block) Macaulay matrix algorithms can handle positive-dimensional solution sets at infinity. We have also addressed the extensions to other polynomial bases, other monomial orderings, and other shift polynomials.
We have described our new, mathematical unifying structure that encompasses scalar- or vector-, single- or multi-shift-invariant subspaces and the relation to their corresponding seed problems. For each of the four scenarios, we have found a compelling connection to system theory, which implies that solving the seed problems is essentially an application of realization theory for one-dimensional or multi-dimensional (mD) systems. This realization problem manifests itself in the right null space of the quasi-Toeplitz matrices.
By extending the Cayley-Hamilton theorem to a family of commuting matrices, we have established a connection between the row space of a Macaulay matrix and the difference equations that characterize the mD system considered in the right null space.
We have exploited the duality between the column space of the Macaulay matrix and its null space to design more efficient algorithms (Objective 3).
Objective 2: Misfit-latency data modeling
We have demonstrated how we can determine the globally optimal solution for the least-squares misfit identification problem across several model classes: we reformulate it as a multiparameter eigenvalue problem (MEP) and solve this through realization theory in the null space of the corresponding block Macaulay matrix. Furthermore, we have tackled problems involving more data points than before, by improving our algorithms (Objective 3) and moving to a high-performance computing environment.
The globally optimal solution of the model reduction problem in the H2-norm can also be found by solving an MEP.
Objective 3: Numerical linear algebra algorithms
We have translated our theoretical derivations into numerical linear algebra algorithms that yield reliable results. Moreover, we have increased efficiency by exploiting the inherent structure and avoiding the computation of the null space.
An important deliverable is the development of a toolbox that integrates the aforementioned algorithms. MacaulayLab (www.macaulaylab.net) a MATLAB toolbox, provides a user-friendly approach for addressing multivariate polynomial equations and MEPs. This toolbox includes a database of test problems.
The (block) Macaulay matrix algorithms can handle positive-dimensional solution sets at infinity. We have also addressed the extensions to other polynomial bases, other monomial orderings, and other shift polynomials.
Progress beyond state of the art
We have pioneered a Macaulay matrix algorithm for polynomial equations, utilizing the column space of the Macaulay matrix.
We have derived, for the first time, numerical linear algebra algorithms to solve rectangular multiparameter eigenvalue problems (MEPs).
We have introduced MacaulayLab, a new software toolbox that incorporates our algorithms for solving multivariate polynomial equations and MEPs.
Unlike prevailing methods, our novel misfit-latency identification algorithms ensure a globally optimal model.
Our new approach to derive the difference equations governing multidimensional (mD) systems, offers a parameterization that will enable optimization-based system identification for mD models.
We have unveiled that multivariate polynomial optimization over complex variables leads to an MEP, which we can solve to achieve the global optimum.
Our innovative numerical linear algebra algorithms provide solutions for the globally optimal H2-norm model reduction challenge.
Expected results
We will explore the theoretical properties of block Macaulay matrices, particularly the interpretation of their fundamental subspaces.
Our globally optimal system identification strategy will be generalized to single-input single-output (SISO) models, multiple-input multiple-output (MIMO) models and multidimensional (mD) systems. The misfit-latency identification problem can be reformulated as a structured total least squares (STLS) problem, solvable with a Riemannian singular value decomposition (R-SVD). We aim to either devise an algorithm for the R-SVD guaranteeing global optimality or establish its infeasibility. Utilizing the misfit-identification framework, we will tackle the problem of identifiability. It leads to an MEVP whose coefficient matrix incorporates higher-order derivatives of variables.
We will create numerical linear algebra algorithms that compute all critical values of a polynomial cost function directly, without first locating critical points. For the multiparameter eigenvalue problem, we will devise a specialized solver that finds only the real-valued eigenvalues.
We will enhance algorithm efficiency through leveraging matrix structure and sparsity and via subspace methods for faster convergence in our multiparameter eigenvalue solver.
We are augmenting the MacaulayLab toolbox with new features. We are developing a second toolbox specifically aimed at globally optimal system identification, promising benchmarking utility.
We have pioneered a Macaulay matrix algorithm for polynomial equations, utilizing the column space of the Macaulay matrix.
We have derived, for the first time, numerical linear algebra algorithms to solve rectangular multiparameter eigenvalue problems (MEPs).
We have introduced MacaulayLab, a new software toolbox that incorporates our algorithms for solving multivariate polynomial equations and MEPs.
Unlike prevailing methods, our novel misfit-latency identification algorithms ensure a globally optimal model.
Our new approach to derive the difference equations governing multidimensional (mD) systems, offers a parameterization that will enable optimization-based system identification for mD models.
We have unveiled that multivariate polynomial optimization over complex variables leads to an MEP, which we can solve to achieve the global optimum.
Our innovative numerical linear algebra algorithms provide solutions for the globally optimal H2-norm model reduction challenge.
Expected results
We will explore the theoretical properties of block Macaulay matrices, particularly the interpretation of their fundamental subspaces.
Our globally optimal system identification strategy will be generalized to single-input single-output (SISO) models, multiple-input multiple-output (MIMO) models and multidimensional (mD) systems. The misfit-latency identification problem can be reformulated as a structured total least squares (STLS) problem, solvable with a Riemannian singular value decomposition (R-SVD). We aim to either devise an algorithm for the R-SVD guaranteeing global optimality or establish its infeasibility. Utilizing the misfit-identification framework, we will tackle the problem of identifiability. It leads to an MEVP whose coefficient matrix incorporates higher-order derivatives of variables.
We will create numerical linear algebra algorithms that compute all critical values of a polynomial cost function directly, without first locating critical points. For the multiparameter eigenvalue problem, we will devise a specialized solver that finds only the real-valued eigenvalues.
We will enhance algorithm efficiency through leveraging matrix structure and sparsity and via subspace methods for faster convergence in our multiparameter eigenvalue solver.
We are augmenting the MacaulayLab toolbox with new features. We are developing a second toolbox specifically aimed at globally optimal system identification, promising benchmarking utility.