We have worked on achieving a unifying framework between deep learning, neural networks and kernel machines. Most promising are to consider core models based on Restricted Kernel Machines or based on Least Square Support Vector Machines. It enables to work either in a parametric way (e.g. deep neural networks, convolutional feature maps) or kernel-based in its dual representation. Because of its connection with Restricted Boltzmann machines, Restricted Kernel Machines provide a setting for generative modelling. Moreover, it is suitable for multi-view and tensor-based models, deep learning, latent space exploration, explainability and robustness. Remarkable new connections have been shown between self-attention in transformers and kernel singular value decomposition within the least squares support vector machines framework with primal and dual representations. These new settings with solid foundations and understanding of their primal and dual representations can be exploited towards a wide range of different application fields.
Recent publications:
Pandey A., Schreurs J., Suykens J.A.K. Generative Restricted Kernel Machines: A Framework for Multi-view Generation and Disentangled Feature Learning, Neural Networks, Vol.135 pp 177-191, March 2021
Pandey A., Fanuel M., Schreurs J., Suykens J.A.K. ``Disentangled Representation Learning and Generation with Manifold Optimization'', Neural Computation, vol. 34, no. 10, 2022, pp. 2009-2036.
Fanuel M., Schreurs J., Suykens J.A.K. ``Nystrom landmark sampling and regularized Christoffel functions'', Machine Learning, vol. 11, 2022, pp. 2213-2254.
Tonin F., Lambert A., Patrinos P., Suykens J.A.K. ``Extending Kernel PCA through Dualization: Sparsity, Robustness and Fast Algorithms'', ICML 2023.
He M., He F., Shi L., Huang X., Suykens J.A.K. ``Learning with Asymmetric Kernels: Least Squares and Feature Interpretation'', IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 8, Aug. 2023, pp. 10044-10054.
Chen Y., Tao Q., Tonin F., Suykens J.A.K. ``Primal-Attention: Self-attention through Asymmetric Kernel SVD in Primal Representation'', NeurIPS 2024.
Tonin F., Tao Q., Patrinos P., Suykens J.A.K. ``Deep Kernel Principal Component Analysis for Multi-level Feature Learning'', Neural Networks, vol. 170, Feb. 2024, pp. 578-595.
Tao Q., Tonin F., Patrinos P., Suykens J., ``Tensor-based Multi-view Spectral Clustering via Shared Latent Space'', Information Fusion, vol. 108, Aug. 2024, pp. 1-15.
Tao Q., Tonin F., Lambert A., Chen Y., Patrinos P., Suykens J.A.K. ``Learning in Feature Spaces via Coupled Covariances: Asymmetric Kernel SVD and Nystrom method", ICML 2024.
Chen Y., Tao Q., Tonin F., Suykens J., ``Self-Attention through Kernel-Eigen Pair Sparse Variational Gaussian Processes'', ICML 2024.
Achten S., Tonin F., Patrinos P., Suykens J. A. K., ``Unsupervised Neighborhood Propagation Kernel Layers for Semi-supervised Node Classification'', in Proc. of the AAAI Conference on Artificial Intelligence (AAAI), Vancouver, Canada, Mar. 2024, pp. 10766 - 10774.
