Periodic Reporting for period 2 - ALMA (ALMA: Human Centric Algebraic Machine Learning)
Période du rapport: 2022-03-01 au 2023-08-31
What is the problem/issue being addressed?
Algebraic Machine Learning has recently been proposed as a new learning paradigm that builds upon abstract algebra and model theory. Unlike deep learning and other popular learning algorithms, AML is not a statistical method. Instead, it produces generalizing models from semantic embeddings of data into discrete algebraic structures. The result is a machine learning paradigm that:
1. is far less sensitive to the statistical characteristics of the training data and does not fit (or even use) parameters;
2. has the potential to flexibly and seamlessly combine unstructured and complex information contained in training data with a formal specification of human knowledge including constraints and task goals;
3. has higher mathematical transparency than deep networks and other optimization-based statistical methods and uses sets and graphs as internal representations of data which are ideal for generating human-understandable descriptions of what, why, and how has been learned; and
4. can be implemented in a distributed way that avoids centralized, privacy-invasive collections of large data sets in favor of a collaboration of many local learners.
Why is it important for society?
Humans do not think in a vacuum but need a substantial accumulation of knowledge as a precondition to resolve the simplest real-life problems. Areas with the highest social impact, such as healthcare, where AI can potentially do the most good, are notorious for the enormous amount of knowledge required to tackle any problem. Artificial Intelligence has the potential to execute tasks that humans cannot possibly accomplish alone. However, for this to happen, machines should be able to learn what we already know, to communicate with each other and to communicate and collaborate effectively with us. Current machine learning algorithms have major difficulties combining heterogeneous knowledge for two key reasons: first, modern machine learning algorithms are statistical in nature; and second, there are no effective processes for encoding formal or even structured knowledge in a way that is useful for these algorithms.
AML has the right properties and has the potential to provide a solution to these problems. AML relies on algebraic operations that are less affected by the statistics of the training data. AML provides an effective mechanism that encodes heterogeneous formal knowledge. It can not only learn and produce complex models with multiple interrelated variables, but also generalize and memorize at the same time.
The non-statistical nature of AML, combined with the capability to encode directives and constraints, makes AML very well suited to overcoming problems posed by biased training data. AML may offer a mechanism for ensuring that the machine intelligence is aligned with the desires and ethical principles of human operators. Most importantly, compared to statistical learning ‘black boxes’, AML models offer greater control and better understanding by humans, in both human-to-machine and machine-to-human communication. In its most ambitious form, AML makes augmentation of human intelligence possible. AML opens a path worth exploring, where machines can collaborate with humans in a ‘human-computer partnership’. This partnership involves a mutual understanding of world models and the alignment of goals and ethical values.
What are the overall objectives?
The aim of the project is to leverage the above properties of AML for a new generation of interactive, human-centric machine learning systems. Interactive means that the system should allow human users and
intelligent machines to jointly learn and reason. AML is ideal to enable human users to not only reflect upon the learning process but also to actively drive it, simultaneously enhancing their own cognitive powers through the interaction with the AI. Reflecting the vision of Human Centric AI, we will:
● reduce bias and prevent discrimination by benefiting from the reduced dependence on statistical properties of training data (property 1), integrating formalized human knowledge with regular training data (property 2), and exploring the how and why of the learning process (property 3);
● facilitate trust and reliability by respecting ‘hard’ human-defined constraints in the learning process (property 2) and by enhancing explainability of the learned models (property 3);
● integrate complex ethical constraints into Human-AI systems by going beyond basic bias and discrimination prevention (property 2) to interactively shape between humans and the machine the ethics related to the learning process (property 3); and
● facilitate a new distributed, incremental collaborative learning method by going beyond the dominant centralized and off-line data processing approach (property 4).
Algebraic Machine Learning has recently been proposed as a new learning paradigm that builds upon abstract algebra and model theory. Unlike deep learning and other popular learning algorithms, AML is not a statistical method. Instead, it produces generalizing models from semantic embeddings of data into discrete algebraic structures. The result is a machine learning paradigm that:
1. is far less sensitive to the statistical characteristics of the training data and does not fit (or even use) parameters;
2. has the potential to flexibly and seamlessly combine unstructured and complex information contained in training data with a formal specification of human knowledge including constraints and task goals;
3. has higher mathematical transparency than deep networks and other optimization-based statistical methods and uses sets and graphs as internal representations of data which are ideal for generating human-understandable descriptions of what, why, and how has been learned; and
4. can be implemented in a distributed way that avoids centralized, privacy-invasive collections of large data sets in favor of a collaboration of many local learners.
Why is it important for society?
Humans do not think in a vacuum but need a substantial accumulation of knowledge as a precondition to resolve the simplest real-life problems. Areas with the highest social impact, such as healthcare, where AI can potentially do the most good, are notorious for the enormous amount of knowledge required to tackle any problem. Artificial Intelligence has the potential to execute tasks that humans cannot possibly accomplish alone. However, for this to happen, machines should be able to learn what we already know, to communicate with each other and to communicate and collaborate effectively with us. Current machine learning algorithms have major difficulties combining heterogeneous knowledge for two key reasons: first, modern machine learning algorithms are statistical in nature; and second, there are no effective processes for encoding formal or even structured knowledge in a way that is useful for these algorithms.
AML has the right properties and has the potential to provide a solution to these problems. AML relies on algebraic operations that are less affected by the statistics of the training data. AML provides an effective mechanism that encodes heterogeneous formal knowledge. It can not only learn and produce complex models with multiple interrelated variables, but also generalize and memorize at the same time.
The non-statistical nature of AML, combined with the capability to encode directives and constraints, makes AML very well suited to overcoming problems posed by biased training data. AML may offer a mechanism for ensuring that the machine intelligence is aligned with the desires and ethical principles of human operators. Most importantly, compared to statistical learning ‘black boxes’, AML models offer greater control and better understanding by humans, in both human-to-machine and machine-to-human communication. In its most ambitious form, AML makes augmentation of human intelligence possible. AML opens a path worth exploring, where machines can collaborate with humans in a ‘human-computer partnership’. This partnership involves a mutual understanding of world models and the alignment of goals and ethical values.
What are the overall objectives?
The aim of the project is to leverage the above properties of AML for a new generation of interactive, human-centric machine learning systems. Interactive means that the system should allow human users and
intelligent machines to jointly learn and reason. AML is ideal to enable human users to not only reflect upon the learning process but also to actively drive it, simultaneously enhancing their own cognitive powers through the interaction with the AI. Reflecting the vision of Human Centric AI, we will:
● reduce bias and prevent discrimination by benefiting from the reduced dependence on statistical properties of training data (property 1), integrating formalized human knowledge with regular training data (property 2), and exploring the how and why of the learning process (property 3);
● facilitate trust and reliability by respecting ‘hard’ human-defined constraints in the learning process (property 2) and by enhancing explainability of the learned models (property 3);
● integrate complex ethical constraints into Human-AI systems by going beyond basic bias and discrimination prevention (property 2) to interactively shape between humans and the machine the ethics related to the learning process (property 3); and
● facilitate a new distributed, incremental collaborative learning method by going beyond the dominant centralized and off-line data processing approach (property 4).
Updated definition of the overall architecture covering
a) the ALMA WP dependencies.
b) the definition of the specification of the AML nodes
c) the final implementation of the comms framework used by these nodes to work collaboratively (AML-IP v0.1.0).
New Theorems:
Set of theorems showing the properties of an extension of finite semilattices to atomized semilattices by adding additional elements (atoms).
Set of theorems on how to embed a problem into a semilattice, starting with discussing the embedding problem, then introducing four different kinds of embeddings and showing how to use atomized semilattices to characterize them, and finally giving a set of relevant theorems and their proofs.
Updated AML-DL specification conceived to build embeddings for real-life problems, an extended interpreter tool, an improved consistency checker, and a debugging software tool to visually explore semantic embeddings.
Initial prototype implementation of a high-performance hardware accelerator for AML.
Human-AML Interaction research: Introduced more cognitive theories (i.e. theory of mind)
Evaluation of AML: Comparison between statistical ML methods with AML.
AML Use cases:
a) medical image classification
b) benchmark for robotized ironing and garment folding
c) An interactive prototype pipeline “AMLExplorer” which is aimed to easily train, assess, and apply AML-models to expressive gesture use in music exploration.
a) the ALMA WP dependencies.
b) the definition of the specification of the AML nodes
c) the final implementation of the comms framework used by these nodes to work collaboratively (AML-IP v0.1.0).
New Theorems:
Set of theorems showing the properties of an extension of finite semilattices to atomized semilattices by adding additional elements (atoms).
Set of theorems on how to embed a problem into a semilattice, starting with discussing the embedding problem, then introducing four different kinds of embeddings and showing how to use atomized semilattices to characterize them, and finally giving a set of relevant theorems and their proofs.
Updated AML-DL specification conceived to build embeddings for real-life problems, an extended interpreter tool, an improved consistency checker, and a debugging software tool to visually explore semantic embeddings.
Initial prototype implementation of a high-performance hardware accelerator for AML.
Human-AML Interaction research: Introduced more cognitive theories (i.e. theory of mind)
Evaluation of AML: Comparison between statistical ML methods with AML.
AML Use cases:
a) medical image classification
b) benchmark for robotized ironing and garment folding
c) An interactive prototype pipeline “AMLExplorer” which is aimed to easily train, assess, and apply AML-models to expressive gesture use in music exploration.
Although deep learning is the current state of the art in machine learning, it has important shortcomings and it is unlikely to prevail as the definitive solution to machine intelligence. AML does not rely upon optimization and is more mathematically transparent than current methods. Our new learning method extends the scope of applicability of machine learning because AML 1- can be applied to problems that require of existing formalized knowledge, 2- can combine formal knowledge and training data, 3 - can make training more practical as it depends less upon the statistical properties of the training set that existing techniques, and 4 - allows for large-scale decentralized learning.
The potential benefit to society of these new capabilities is considerable. In key areas such as healthcare, an enormous amount of information is already formalized and available but cannot be used by current machine learning methods. Using formal knowledge (knowledge that is already understood by humans and available in a formal language) can extend the domain of applicability of machine learning to the most important problems of society, including healthcare, decision-making or machine automation.
The potential benefit to society of these new capabilities is considerable. In key areas such as healthcare, an enormous amount of information is already formalized and available but cannot be used by current machine learning methods. Using formal knowledge (knowledge that is already understood by humans and available in a formal language) can extend the domain of applicability of machine learning to the most important problems of society, including healthcare, decision-making or machine automation.