Periodic Reporting for period 1 - SPEAR (Spatial Perception and Embodied Autonomy Research)
Reporting period: 2023-11-01 to 2024-10-31
Aerial robots self-evolve and train to fit their mission and environment.
We are excited to introduce the new groundbreaking Horizon Europe project in artificial intelligence (AI) & robotics entitled "SPEAR: Spatial Perception & Embodied Autonomy Research".
SPEAR’s vision stems from clear observations regarding certain inconsistencies between the engineered design of flying robots and the natural paradigm of flying organisms. The bodies and brains of flying animals vary greatly depending on their ecological niche specialization. A striking example is the difference between species such as mosquitoes and eagles. Among the two, both bodies and brains are vastly different. However, for flying robots until now, design diversity is particularly limited. A look at the history of this domain is particularly telling. By 1922, the de Bothezat helicopter with its four six-bladed rotors had performed its maiden flight, while it was in the early 2000s that small unmanned quadrotors emerged and quickly dominated the drone sector due to their simplicity and agility. Despite the vastly different mission profiles, crewed and uncrewed rotorcrafts share a striking set of similarities. A change of paradigm towards environment- and mission-tailored aerial robots is necessary.
SPEAR aims to revolutionize the field and deliver the vision of mission- and environment-tailored aerial robotics through a collective view of the design of the robots' "body" and "brain," To that end, our research will exploit the synergy of a) evolutionary algorithms, b) a rigid and soft materials, c) complementary robot perception, d) efficient propulsion and e) progress in deep neural networks for navigation strategy learning. The goal is to have AI create novel aerial robots that are more efficient, resilient, and safe through their optimized fit to the mission and the type of environment. This is the grand, disruptive vision of SPEAR. Upon its success, it has the potential to drastically shift aerial robot design from human-conceived creations to automated, task-specific designs.
SPEAR is a new project in response to the HORIZON-CL4-2022-DIGITAL-EMERGING-02-06 solicitation and builds upon the collaboration of NTNU (Coordinator, NO), TU Delft (NL), Luleå University of Technology (SE), Vrije Universiteit Amsterdam (NL), IMEC (BE), ETH Zurich (CH), University of Paderborn (DE), VoxelSensors (BE), Biodrone (NO) and the Center for Security Studies (GR). For more information, please visit our website: https://www.spear-robotics.com/(opens in new window)
We are excited to introduce the new groundbreaking Horizon Europe project in artificial intelligence (AI) & robotics entitled "SPEAR: Spatial Perception & Embodied Autonomy Research".
SPEAR’s vision stems from clear observations regarding certain inconsistencies between the engineered design of flying robots and the natural paradigm of flying organisms. The bodies and brains of flying animals vary greatly depending on their ecological niche specialization. A striking example is the difference between species such as mosquitoes and eagles. Among the two, both bodies and brains are vastly different. However, for flying robots until now, design diversity is particularly limited. A look at the history of this domain is particularly telling. By 1922, the de Bothezat helicopter with its four six-bladed rotors had performed its maiden flight, while it was in the early 2000s that small unmanned quadrotors emerged and quickly dominated the drone sector due to their simplicity and agility. Despite the vastly different mission profiles, crewed and uncrewed rotorcrafts share a striking set of similarities. A change of paradigm towards environment- and mission-tailored aerial robots is necessary.
SPEAR aims to revolutionize the field and deliver the vision of mission- and environment-tailored aerial robotics through a collective view of the design of the robots' "body" and "brain," To that end, our research will exploit the synergy of a) evolutionary algorithms, b) a rigid and soft materials, c) complementary robot perception, d) efficient propulsion and e) progress in deep neural networks for navigation strategy learning. The goal is to have AI create novel aerial robots that are more efficient, resilient, and safe through their optimized fit to the mission and the type of environment. This is the grand, disruptive vision of SPEAR. Upon its success, it has the potential to drastically shift aerial robot design from human-conceived creations to automated, task-specific designs.
SPEAR is a new project in response to the HORIZON-CL4-2022-DIGITAL-EMERGING-02-06 solicitation and builds upon the collaboration of NTNU (Coordinator, NO), TU Delft (NL), Luleå University of Technology (SE), Vrije Universiteit Amsterdam (NL), IMEC (BE), ETH Zurich (CH), University of Paderborn (DE), VoxelSensors (BE), Biodrone (NO) and the Center for Security Studies (GR). For more information, please visit our website: https://www.spear-robotics.com/(opens in new window)
General development that cuts across the project's objectives:
- Developed the first version of the SPEAR Simulation Toolbox and open-sourced it (the "Aerial Gym Simulator") featuring:
- Simulation for arbitrary aerial configurations with diverse sensors and soft joints as needed for the SPEAR vision.
- Enhanced performance metrics over other competitive tools for aerial robot simulation
- Integration with diverse policy learning tools and implementation of pre-designed control and navigation policy solutions.
- This allowed to timely achieve the first project milestone.
KO1: Soft and Squeezable Flying Robots Design
- Simulated and fabricated soft aerial robots with shape awareness.
- Developed a set of design frameworks for soft joints including for lattice structures and the newly introduced elastic rolling cams (ERC).
- Manufactured the programmable ERC rotational joints with integrated sensing systems.
- Demonstrated superior performance of soft robots in squeezing through tight spaces, enduring collisions (>7m/s), and achieving programmable mechanical responses.
KO2: Human-&-AI Co-Generated Soft Flying Embodiments
- Created processes to evolve airframes and control policies with an initial focus on rigid robots.
- Evolution and overall airframe optimization may be driven either by navigation behaviors or explicit metrics for the body's dynamics.
- Airframe optimization is subject to fabrication constraints, for example that propellers should not overlap.
- Compared to conventional airframes, then optimizing for predominantly forward-navigating robots we identify novel stable robot designs with clear improvements.
- Explored fitness metrics, evolutionary techniques, and airframe representations.
KO3: Holistic Spatial Perception & Embodied Autonomy
- Developed and tested the new Andromeda2 sensing prototype.
- Derived a set of scalable neural architectures for position control as well as collision-free navigation.
- Achieved robust sim-to-real transfer for navigation policies and efficient map-free navigation with approximately 2m/s speed in an obstacle-filled environment.
KO4: Domain-Specific Embodiment & Policy Specialization
- Achieved first results on optimizing airframes and policies for niche tasks, outperforming conventional designs.
- Investigated objective functions for specific domains, including for a) vision and depth-based navigation for rigid and soft robots, as well as b) fast exploration tasks using rigid multirotors.
- Conducted early investigations in the problem of domain-variation simulation and focused on automatically generating environments for task-specific navigation policy training
KO5: Field Demonstration and Hardening
- Conducted exploratory tests with conventional multirotors to set comparison baselines for the project:
- Explored 1.1km of an underground mine autonomously at 1.7m/s.
- Navigated dense forests autonomously at >2m/s using learned policies.
- Developed the first version of the SPEAR Simulation Toolbox and open-sourced it (the "Aerial Gym Simulator") featuring:
- Simulation for arbitrary aerial configurations with diverse sensors and soft joints as needed for the SPEAR vision.
- Enhanced performance metrics over other competitive tools for aerial robot simulation
- Integration with diverse policy learning tools and implementation of pre-designed control and navigation policy solutions.
- This allowed to timely achieve the first project milestone.
KO1: Soft and Squeezable Flying Robots Design
- Simulated and fabricated soft aerial robots with shape awareness.
- Developed a set of design frameworks for soft joints including for lattice structures and the newly introduced elastic rolling cams (ERC).
- Manufactured the programmable ERC rotational joints with integrated sensing systems.
- Demonstrated superior performance of soft robots in squeezing through tight spaces, enduring collisions (>7m/s), and achieving programmable mechanical responses.
KO2: Human-&-AI Co-Generated Soft Flying Embodiments
- Created processes to evolve airframes and control policies with an initial focus on rigid robots.
- Evolution and overall airframe optimization may be driven either by navigation behaviors or explicit metrics for the body's dynamics.
- Airframe optimization is subject to fabrication constraints, for example that propellers should not overlap.
- Compared to conventional airframes, then optimizing for predominantly forward-navigating robots we identify novel stable robot designs with clear improvements.
- Explored fitness metrics, evolutionary techniques, and airframe representations.
KO3: Holistic Spatial Perception & Embodied Autonomy
- Developed and tested the new Andromeda2 sensing prototype.
- Derived a set of scalable neural architectures for position control as well as collision-free navigation.
- Achieved robust sim-to-real transfer for navigation policies and efficient map-free navigation with approximately 2m/s speed in an obstacle-filled environment.
KO4: Domain-Specific Embodiment & Policy Specialization
- Achieved first results on optimizing airframes and policies for niche tasks, outperforming conventional designs.
- Investigated objective functions for specific domains, including for a) vision and depth-based navigation for rigid and soft robots, as well as b) fast exploration tasks using rigid multirotors.
- Conducted early investigations in the problem of domain-variation simulation and focused on automatically generating environments for task-specific navigation policy training
KO5: Field Demonstration and Hardening
- Conducted exploratory tests with conventional multirotors to set comparison baselines for the project:
- Explored 1.1km of an underground mine autonomously at 1.7m/s.
- Navigated dense forests autonomously at >2m/s using learned policies.
The beyond-state-of-the-art key research and innovation results achieved so far in SPEAR are as follows:
- Creation of a novel simulation tool tailored to aerial robot evolution with efficient navigation policy training.
- Developed novel soft programmable joints and a first set of soft aerial robots to inform the evolutionary process by human experience. The robots present superior collision tolerance and squeeze-ability compared to rigid flying robots.
- Derived a first set of results on evolving airframes based on niche tasks and identified robot designs that outperform conventional flat multirotors for their specific task. Initially, we focused on maneuverability and fast forward-facing flight.
- Investigated a set of exteroceptive navigation policies that transfer robustly to reality while being trained only in simulation.
- Derived safety filters by the use of the external reference governor.
- Developed the first prototype of the novel event-based active ranging system called Andromeda2
- Creation of a novel simulation tool tailored to aerial robot evolution with efficient navigation policy training.
- Developed novel soft programmable joints and a first set of soft aerial robots to inform the evolutionary process by human experience. The robots present superior collision tolerance and squeeze-ability compared to rigid flying robots.
- Derived a first set of results on evolving airframes based on niche tasks and identified robot designs that outperform conventional flat multirotors for their specific task. Initially, we focused on maneuverability and fast forward-facing flight.
- Investigated a set of exteroceptive navigation policies that transfer robustly to reality while being trained only in simulation.
- Derived safety filters by the use of the external reference governor.
- Developed the first prototype of the novel event-based active ranging system called Andromeda2