Final Report Summary - CARGO-ANTS (Cargo handling by Automated Next generation Transportation Systems for ports and terminals)
Executive Summary:
The Cargo-ANTs project, "Cargo handling by Automated Next generation Transportation Systems for ports and terminals", aims to face two opposite constraints affecting nowadays world of container handling: the continuous growth in global trade and the new stringent environmental regulations, as demanded by a higher claim for sustainability. In essence, the handling of freight (both at ports as well as at hinterland connections) has to become significantly more effective for the smallest individual scale to voluminous container flows.
The Cargo-ANTs project aims at the development of innovative transhipment technologies to:
- Reduce terminal congestion, by increasing container throughput;
- Focus on continuous, automated and consistent operation;
- Insist on high controllability and reliability;
- Strive for the highest safety levels.
Existing modern technology solution is based on a grid of transponders installed on the terminal ground allowing full control on optimized Automated Vehicles. Enlarging the autonomous driving terrain is affected by severe constraints related to: large economic investments (preventing this technology to be a realistic option for smaller terminals) and the time consumption (stops of terminal operations).
The project Cargo-ANTs has developed an innovative solution widening the flexibility realizable on grid-independent infrastructure, significantly lowering the investment requirements and hence extending the highly automated and yet extremely flexible flow of individual units of freight from- and to- cargo ships in harbour terminals and within inland terminals (thus specifically to smaller terminals).
This goal has been achieved by adopting autonomous robotics research to develop a novel technology, on the following main innovation fields of investigation:
• full vehicle perception and dynamic- and static- object detection
• local- and global- mapping
• autonomous path generation and following
Among others, the strength of the Cargo ANTs solution is related to:
o Commodity: a full set of market available sensors allowing for both detection-, tracking- and classification- of objects as well as for vehicle localization;
o Awareness: a deterministic risk estimation and vehicle control is achieved by implementing a self-consistent system based on kino-dynamic model specific for the vehicles;
o Re-planning: a novel approach for mapping and trajectory generation based on planning algorithm is developed allowing for allocation of vehicle’s task, global path planning and path adaption.
The achieved solution turn out to be adaptable to different stakeholder needs, spanning from port and terminal operators to other players in the logistic chain (large warehouse, airports or large parking lots).
Although not fully addressed, a variety of potential broader impact can be easily imagined, as for instance the direct integration of selected innovative e-freight solutions which hold the potential for streamlining co- and synchro- modality logistics.
Furthermore, the autonomous path generation opens the project solution to be deployed in inter-terminal transportation and to lighten the workloads assigned to the terminal centralized management system.
In summary, the achieved project objectives are:
• an automated shared work yard for intelligent AGVs and highly Automated Trucks
• innovative solutions for planning, decision, control and safety strategies for these vehicles
• an environmental perception system and a grid-independent positioning system
These results allows for an increased performance and throughput of freight transportation in main ports and freight terminals, by maintaining a high level of safety.
Project Context and Objectives:
Present harbour- and inland- container terminals are presently limited by a non-negligible manually driven manoeuvres. Despite the progress in autonomous vehicles, the benefits have not widespread in the container handling due to various motivations, among others different shipping modalities of several terminals and safety related restriction (fencing).
Moreover, the implemented automatized systems have needed a substantial investment to equip the terminal ground with a grid of transponders allowing full control on optimized Automated Guided Vehicles, AGV. This large economic investment is preventing smaller terminals to adopt similar solutions; on the other hand, extending a ground-based infrastructure requires a stop in the business daily operations which strongly limits the versatility to enlarge the autonomous driving terrains.
The scope of Cargo-ANTs is to develop an innovative solution which allows to increase the automation level in container terminals. The deployment of this solution will exhibit the efficiency for harbour- or inland- terminals in terms of operational throughput, safety and automation achievable in grid-less infrastructures.
The project has been driven by three key area of developments:
➢ Investment reduction
➢ Safety raise
➢ Efficiency
which have led to the following results:
▪ Innovative positioning system
▪ localization and control of the vehicle (AGV or AT),
▪ increase environmental awareness and planning trajectories
▪ intelligent and cooperative interaction between ANTs
The technological innovations have been defined in order to fit into a modern concept of container handling, as depicted in the following Fig. 1
Fig. 1 Scheme for a modern concept of container handling, from vessel unload to final destination. Automated driving is realized in the primary terminal (w/ and w/o grid infrastructure) and in the connection to a secondary terminal. Manual driving will finally serve an inland terminal and the final destination.
The Automated Guided Vehicle will transport containers from the crane to the closer stocking area, the Automated Truck will transport them further to terminal perimeter in automated mode and finally can deliver- or transport containers to inland terminal, in manual mode.
The project innovations will be of major profitability for those terminals dealing with a large volume of container traffic, but can be adapted to smaller terminal as well. Furthermore, the autonomy intrinsic in the path planning tool represents a step towards decentralization, lightening the role of the global site planner to assign the task without being in charge of the low-level trajectory organization too.
Objectives
The innovative technology is based on the developments achieved in detecting the vehicle- and environmental- characteristics (full vehicle perception and local-, global- mapping) which then allows for an autonomous path generation and path following.
The object- and vehicle- detection is at the basis of a self-consistent risk estimation that permits the vehicle to deterministically recognise new potentially dangerous situations and reacts according to well established safety procedures.
The project consists of a dissemination work package and 3 technical work packages:
• Dissemination and exploitation
This work package focuses on an efficient dissemination and exploitation of the results of Cargo-ANTs and an efficient management of the intellectual property. It deals with: providing visibility of the project - disseminating the scientific results - preparing and maintaining an exploitation plan - protecting the intellectual property.
The specific objectives are:
o Demonstration of AT and AGV in simulations and in a real-life environment, exhibiting the capabilities of the prototype vehicles to the main stakeholders, customers and end-users;
o Dissemination of the scientific and technical results, identifying the results to be published, the most appropriate journals and the conferences of most interest;
o Providing visibility of the project results to a broader public, realizing a private and public website;
o Maintenance of an exploitation plan based on the needs of the customers and end-users, covering not only applications in ports and terminals but also examining how the project results can be used in other areas, i.e. automotive branch;
o Discussion and verification of the progress and results of the technology driven project with customers, end-users, authorities and specialists for its operational, legal and possibly other socio-economic aspects that may impact acceptance and implementation;
o Protection of intellectual property, including assessment of project results qualifying for IPR protection and the identification of the necessary (first) steps for patenting procedure within the regulations of the Consortium Agreement.
• Requirements, Design and Safety
This work package is responsible for the definition of a system architecture that can be applied across the project and the definition of a development process for safe and certifiable automated systems. The specific objectives are:
o Detailed use-case specifications;
o Identify requirements for each level according to V-model;
o Develop a system architecture that can be applied to the different ANTs;
o Develop a development process for safe and certifiable automated systems.
• Vehicle Localization and object detection
This work package deals with the development of software modules to localize the vehicles in a map of the environment and to detect other static and moving objects during navigation. To achieve this, information from multiple sensor sources need to be fused together.
The specific objectives are:
o Develop software modules for localization of vehicles and detection of objects and obstacles for the pre-defined uses-cases defined;
o Provide estimates on localization and tracking of all moving elements;
o Provide the developed software modules for successive integration
The developed software modules has been designed and tuned to facilitate the integration activities on the platform planned in the last work package;
o Final design and implementation description of the software architecture;
o Mapping from software modules to the hardware components present on the AGV and AT;
o Description of the algorithms used for absolute object and environment mapping;
o Validation results of the tests performed with these software modules., tested by analysis of the recorded data at test locations (harbour area and parking lot) mimicking terminal configuration.
• Planning, Decision and Action
The focus is on global and local path planning methods as developed together with the required interface to an overall site planning and to vehicle control and safety interventions. The latter are the command central for the traffic in the environment, the high level planner (by means of a global path planning algorithm) in combination with the local planner on the vehicle produces a path that the vehicle should follow, also known as re-planning or path adaption. Moreover, the work package also deals with the safety functionalities to be included in the vehicle for emergency manoeuvres, hence emergency braking functionality for objects in the path are implemented and tested.
The specific objectives are:
o Final design and implementation description of the software architecture;
o Develop global and local path planning sub-systems along with vehicle control and path following sub-systems;
o Integrate the map, communication and localization inputs from a previous work package into the planning algorithms, according to the hardware components present on the ANTs;
o Build a mock-up high-level site planning control;
o Realize a correct interaction with the site planner and integrate safety interventions into the vehicle control;
o Description and finalization of the algorithms used for task assignment, global planning and path adaptation.
• Verification & Validation, Safety assessment and Integration
This work package is responsible for the integration and the verification of the previously developed results, enabling Cargo-ANTS project to evolve more easily into a producible end-product (either being an Automated Truck or a next-generation AGV).
The specific objectives are:
o Verification of the requirements, system architecture and subsystems;
o Integration test and validation of Level 1 stakeholder requirements;
o Establish the structure of a limited safety case for certification and execution of a selected part of the development;
o Integration and testing of AT and AGV in lab environments, test-track and in a real-life environment.
Results
Following the above mentioned objectives, the next table summarizes the results achieved.
GOAL DESCPTION STATUS / NOTES
Design for integration The functional-, software- and physical- architectures have been designed and realized for the two vehicles DONE
Collision avoidance Two separate environmental perception software have been developed for the two ANTs
Analogously, the two vehicles have been equipped with different sensor set and high and low-level controllers DONE
Localization and detection A host vehicle localization and specific object detection has been improved adopting the SLAM, Simultaneous Localization And Mapping, developed in the field of robotics DONE
Planning, decision and action Autonomous trajectory planning has been realized, considering both the global map of the environment as well as the local map of the surrounding of the vehicle DONE
Decision and Action would have required the interaction with a container terminal management system, which was out of the scope of the project. Nevertheless, a mock-up example provides the input to add in the future these strategic information DONE
Safety assessment The goal within the safety aspects was intended to focus on the “limited safety case description”, and more precisely on the items to be considered for the realization (increased Technology Readiness level) in the production of the prototypal ANTs.
This activity has not been completed due to the bankruptcy of APTS. This subcontractor was in charge of providing access to an AGV as well as managing the levels and providing process management for certification, based on its experienced in the area of automated vehicles (bus, containers carrier, guidance systems) CANCELLED
Integration The software integration activities have undergone the issue of AGV availability. But all software components are integrated and tested in simulators to show the complete functionality. Software testing has been performed with recorded sensor data
Thanks to the presence of the Truck manufacturer, the instrumentation and final software integration into the Automated Truck was fully completed DONE
On the other side, the impossibility to access a prototypal AGV did not allow the physical integration of the dedicated software.
The decision to develop an AGV-simulator allowed to carry on the integration of modules, interfaces and software communication on the virtual vehicle PARTLY DONE
Demonstration As a consequence of the missing AGV, the final event was splitting the achievements for the two vehicles. The demonstration took place in Gothenburg and the main stakeholders, customers and end-users were invited
For the Automated Truck, the exhibition provided the evidence of the innovative solutions developed by the project partners.
Obviously, coordination with the second ANT could not be proven PARTLY DONE
For the AGV, the set of use cases defined in WP3 could be reproduced only via simulation, demonstrating both the vehicle control (including collision avoidance manoeuvres) as well as the compatibility and interaction with the localization and re-planning software DONE ONLY IN SIMULATIONS
The demonstration run during the final event are described in Section 5.
The left picture of Fig. 2 simulates an AGV running on a typical terminal configuration (global mapping), detecting obstacles as the container corners and following its predefined trajectory; the right plot of Fig. 2 illustrates the path adaptation in case of conflicting ANTs trajectories (via the Covariant Hamiltonian Optimization for Motion Planning).
Fig. 3 shows the integration of two software modules for local mapping- and local planning- functionalities.
Fig. 2 Left: localization and mapping in AGV simulation – Right: path adaptation for conflicting routes
Fig. 3 integration of local planning with path adaptor
Finally, the integrated software modules have been adopted to run simulations and on real tests on the specific ANTs. Fig. 4 illustrates on the left the simulation of an AGV on a container terminal (yellow lines detecting container corners) and on the right, an AT running specific use cases on a test track (yellow lines detecting smaller objects).
Fig. 4 Simulated AGV on a container terrain and real execution of localization module for the AT on a physical test track
Project Results:
Final demonstration event
A final demonstration event was held in Gothenburg on August, 29th, at Stora Holm.
The event has been advertised by means of a dedicated flyer and has been advertised via mails, private communication and obviously the project webpage. The latter has represented the primarily method for project external institution to register its attendance to the final event.
The event consisted on a series of project results presentations including simulations of all case scenarios involving AGVs and ATs, as well as three live demonstration with a real Volvo truck showing the following autonomous navigation skills: autonomous map building and localization, real-time localization in a previously built map, and autonomous driving and obstacle avoidance using a local obstacle map.
The agenda included
a. internal discussions within the consortia about lessons learnt from the project for the AGV and AT use cases,
b. public introduction to Stora Holm,
c. technical project presentations of achievements,
d. AGV simulation results,
e. AT test runs,
f. conclusions.
The final demonstration event was an important achievement, showing successfully the most important autonomous navigation components running on-board a real truck. A representative from the European Commission attended the event and was invited to ride inside the truck during the real demos.
Fig. 9 Group picture of the EC representative and of the partners
The following from Fig. 10 to Fig. 13 include some of the most relevant results shown during this final event:
Fig. 10 Simulation results of the kinematics of an AGV and of a port container terminal.
Fig. 11 Simulation results of AGV localization in a previously recorded map
Fig. 12 Real-time execution results of localization and mapping of an AT at Stora Holm
Fig. 13 Real-time execution of obstacle avoidance and path adaptor at Stora Holm
Use and Dissemination of Foreground
The dissemination of foreground results of the Cargo-ANTs project took place by mean of various channels, as workshops and events, publication at scientific journals and conferences, the web site, live demonstrations and media appearances.
Intellectual Property
Definitions of the EC recognize two distinct types of intellectual property (IP) in FP7:
• Background IP (BGIP) is the IP that partners bring with them to the project.
• Foreground IP (FGIP) is the IP generated during the project
Three different project results (foreground) were identified during the final demonstration event that are subject of intellectual protection: the planning software, the radar odometry module, and the localization and mapping software. These results are succinctly described in section 6 in this document. A more elaborated description of these results is two folded: the public part of these developments is included in the material already presented in the related scholarly publications that resulted from the project. The restricted part is the sole ownership of the groups that participated in their development, and more specifically, of the academic partner Halmstad University for the case of the planning software, and of CSIC for the case of the stereo radar odometry and the localization and mapping software.
The intellectual property right officers in both academic institutions will investigate on the possibility of requesting protection and licensing of these developments.
Potential Impact:
The solution Cargo-ANTs provides, is relevant for different stakeholders, specifically for port and terminal operators, but also for other players in the logistics chain, including those that are not only occupied with (sea-) container handling. These include major warehouses and factories that have to move goods from A to B at their site, airports that are responsible for luggage transport, and not to forget the automotive industry that is working on future concepts to find and automatically move towards a parking lot in an open parking place. It is clear that the solutions of Cargo-ANTs are relevant for a wide audience, also outside the logistic chain that is responsible for container handling. Therefore, the dissemination strategy of the Cargo-ANTs project is an essential and central aspect to maximize cross-fertilization.
To come up with an adequate exploitation plan, correctly steered by the market agents, the consortium sought during the duration of the project active involvement of stakeholders via the Liaison group to propose and approve use cases, to agree on systems requirements, to discuss implications on the workforce, and to monitor progress and results.
The empowered work package in charge for dissemination and exploitation has experienced an increased number of activities in the second half of the project.
From the dissemination of scientific results, a list of journal articles and conference papers have been presented describing various project contributions.
Towards a wider audience, the consortia have repeatedly inquired the engagement and availability of the cargo terminals of the port of Rotterdam and the port of Gothenburg. As alternatives to these opportunities, other sound locations have been visited as the test-track “AstaZero” (Sweden) and the “Stora Holm”, close to Gothenburg.
Moreover, a web site has been updated containing appealing information for stakeholders and end-users.
Dissemination
Stakeholder needs are summarized as Level 1 requirements in deliverable 3.1. They are summarised here through a series of agreements with the members of the Liaison Group.
on use cases
Stakeholders are mainly interested in advances in use cases involving multi-vehicle scenarios and safety situations. They are primarily interested in use cases demonstrated on the site itself, but also to nearby facilities, in the direct vicinity of the harbour site.
on operational requirements
The main identified operational requirement is to use a grid-less solution for autonomous navigation of both AGV's and AT's. Ground magnetic guiding grids are rigid infrastructures that require a significant (financial) investment from the start, maintenance during its use and have very limited flexibility on operations.
on operational performance
Performance is mainly related on robustness of the solutions, since they have to work 24h/24h, 365d/365d, under harsh environment conditions such as fog, dust, rain or darkness. Stakeholders will adopt new technologies only if they are robust enough to justify cost investments with low risk on innovation. Besides robustness, efficient interaction between the vehicles (no deadlocks and close following) is of importance to come to an optimal use and efficiency of the site operation.
The critical components for a final robust system are all navigation layers, from low level steering and drive controllers, up to high level localization within the map. The latter is especially critical since it closes the navigation loop, while it relies on state of the art sensing and perception algorithms and technologies. Therefore, long-term operational testing would be required, before any commercialization step.
Exploitation
The exploitation plan includes an analysis of the socio-economic acceptance of technology and concepts by stakeholders (including end-users, workforce, and local authorities). Legal aspects concerning Cargo-ANTs solution, in both harbour sites and autonomous vehicles are also analysed, as well as management of intellectual property of foreground project results. This document also reports the testing, validation and certification steps that would be required to transfer the knowledge of technology prototypes created by the project to final ready-to-market solutions. Finally, this deliverable also identifies a set of separate components with enough potential to be exploited independently, in the wide domain of intelligent transportation systems.
A dedicated deliverable also reports on several achievements, especially on identifying market needs, and listing the steps from prototype to end-product, including the development phase, the testing activities & the required certification. Individual software components developed in WP’s 4 and 5 such as LIDAR polyline mapping, stereo radar odometry, or local path adaptations have been identified as components that could be exploited separately.
The stereo radar odometry module is shown in Fig. 5 (by means of polyline mapping and local path adaptation modules)
Fig. 5 Stereo radar odometry module.
Viability
The exploitation plan includes an analysis of the socio-economic acceptation of technology and concepts by stakeholders (including end-users, workforce, and local authorities). This document also reports the testing, validation and certification steps that would be required to transfer the knowledge of technology prototypes created by the project to final ready-to-market solutions. Finally, this deliverable also identifies a set of separated components with enough potential to be exploited independently, in the wide domain of intelligent transportation systems
Acceptance
Harbour workforce is the main user of Cargo-ANTs solution. User-acceptance shall be achieved based on the Validation Test and Functional Safety Case. Both depend on the System Architecture (pass/fail criteria depending on Level 1 requirements).
Concerning performance metrics, current AGV's at the ECT terminal in Rotterdam work on average 5.000 hours per year and drive approximately 30.000 km per year. There exist Mean Time Between Failures (MTBF) standards specified in the contract between the supplier, VDL and ECT. MTBF of an autonomous navigation system such as that developed in the Cargo ANTs project could be computed based on a series of simulated experiments where sensor data can be progressively corrupted to emulate different noise and outlier conditions. In that line, the Cargo ANTs project has set up a simulated environment based on the Gazebo Robotics simulator, which has been used to develop the navigation software components.
The truck industry has also performance metrics but related with the driver as a user. For the case of driver-less vehicles, as a first approach, trucks could be considered as AGV’s in terms of metrics. In that case other interesting metrics can be computed:
o Energy consumption
o CO2 emissions
o Workforce reduction
o Work accidents reduction
o Container Stack surface increment
Cost benefit analysis of the solution
The benefit of a similar AGV should however be considered on top of a series of variables very much related to the area of application.
A concrete evaluation of the benefit provided by a smart AGV should consider:
a. The investment saved in the infrastructure, avoiding very expensive transponder grid realization on the ground.
b. The versatility to enlarge the terminal area, increasing the number of running AGV will not involve a stop in the terminal workflow while enlarging a transponder grid ground would require a long stop for the realization and a transition period to integrate the new terrain in the centralized control system, CCS.
c. The self-planning of the smart-AGV, which reduces the workloads of the CCS. In other words, the probability to exceed the CCS capacity will be hugely decreased avoiding the control system to become a bottleneck for the terminal operations.
d. An higher level of automation in inter-terminal transportation can be achieved. This will de facto extend the area of competence of AGV and AT within the terminal.
e. The public road could be finally approached by smart-AGV thanks to the gained independency from the transponder grid ground.
In order to compare solutions, costs of ground magnetic grid installation and maintenance are briefly analysed, which are mainly labour-oriented and depend on the dimensions of the site. Under the hypothesis that a magnetic grid infrastructure in a port setting cost is 250K€/hectare, and a mean terminal port in Europe is around 4 hectares wide, an investment of about 1M€ is avoided if using a grid-less solution
proposed by Cargo-ANTs. Moreover, magnetic grid have maintenance cost associate depending on the level of incidences, and also implicitly deploys a fixed layout of the port, which can also cause a loss of incomes due to worse and less flexible operations.
Instead, Cargo-ANTs solutions exploits the grid-less approach, at a minimal extra cost per vehicle of the order of 40K€ in production line (mainly LIDAR sensors and on-board computer). Motion controllers and encoders are already present in a grid-based AGV. Required extra mechanical design for component integration and software development cost do not increment the production cost per vehicle.
Moreover, Cargo-ANTs grid-less navigation solution could potentially also be used in Guidance Systems for bus & coach, reducing operational constrains upon the customers infrastructure, providing a more efficient system at lower operational cost.
Investment and Maintenance Issues
The consortium will contact VDL Steelweld / ECT to know details on this issue, especially for AGV’s operating in the Rotterdam harbour.
Due to the missing access to the prototypal AGV, no information is available related to the maintenance cost of the final AGV. Moreover, the lacking engagement of the manufacturer as well as the reluctance of the end user to share the related data of the presently running AGV, does not allow a complete comparison on these topics.
In the other side, the AT vehicle has been built in the context of the project, according the assigned budget. However, the experimental sessions within the Cargo-ANTs project were focused on developing and testing different levels of the navigation solution, but not in long-term operations, so maintenance operations were out of the scope of these sessions.
Nevertheless, it is worth to mention the following aspects characterizing the innovative solutions developed within the Cargo-ANTs project. The environmental perception set has been constructed out of market available sensors. The maturity of these sensors is not considered to be harming the specific maintenance costs but the LIDAR sensors are very vulnerable when large G-forces are applied (e.g. when a crane drops a container on the AGV). The use of LIDAR systems with moving part will introduce more maintenance to the AGV and also the cleaning of the front glass has to be done regularly. Instead, RADAR systems are less vulnerable to these external forces, but are not enough for mapping and localization requirements.
On the other hand, it is the engineering design to mount this sensors that can affect the Mean Time Between Failures, a parameter very much dependent on daily conditions, among others soft/hard container loading, status of the road, vehicle speed and crashes.
From prototype to end-product : development phase, testing & certification
By verifying, validating and integrating the positioning system into an AGV on a potential SIL-level, stakeholders and future customers would gain more confidence and trustability into the feasibility of such a system, which could then lead to a PMC, Product Market Combination. Potentially integrating the positioning system into a next generation of Computerized Guidance Systems would broaden our level of service and expertise for development of automated guidance systems for usage in different domains. The achievement of a SIL-4 certification for a safe and efficient next generation AGV opens the opportunity to set a new world wide technical standard for public transport market.
Successful validation of a fully autonomous system, in particular an AT, in a closed environment would boost the confidence of all the stakeholders by pushing it one level closer to being technically feasible and commercially viable solution in the area of safe autonomous systems for motorways, urban roads. By introducing ATs in the truck industry we open up the floor for new standards in the domain and thereby facilitate and demonstrate the use of automation for Safe and efficient transport.
The main steps from prototype to end-product would be the following:
1. Find customer interested in buying Cargo-ANTs solution.
2. Perform System Definition Review.
3. Specify requirements based on Cargo-ANTs project result and customer demands.
4. Create Functional Architecture (Design).
5. Create Computational Architecture.
6. Procure HW requirements (Sensors, Systems, etc.).
7. Create Component level Architecture.
8. Develop the research platform in the truck (prototype in TRL4).
9. Perform Critical Design Update.
10. Test end-product - confirm ISO26262.
11. Perform System Acceptance Review.
12. Demonstrate to the stake-holders and to potential
13. Perform Vehicle Acceptance Review
Exploitation of separated components
Platform
The innovative solutions realized in Cargo ANTs have been developed based on the needs of container terminals, dealing with a large number of daily container handling.
However, the application of environmental perception, chip market sensors, ego vehicle localization, object detection and classification, local and global mapping interaction as well as trajectory computation and re-planning are not specific for harbour environment.
An educated guess immediately leads to think similar solutions to be applied in:
• Airports: both for luggage handling- as well as for passenger transportation- system.
• Small terminals: adapting the dimensions- and functionalities- of automated guided vehicle.
• Factory storehouses, dealing with other container types, not necessarily shipping containers.
Planning Software
Halmstad’s contributions in Cargo-ANTs consisted of an integrated planning framework for the motion planning of multiple vehicles, this included methods for task allocation, global path planning and local trajectory planning. Halmstad expects to exploit such modules in current and future research projects.
Specifically, the local and global path planners are expected to be employed in current projects in Halmstad University’s research profile CAISR (Centre for Applied Intelligent Systems Research). Thus, the planning software is foreseen to be exploited in the research activities conducted at the intersection of CAISR’s research area of Mechatronics/Robotics and Intelligent Vehicles application area. One of the PhD thesis has been worked on studying the present container terminal scenarios and adapting the existing site planner developed for CargoANTs to suit a more realistic scenario.
The developed software for local trajectory planning method is presently been investigated and employed in an another project AntWay for the development of Autonomous Trucks in collaboration with Vinnova, Volvo Trucks AB, Kollmorgan and Chalmers University.
Likewise, the modules developed for planning will be extended as part of a current project proposal submitted to the Swedish Research council (Vetenskapsrådet) with focus on Belief Space Planning for Robust Autonomous Navigation. In such a project we expect to extend the planning methods developed in Cargo-ANTs to consider the uncertainty generated from the vehicle’s perception
Localization and Mapping Software
The Localization and mapping (SLAM) solution developed for the Cargo-ANTs project is fully flexible to be exported to other markets and applications. It is based on an optimization of geometric constraints imposed by sensing the environment. In the Cargo-ANTs projects, such constraints are provided by 2D LIDAR sensing and wheel odometry. A signal processing step which identifies straight segments, corners and polylines is applied to raw LIDAR ranging data, and this geometric features are used to impose such constraints for the optimization. However, many other geometric constraints, coming from 2D LIDAR, but also from other sensing modalities (specially cameras, 3D scanners, RADAR’s, RF beacons or GPS) can be easily adopted by the localization and mapping solution, leading to a more robust and accurate final performance.
This approach definitively leads to have a general solution for autonomous vehicle localization and mapping, which can be applied beyond harbour environments. Shuttles operating in restricted areas or indoor logistics may be among the markets that could adopt from now the developed approach.
RADAR odometry has been also investigated, resulting in an innovative approach able to compute the vehicle odometry by using only a pair of on-board automotive RADAR devices. This result can be also exported to other domains requiring complementary approaches for the odometry estimate.
Summary
Regarding the exploitation opportunities that the Cargo ANTs project could offer, the following conclusions can be drawn:
➢ The Liaison members are aligned with the use cases and specific requirements.
➢ At the end of the project, the viability of the concept is still hard to quantify in detail due to a lack of maturity of the solution. However some estimates are done which point out that the grid-less approach is interesting in budgetary terms.
➢ Legal aspects and processes to follow are known as far as it concerns confined areas. As soon as it comes to (semi-) public roads, actual legislation is poor.
➢ IP protection will be analysed for three specific software modules in Cargo ANTs: the path planning module, the localization and mapping module, and the stereo radar odometry module.
➢ Main steps from prototype to end-product are identified.
➢ A set of separated components are identified with enough potential to be exploited independently, in the wide domain of intelligent transportation systems.
List of Websites:
The project consortium brought together research institutes and vehicle manufacturer, thus establishing a balanced framework for the implementation of innovative robotics solutions to prototypal vehicles.
The communication and the logging data exchange has been facilitated by an internal repository while the dissemination to a wider public has been pursued by mean of the project webpage.
Consortium
The project consortium is composed out of 6 partners, as shortly described in the following list.
Organization: Nederlandse organisatie voor toegepast
natuurwetenschappelijk onderzoek
Short Name: TNO
Role: Project Coordinator and WP1 and WP4 leader
Reference: Dr. Thomas Chiarappa
As first proposer and project coordinator, TNO is participating actively on every work package.
The Netherlands Organization for Applied Scientific Research (in short, TNO), is a knowledge organization for companies, governmental bodies and public organizations and it creates, develops and applies scientific knowledge in various domains ranging from industry to society aspects.
It is the fundamental goal of TNO to direct its research activities toward creative and practical innovations in the form of new products, services, and processes, fully customized for businesses and governmental institutions. One of the research themes of TNO is “Mobility”, comprising topics as (intelligent) vehicle technology, ITS, cooperative and automated driving, road side technology, infrastructure technology, intelligent traffic management, environment and human factors in transport
Organization: Volvo Technology AB
Short Name: Volvo
Role: WP3 and WP5 leader
Reference: Johan Tofeldt
Volvo Technology AB (in short Volvo) and its third party Volvo Lastvagnar AB (in short Volvo Trucks), are legal entities within the Volvo Group, one of the world’s leading manufacturers of trucks, buses and construction equipment, drive systems for marine and industrial applications, and services.
Volvo has been designed to carry the leadership of both WP3 and WP5; moreover, Volvo is heavily involved to the integration of the project solutions (components and systems) into the prototypal Automated Truck.
Volvo Technology is an advanced technology and research engineering entity, developing and integrating new product and business concepts and technology for hard as well as soft products within the transport and vehicle industry. The primarily key technology areas are: Transport Solutions & Services, Energy Efficiency & Environment, Vehicle Technology & Safety, Electrical & Embedded Systems, Supply Chain and Technology Strategy & Innovation.
Organization: Agencia Estatal - Consejo Superior de
Investigaciones Cientificas
Short Name: CSIC
Role: WP2 leader
Reference: Prof. Juan Andrade Cetto
The Spanish Scientific Research Council (in short, CSIC) and its third party Institut de Robòtica i Informàtica Industrial (in short, IRI), are the main responsible for the technology development of dedicated robotics solutions optimised on the physical vehicles demands.
The Institute focuses its activity in human-centered robotics research with expertise in various areas of robotics, such as computational kinematics and geometry, computer vision, industrial robotics, mobile robotics (legged and wheeled robots), artificial intelligence, and energy systems.
The Institute currently has two urban humanoid wheeled robots based on Segway platforms (Tibi and Dabo), a four-wheel all terrain robot (Teo), a six-legged Lauron III platform, and homemade torsos that add multisensor capabilities such as laser and stereovision on a rotating head.
Organization: Halmstad University, Center for Applied
Intelligent Systems Research (CAISR)
Short Name: HH
Role: Contributor to technical WP, focus on WP 5-6
Reference: Dr. Roland Philippsen
The Swedish University of Halmstad, Högskolan i Halmstad (in short, HH), are the main responsible for the development of an autonomous path generation and path following software.
HH’s involvement have been extremely effective in the cooperation with the mapping and localization software as well as in the integration into the prototypal Automated Truck.
The Centre for Applied Intelligent Systems Research (CAISR) is a long-term research program on intelligent systems funded by the University and the Swedish Knowledge Foundation (KK-Stiftelsen - KKS) with support from Swedish Industry. The focus of CAISR is on systems that are human aware, situation aware, and to some extent self-aware. Its areas of expertise are signal analysis, machine learning, mechatronics, and robotics, with an emphasis on cooperating systems.
Several industrial partners, including established multinational players alongside technology start-ups, are collaborating with CAISR in joint projects. The two key application areas are intelligent vehicles and health care technology.
Organization: ICT Automatisiering n.V.
Short Name: ICT
Role: WP6 leader
Reference: Dr. R. Goossens
The Dutch engineering company ICT Automatisering N.V. (in short, ICT) has been identified as leader for the software integration tasks characterizing WP6.
ICT’s Automotive Vertical is an independent ‘single source’ engineering service provider, fully specialized in automotive applications as embedded software for automotive systems in the domains infotainment, multimedia and navigation and the more traditional ones like body, chassis and engine. Keywords for these domains are ‘close to hardware’, real time and safety critical.
ICT Automotive provides for all software engineering services, ranging from single modules to
complete ECU developments through to ‘fusion’ of such products in the architecture and
infrastructure of the vehicle up to the fully ‘connected’ vehicles of tomorrow. ICT Automotive also has specific know-how on incorporating functional safety for automotive control systems.
The company strength in people and technologies always allowed ICT to deliver solutions tailored to clients’ needs, using standard and state-of-the-art technology.
Webpage
The external webpage presented in Fig. 6 (http://www.cargo-ants.eu/) has been created as a divulgation tool, where to upload the project goals, the partner’s challenges and the achieved results to external parties, as stakeholders and end-users. Moreover, the external webpage is also the primary address for the participation of the project final event and demonstration.
Fig. 6 project webpage: home and link to sections
The webpage is divided into 5+2 sections, the last two being purely logistic:
➢About Cargo-ANTs: defining the scope of the project
➢News: presenting the series of events the partners have organized to align the technical developments
➢Consortium: describing the partners fields of expertise
➢Download: illustrating the use cases and the results obtained
➢Dissemination: collecting the scientific publications resulting out of the project activities
➢Final event:
➢Contact us:
Aiming at wide spreading the innovative solutions developed in the project, the sections “Download” and “Dissemination” contains a short explanation of the most intriguing scientific achievements supported by download-abled videos both for the use-cases as well as for the simulation video.
Furthermore, the section lists also the scientific publications elaborated during the project.
The following Fig. 7 and Fig. 8 demonstrate the readability of the exploitation sections of the webpage.
Fig. 7 project webpage: section Download
Fig. 8 project webpage: section Dissemination
Beside the presented papers (listed in the next chapter) the scientific methods developed in Cargo ANTs have been illustrated in the Halmstad University and IRI University webpage, to which the “Dissemination” section is linked.
The Cargo-ANTs project, "Cargo handling by Automated Next generation Transportation Systems for ports and terminals", aims to face two opposite constraints affecting nowadays world of container handling: the continuous growth in global trade and the new stringent environmental regulations, as demanded by a higher claim for sustainability. In essence, the handling of freight (both at ports as well as at hinterland connections) has to become significantly more effective for the smallest individual scale to voluminous container flows.
The Cargo-ANTs project aims at the development of innovative transhipment technologies to:
- Reduce terminal congestion, by increasing container throughput;
- Focus on continuous, automated and consistent operation;
- Insist on high controllability and reliability;
- Strive for the highest safety levels.
Existing modern technology solution is based on a grid of transponders installed on the terminal ground allowing full control on optimized Automated Vehicles. Enlarging the autonomous driving terrain is affected by severe constraints related to: large economic investments (preventing this technology to be a realistic option for smaller terminals) and the time consumption (stops of terminal operations).
The project Cargo-ANTs has developed an innovative solution widening the flexibility realizable on grid-independent infrastructure, significantly lowering the investment requirements and hence extending the highly automated and yet extremely flexible flow of individual units of freight from- and to- cargo ships in harbour terminals and within inland terminals (thus specifically to smaller terminals).
This goal has been achieved by adopting autonomous robotics research to develop a novel technology, on the following main innovation fields of investigation:
• full vehicle perception and dynamic- and static- object detection
• local- and global- mapping
• autonomous path generation and following
Among others, the strength of the Cargo ANTs solution is related to:
o Commodity: a full set of market available sensors allowing for both detection-, tracking- and classification- of objects as well as for vehicle localization;
o Awareness: a deterministic risk estimation and vehicle control is achieved by implementing a self-consistent system based on kino-dynamic model specific for the vehicles;
o Re-planning: a novel approach for mapping and trajectory generation based on planning algorithm is developed allowing for allocation of vehicle’s task, global path planning and path adaption.
The achieved solution turn out to be adaptable to different stakeholder needs, spanning from port and terminal operators to other players in the logistic chain (large warehouse, airports or large parking lots).
Although not fully addressed, a variety of potential broader impact can be easily imagined, as for instance the direct integration of selected innovative e-freight solutions which hold the potential for streamlining co- and synchro- modality logistics.
Furthermore, the autonomous path generation opens the project solution to be deployed in inter-terminal transportation and to lighten the workloads assigned to the terminal centralized management system.
In summary, the achieved project objectives are:
• an automated shared work yard for intelligent AGVs and highly Automated Trucks
• innovative solutions for planning, decision, control and safety strategies for these vehicles
• an environmental perception system and a grid-independent positioning system
These results allows for an increased performance and throughput of freight transportation in main ports and freight terminals, by maintaining a high level of safety.
Project Context and Objectives:
Present harbour- and inland- container terminals are presently limited by a non-negligible manually driven manoeuvres. Despite the progress in autonomous vehicles, the benefits have not widespread in the container handling due to various motivations, among others different shipping modalities of several terminals and safety related restriction (fencing).
Moreover, the implemented automatized systems have needed a substantial investment to equip the terminal ground with a grid of transponders allowing full control on optimized Automated Guided Vehicles, AGV. This large economic investment is preventing smaller terminals to adopt similar solutions; on the other hand, extending a ground-based infrastructure requires a stop in the business daily operations which strongly limits the versatility to enlarge the autonomous driving terrains.
The scope of Cargo-ANTs is to develop an innovative solution which allows to increase the automation level in container terminals. The deployment of this solution will exhibit the efficiency for harbour- or inland- terminals in terms of operational throughput, safety and automation achievable in grid-less infrastructures.
The project has been driven by three key area of developments:
➢ Investment reduction
➢ Safety raise
➢ Efficiency
which have led to the following results:
▪ Innovative positioning system
▪ localization and control of the vehicle (AGV or AT),
▪ increase environmental awareness and planning trajectories
▪ intelligent and cooperative interaction between ANTs
The technological innovations have been defined in order to fit into a modern concept of container handling, as depicted in the following Fig. 1
Fig. 1 Scheme for a modern concept of container handling, from vessel unload to final destination. Automated driving is realized in the primary terminal (w/ and w/o grid infrastructure) and in the connection to a secondary terminal. Manual driving will finally serve an inland terminal and the final destination.
The Automated Guided Vehicle will transport containers from the crane to the closer stocking area, the Automated Truck will transport them further to terminal perimeter in automated mode and finally can deliver- or transport containers to inland terminal, in manual mode.
The project innovations will be of major profitability for those terminals dealing with a large volume of container traffic, but can be adapted to smaller terminal as well. Furthermore, the autonomy intrinsic in the path planning tool represents a step towards decentralization, lightening the role of the global site planner to assign the task without being in charge of the low-level trajectory organization too.
Objectives
The innovative technology is based on the developments achieved in detecting the vehicle- and environmental- characteristics (full vehicle perception and local-, global- mapping) which then allows for an autonomous path generation and path following.
The object- and vehicle- detection is at the basis of a self-consistent risk estimation that permits the vehicle to deterministically recognise new potentially dangerous situations and reacts according to well established safety procedures.
The project consists of a dissemination work package and 3 technical work packages:
• Dissemination and exploitation
This work package focuses on an efficient dissemination and exploitation of the results of Cargo-ANTs and an efficient management of the intellectual property. It deals with: providing visibility of the project - disseminating the scientific results - preparing and maintaining an exploitation plan - protecting the intellectual property.
The specific objectives are:
o Demonstration of AT and AGV in simulations and in a real-life environment, exhibiting the capabilities of the prototype vehicles to the main stakeholders, customers and end-users;
o Dissemination of the scientific and technical results, identifying the results to be published, the most appropriate journals and the conferences of most interest;
o Providing visibility of the project results to a broader public, realizing a private and public website;
o Maintenance of an exploitation plan based on the needs of the customers and end-users, covering not only applications in ports and terminals but also examining how the project results can be used in other areas, i.e. automotive branch;
o Discussion and verification of the progress and results of the technology driven project with customers, end-users, authorities and specialists for its operational, legal and possibly other socio-economic aspects that may impact acceptance and implementation;
o Protection of intellectual property, including assessment of project results qualifying for IPR protection and the identification of the necessary (first) steps for patenting procedure within the regulations of the Consortium Agreement.
• Requirements, Design and Safety
This work package is responsible for the definition of a system architecture that can be applied across the project and the definition of a development process for safe and certifiable automated systems. The specific objectives are:
o Detailed use-case specifications;
o Identify requirements for each level according to V-model;
o Develop a system architecture that can be applied to the different ANTs;
o Develop a development process for safe and certifiable automated systems.
• Vehicle Localization and object detection
This work package deals with the development of software modules to localize the vehicles in a map of the environment and to detect other static and moving objects during navigation. To achieve this, information from multiple sensor sources need to be fused together.
The specific objectives are:
o Develop software modules for localization of vehicles and detection of objects and obstacles for the pre-defined uses-cases defined;
o Provide estimates on localization and tracking of all moving elements;
o Provide the developed software modules for successive integration
The developed software modules has been designed and tuned to facilitate the integration activities on the platform planned in the last work package;
o Final design and implementation description of the software architecture;
o Mapping from software modules to the hardware components present on the AGV and AT;
o Description of the algorithms used for absolute object and environment mapping;
o Validation results of the tests performed with these software modules., tested by analysis of the recorded data at test locations (harbour area and parking lot) mimicking terminal configuration.
• Planning, Decision and Action
The focus is on global and local path planning methods as developed together with the required interface to an overall site planning and to vehicle control and safety interventions. The latter are the command central for the traffic in the environment, the high level planner (by means of a global path planning algorithm) in combination with the local planner on the vehicle produces a path that the vehicle should follow, also known as re-planning or path adaption. Moreover, the work package also deals with the safety functionalities to be included in the vehicle for emergency manoeuvres, hence emergency braking functionality for objects in the path are implemented and tested.
The specific objectives are:
o Final design and implementation description of the software architecture;
o Develop global and local path planning sub-systems along with vehicle control and path following sub-systems;
o Integrate the map, communication and localization inputs from a previous work package into the planning algorithms, according to the hardware components present on the ANTs;
o Build a mock-up high-level site planning control;
o Realize a correct interaction with the site planner and integrate safety interventions into the vehicle control;
o Description and finalization of the algorithms used for task assignment, global planning and path adaptation.
• Verification & Validation, Safety assessment and Integration
This work package is responsible for the integration and the verification of the previously developed results, enabling Cargo-ANTS project to evolve more easily into a producible end-product (either being an Automated Truck or a next-generation AGV).
The specific objectives are:
o Verification of the requirements, system architecture and subsystems;
o Integration test and validation of Level 1 stakeholder requirements;
o Establish the structure of a limited safety case for certification and execution of a selected part of the development;
o Integration and testing of AT and AGV in lab environments, test-track and in a real-life environment.
Results
Following the above mentioned objectives, the next table summarizes the results achieved.
GOAL DESCPTION STATUS / NOTES
Design for integration The functional-, software- and physical- architectures have been designed and realized for the two vehicles DONE
Collision avoidance Two separate environmental perception software have been developed for the two ANTs
Analogously, the two vehicles have been equipped with different sensor set and high and low-level controllers DONE
Localization and detection A host vehicle localization and specific object detection has been improved adopting the SLAM, Simultaneous Localization And Mapping, developed in the field of robotics DONE
Planning, decision and action Autonomous trajectory planning has been realized, considering both the global map of the environment as well as the local map of the surrounding of the vehicle DONE
Decision and Action would have required the interaction with a container terminal management system, which was out of the scope of the project. Nevertheless, a mock-up example provides the input to add in the future these strategic information DONE
Safety assessment The goal within the safety aspects was intended to focus on the “limited safety case description”, and more precisely on the items to be considered for the realization (increased Technology Readiness level) in the production of the prototypal ANTs.
This activity has not been completed due to the bankruptcy of APTS. This subcontractor was in charge of providing access to an AGV as well as managing the levels and providing process management for certification, based on its experienced in the area of automated vehicles (bus, containers carrier, guidance systems) CANCELLED
Integration The software integration activities have undergone the issue of AGV availability. But all software components are integrated and tested in simulators to show the complete functionality. Software testing has been performed with recorded sensor data
Thanks to the presence of the Truck manufacturer, the instrumentation and final software integration into the Automated Truck was fully completed DONE
On the other side, the impossibility to access a prototypal AGV did not allow the physical integration of the dedicated software.
The decision to develop an AGV-simulator allowed to carry on the integration of modules, interfaces and software communication on the virtual vehicle PARTLY DONE
Demonstration As a consequence of the missing AGV, the final event was splitting the achievements for the two vehicles. The demonstration took place in Gothenburg and the main stakeholders, customers and end-users were invited
For the Automated Truck, the exhibition provided the evidence of the innovative solutions developed by the project partners.
Obviously, coordination with the second ANT could not be proven PARTLY DONE
For the AGV, the set of use cases defined in WP3 could be reproduced only via simulation, demonstrating both the vehicle control (including collision avoidance manoeuvres) as well as the compatibility and interaction with the localization and re-planning software DONE ONLY IN SIMULATIONS
The demonstration run during the final event are described in Section 5.
The left picture of Fig. 2 simulates an AGV running on a typical terminal configuration (global mapping), detecting obstacles as the container corners and following its predefined trajectory; the right plot of Fig. 2 illustrates the path adaptation in case of conflicting ANTs trajectories (via the Covariant Hamiltonian Optimization for Motion Planning).
Fig. 3 shows the integration of two software modules for local mapping- and local planning- functionalities.
Fig. 2 Left: localization and mapping in AGV simulation – Right: path adaptation for conflicting routes
Fig. 3 integration of local planning with path adaptor
Finally, the integrated software modules have been adopted to run simulations and on real tests on the specific ANTs. Fig. 4 illustrates on the left the simulation of an AGV on a container terminal (yellow lines detecting container corners) and on the right, an AT running specific use cases on a test track (yellow lines detecting smaller objects).
Fig. 4 Simulated AGV on a container terrain and real execution of localization module for the AT on a physical test track
Project Results:
Final demonstration event
A final demonstration event was held in Gothenburg on August, 29th, at Stora Holm.
The event has been advertised by means of a dedicated flyer and has been advertised via mails, private communication and obviously the project webpage. The latter has represented the primarily method for project external institution to register its attendance to the final event.
The event consisted on a series of project results presentations including simulations of all case scenarios involving AGVs and ATs, as well as three live demonstration with a real Volvo truck showing the following autonomous navigation skills: autonomous map building and localization, real-time localization in a previously built map, and autonomous driving and obstacle avoidance using a local obstacle map.
The agenda included
a. internal discussions within the consortia about lessons learnt from the project for the AGV and AT use cases,
b. public introduction to Stora Holm,
c. technical project presentations of achievements,
d. AGV simulation results,
e. AT test runs,
f. conclusions.
The final demonstration event was an important achievement, showing successfully the most important autonomous navigation components running on-board a real truck. A representative from the European Commission attended the event and was invited to ride inside the truck during the real demos.
Fig. 9 Group picture of the EC representative and of the partners
The following from Fig. 10 to Fig. 13 include some of the most relevant results shown during this final event:
Fig. 10 Simulation results of the kinematics of an AGV and of a port container terminal.
Fig. 11 Simulation results of AGV localization in a previously recorded map
Fig. 12 Real-time execution results of localization and mapping of an AT at Stora Holm
Fig. 13 Real-time execution of obstacle avoidance and path adaptor at Stora Holm
Use and Dissemination of Foreground
The dissemination of foreground results of the Cargo-ANTs project took place by mean of various channels, as workshops and events, publication at scientific journals and conferences, the web site, live demonstrations and media appearances.
Intellectual Property
Definitions of the EC recognize two distinct types of intellectual property (IP) in FP7:
• Background IP (BGIP) is the IP that partners bring with them to the project.
• Foreground IP (FGIP) is the IP generated during the project
Three different project results (foreground) were identified during the final demonstration event that are subject of intellectual protection: the planning software, the radar odometry module, and the localization and mapping software. These results are succinctly described in section 6 in this document. A more elaborated description of these results is two folded: the public part of these developments is included in the material already presented in the related scholarly publications that resulted from the project. The restricted part is the sole ownership of the groups that participated in their development, and more specifically, of the academic partner Halmstad University for the case of the planning software, and of CSIC for the case of the stereo radar odometry and the localization and mapping software.
The intellectual property right officers in both academic institutions will investigate on the possibility of requesting protection and licensing of these developments.
Potential Impact:
The solution Cargo-ANTs provides, is relevant for different stakeholders, specifically for port and terminal operators, but also for other players in the logistics chain, including those that are not only occupied with (sea-) container handling. These include major warehouses and factories that have to move goods from A to B at their site, airports that are responsible for luggage transport, and not to forget the automotive industry that is working on future concepts to find and automatically move towards a parking lot in an open parking place. It is clear that the solutions of Cargo-ANTs are relevant for a wide audience, also outside the logistic chain that is responsible for container handling. Therefore, the dissemination strategy of the Cargo-ANTs project is an essential and central aspect to maximize cross-fertilization.
To come up with an adequate exploitation plan, correctly steered by the market agents, the consortium sought during the duration of the project active involvement of stakeholders via the Liaison group to propose and approve use cases, to agree on systems requirements, to discuss implications on the workforce, and to monitor progress and results.
The empowered work package in charge for dissemination and exploitation has experienced an increased number of activities in the second half of the project.
From the dissemination of scientific results, a list of journal articles and conference papers have been presented describing various project contributions.
Towards a wider audience, the consortia have repeatedly inquired the engagement and availability of the cargo terminals of the port of Rotterdam and the port of Gothenburg. As alternatives to these opportunities, other sound locations have been visited as the test-track “AstaZero” (Sweden) and the “Stora Holm”, close to Gothenburg.
Moreover, a web site has been updated containing appealing information for stakeholders and end-users.
Dissemination
Stakeholder needs are summarized as Level 1 requirements in deliverable 3.1. They are summarised here through a series of agreements with the members of the Liaison Group.
on use cases
Stakeholders are mainly interested in advances in use cases involving multi-vehicle scenarios and safety situations. They are primarily interested in use cases demonstrated on the site itself, but also to nearby facilities, in the direct vicinity of the harbour site.
on operational requirements
The main identified operational requirement is to use a grid-less solution for autonomous navigation of both AGV's and AT's. Ground magnetic guiding grids are rigid infrastructures that require a significant (financial) investment from the start, maintenance during its use and have very limited flexibility on operations.
on operational performance
Performance is mainly related on robustness of the solutions, since they have to work 24h/24h, 365d/365d, under harsh environment conditions such as fog, dust, rain or darkness. Stakeholders will adopt new technologies only if they are robust enough to justify cost investments with low risk on innovation. Besides robustness, efficient interaction between the vehicles (no deadlocks and close following) is of importance to come to an optimal use and efficiency of the site operation.
The critical components for a final robust system are all navigation layers, from low level steering and drive controllers, up to high level localization within the map. The latter is especially critical since it closes the navigation loop, while it relies on state of the art sensing and perception algorithms and technologies. Therefore, long-term operational testing would be required, before any commercialization step.
Exploitation
The exploitation plan includes an analysis of the socio-economic acceptance of technology and concepts by stakeholders (including end-users, workforce, and local authorities). Legal aspects concerning Cargo-ANTs solution, in both harbour sites and autonomous vehicles are also analysed, as well as management of intellectual property of foreground project results. This document also reports the testing, validation and certification steps that would be required to transfer the knowledge of technology prototypes created by the project to final ready-to-market solutions. Finally, this deliverable also identifies a set of separate components with enough potential to be exploited independently, in the wide domain of intelligent transportation systems.
A dedicated deliverable also reports on several achievements, especially on identifying market needs, and listing the steps from prototype to end-product, including the development phase, the testing activities & the required certification. Individual software components developed in WP’s 4 and 5 such as LIDAR polyline mapping, stereo radar odometry, or local path adaptations have been identified as components that could be exploited separately.
The stereo radar odometry module is shown in Fig. 5 (by means of polyline mapping and local path adaptation modules)
Fig. 5 Stereo radar odometry module.
Viability
The exploitation plan includes an analysis of the socio-economic acceptation of technology and concepts by stakeholders (including end-users, workforce, and local authorities). This document also reports the testing, validation and certification steps that would be required to transfer the knowledge of technology prototypes created by the project to final ready-to-market solutions. Finally, this deliverable also identifies a set of separated components with enough potential to be exploited independently, in the wide domain of intelligent transportation systems
Acceptance
Harbour workforce is the main user of Cargo-ANTs solution. User-acceptance shall be achieved based on the Validation Test and Functional Safety Case. Both depend on the System Architecture (pass/fail criteria depending on Level 1 requirements).
Concerning performance metrics, current AGV's at the ECT terminal in Rotterdam work on average 5.000 hours per year and drive approximately 30.000 km per year. There exist Mean Time Between Failures (MTBF) standards specified in the contract between the supplier, VDL and ECT. MTBF of an autonomous navigation system such as that developed in the Cargo ANTs project could be computed based on a series of simulated experiments where sensor data can be progressively corrupted to emulate different noise and outlier conditions. In that line, the Cargo ANTs project has set up a simulated environment based on the Gazebo Robotics simulator, which has been used to develop the navigation software components.
The truck industry has also performance metrics but related with the driver as a user. For the case of driver-less vehicles, as a first approach, trucks could be considered as AGV’s in terms of metrics. In that case other interesting metrics can be computed:
o Energy consumption
o CO2 emissions
o Workforce reduction
o Work accidents reduction
o Container Stack surface increment
Cost benefit analysis of the solution
The benefit of a similar AGV should however be considered on top of a series of variables very much related to the area of application.
A concrete evaluation of the benefit provided by a smart AGV should consider:
a. The investment saved in the infrastructure, avoiding very expensive transponder grid realization on the ground.
b. The versatility to enlarge the terminal area, increasing the number of running AGV will not involve a stop in the terminal workflow while enlarging a transponder grid ground would require a long stop for the realization and a transition period to integrate the new terrain in the centralized control system, CCS.
c. The self-planning of the smart-AGV, which reduces the workloads of the CCS. In other words, the probability to exceed the CCS capacity will be hugely decreased avoiding the control system to become a bottleneck for the terminal operations.
d. An higher level of automation in inter-terminal transportation can be achieved. This will de facto extend the area of competence of AGV and AT within the terminal.
e. The public road could be finally approached by smart-AGV thanks to the gained independency from the transponder grid ground.
In order to compare solutions, costs of ground magnetic grid installation and maintenance are briefly analysed, which are mainly labour-oriented and depend on the dimensions of the site. Under the hypothesis that a magnetic grid infrastructure in a port setting cost is 250K€/hectare, and a mean terminal port in Europe is around 4 hectares wide, an investment of about 1M€ is avoided if using a grid-less solution
proposed by Cargo-ANTs. Moreover, magnetic grid have maintenance cost associate depending on the level of incidences, and also implicitly deploys a fixed layout of the port, which can also cause a loss of incomes due to worse and less flexible operations.
Instead, Cargo-ANTs solutions exploits the grid-less approach, at a minimal extra cost per vehicle of the order of 40K€ in production line (mainly LIDAR sensors and on-board computer). Motion controllers and encoders are already present in a grid-based AGV. Required extra mechanical design for component integration and software development cost do not increment the production cost per vehicle.
Moreover, Cargo-ANTs grid-less navigation solution could potentially also be used in Guidance Systems for bus & coach, reducing operational constrains upon the customers infrastructure, providing a more efficient system at lower operational cost.
Investment and Maintenance Issues
The consortium will contact VDL Steelweld / ECT to know details on this issue, especially for AGV’s operating in the Rotterdam harbour.
Due to the missing access to the prototypal AGV, no information is available related to the maintenance cost of the final AGV. Moreover, the lacking engagement of the manufacturer as well as the reluctance of the end user to share the related data of the presently running AGV, does not allow a complete comparison on these topics.
In the other side, the AT vehicle has been built in the context of the project, according the assigned budget. However, the experimental sessions within the Cargo-ANTs project were focused on developing and testing different levels of the navigation solution, but not in long-term operations, so maintenance operations were out of the scope of these sessions.
Nevertheless, it is worth to mention the following aspects characterizing the innovative solutions developed within the Cargo-ANTs project. The environmental perception set has been constructed out of market available sensors. The maturity of these sensors is not considered to be harming the specific maintenance costs but the LIDAR sensors are very vulnerable when large G-forces are applied (e.g. when a crane drops a container on the AGV). The use of LIDAR systems with moving part will introduce more maintenance to the AGV and also the cleaning of the front glass has to be done regularly. Instead, RADAR systems are less vulnerable to these external forces, but are not enough for mapping and localization requirements.
On the other hand, it is the engineering design to mount this sensors that can affect the Mean Time Between Failures, a parameter very much dependent on daily conditions, among others soft/hard container loading, status of the road, vehicle speed and crashes.
From prototype to end-product : development phase, testing & certification
By verifying, validating and integrating the positioning system into an AGV on a potential SIL-level, stakeholders and future customers would gain more confidence and trustability into the feasibility of such a system, which could then lead to a PMC, Product Market Combination. Potentially integrating the positioning system into a next generation of Computerized Guidance Systems would broaden our level of service and expertise for development of automated guidance systems for usage in different domains. The achievement of a SIL-4 certification for a safe and efficient next generation AGV opens the opportunity to set a new world wide technical standard for public transport market.
Successful validation of a fully autonomous system, in particular an AT, in a closed environment would boost the confidence of all the stakeholders by pushing it one level closer to being technically feasible and commercially viable solution in the area of safe autonomous systems for motorways, urban roads. By introducing ATs in the truck industry we open up the floor for new standards in the domain and thereby facilitate and demonstrate the use of automation for Safe and efficient transport.
The main steps from prototype to end-product would be the following:
1. Find customer interested in buying Cargo-ANTs solution.
2. Perform System Definition Review.
3. Specify requirements based on Cargo-ANTs project result and customer demands.
4. Create Functional Architecture (Design).
5. Create Computational Architecture.
6. Procure HW requirements (Sensors, Systems, etc.).
7. Create Component level Architecture.
8. Develop the research platform in the truck (prototype in TRL4).
9. Perform Critical Design Update.
10. Test end-product - confirm ISO26262.
11. Perform System Acceptance Review.
12. Demonstrate to the stake-holders and to potential
13. Perform Vehicle Acceptance Review
Exploitation of separated components
Platform
The innovative solutions realized in Cargo ANTs have been developed based on the needs of container terminals, dealing with a large number of daily container handling.
However, the application of environmental perception, chip market sensors, ego vehicle localization, object detection and classification, local and global mapping interaction as well as trajectory computation and re-planning are not specific for harbour environment.
An educated guess immediately leads to think similar solutions to be applied in:
• Airports: both for luggage handling- as well as for passenger transportation- system.
• Small terminals: adapting the dimensions- and functionalities- of automated guided vehicle.
• Factory storehouses, dealing with other container types, not necessarily shipping containers.
Planning Software
Halmstad’s contributions in Cargo-ANTs consisted of an integrated planning framework for the motion planning of multiple vehicles, this included methods for task allocation, global path planning and local trajectory planning. Halmstad expects to exploit such modules in current and future research projects.
Specifically, the local and global path planners are expected to be employed in current projects in Halmstad University’s research profile CAISR (Centre for Applied Intelligent Systems Research). Thus, the planning software is foreseen to be exploited in the research activities conducted at the intersection of CAISR’s research area of Mechatronics/Robotics and Intelligent Vehicles application area. One of the PhD thesis has been worked on studying the present container terminal scenarios and adapting the existing site planner developed for CargoANTs to suit a more realistic scenario.
The developed software for local trajectory planning method is presently been investigated and employed in an another project AntWay for the development of Autonomous Trucks in collaboration with Vinnova, Volvo Trucks AB, Kollmorgan and Chalmers University.
Likewise, the modules developed for planning will be extended as part of a current project proposal submitted to the Swedish Research council (Vetenskapsrådet) with focus on Belief Space Planning for Robust Autonomous Navigation. In such a project we expect to extend the planning methods developed in Cargo-ANTs to consider the uncertainty generated from the vehicle’s perception
Localization and Mapping Software
The Localization and mapping (SLAM) solution developed for the Cargo-ANTs project is fully flexible to be exported to other markets and applications. It is based on an optimization of geometric constraints imposed by sensing the environment. In the Cargo-ANTs projects, such constraints are provided by 2D LIDAR sensing and wheel odometry. A signal processing step which identifies straight segments, corners and polylines is applied to raw LIDAR ranging data, and this geometric features are used to impose such constraints for the optimization. However, many other geometric constraints, coming from 2D LIDAR, but also from other sensing modalities (specially cameras, 3D scanners, RADAR’s, RF beacons or GPS) can be easily adopted by the localization and mapping solution, leading to a more robust and accurate final performance.
This approach definitively leads to have a general solution for autonomous vehicle localization and mapping, which can be applied beyond harbour environments. Shuttles operating in restricted areas or indoor logistics may be among the markets that could adopt from now the developed approach.
RADAR odometry has been also investigated, resulting in an innovative approach able to compute the vehicle odometry by using only a pair of on-board automotive RADAR devices. This result can be also exported to other domains requiring complementary approaches for the odometry estimate.
Summary
Regarding the exploitation opportunities that the Cargo ANTs project could offer, the following conclusions can be drawn:
➢ The Liaison members are aligned with the use cases and specific requirements.
➢ At the end of the project, the viability of the concept is still hard to quantify in detail due to a lack of maturity of the solution. However some estimates are done which point out that the grid-less approach is interesting in budgetary terms.
➢ Legal aspects and processes to follow are known as far as it concerns confined areas. As soon as it comes to (semi-) public roads, actual legislation is poor.
➢ IP protection will be analysed for three specific software modules in Cargo ANTs: the path planning module, the localization and mapping module, and the stereo radar odometry module.
➢ Main steps from prototype to end-product are identified.
➢ A set of separated components are identified with enough potential to be exploited independently, in the wide domain of intelligent transportation systems.
List of Websites:
The project consortium brought together research institutes and vehicle manufacturer, thus establishing a balanced framework for the implementation of innovative robotics solutions to prototypal vehicles.
The communication and the logging data exchange has been facilitated by an internal repository while the dissemination to a wider public has been pursued by mean of the project webpage.
Consortium
The project consortium is composed out of 6 partners, as shortly described in the following list.
Organization: Nederlandse organisatie voor toegepast
natuurwetenschappelijk onderzoek
Short Name: TNO
Role: Project Coordinator and WP1 and WP4 leader
Reference: Dr. Thomas Chiarappa
As first proposer and project coordinator, TNO is participating actively on every work package.
The Netherlands Organization for Applied Scientific Research (in short, TNO), is a knowledge organization for companies, governmental bodies and public organizations and it creates, develops and applies scientific knowledge in various domains ranging from industry to society aspects.
It is the fundamental goal of TNO to direct its research activities toward creative and practical innovations in the form of new products, services, and processes, fully customized for businesses and governmental institutions. One of the research themes of TNO is “Mobility”, comprising topics as (intelligent) vehicle technology, ITS, cooperative and automated driving, road side technology, infrastructure technology, intelligent traffic management, environment and human factors in transport
Organization: Volvo Technology AB
Short Name: Volvo
Role: WP3 and WP5 leader
Reference: Johan Tofeldt
Volvo Technology AB (in short Volvo) and its third party Volvo Lastvagnar AB (in short Volvo Trucks), are legal entities within the Volvo Group, one of the world’s leading manufacturers of trucks, buses and construction equipment, drive systems for marine and industrial applications, and services.
Volvo has been designed to carry the leadership of both WP3 and WP5; moreover, Volvo is heavily involved to the integration of the project solutions (components and systems) into the prototypal Automated Truck.
Volvo Technology is an advanced technology and research engineering entity, developing and integrating new product and business concepts and technology for hard as well as soft products within the transport and vehicle industry. The primarily key technology areas are: Transport Solutions & Services, Energy Efficiency & Environment, Vehicle Technology & Safety, Electrical & Embedded Systems, Supply Chain and Technology Strategy & Innovation.
Organization: Agencia Estatal - Consejo Superior de
Investigaciones Cientificas
Short Name: CSIC
Role: WP2 leader
Reference: Prof. Juan Andrade Cetto
The Spanish Scientific Research Council (in short, CSIC) and its third party Institut de Robòtica i Informàtica Industrial (in short, IRI), are the main responsible for the technology development of dedicated robotics solutions optimised on the physical vehicles demands.
The Institute focuses its activity in human-centered robotics research with expertise in various areas of robotics, such as computational kinematics and geometry, computer vision, industrial robotics, mobile robotics (legged and wheeled robots), artificial intelligence, and energy systems.
The Institute currently has two urban humanoid wheeled robots based on Segway platforms (Tibi and Dabo), a four-wheel all terrain robot (Teo), a six-legged Lauron III platform, and homemade torsos that add multisensor capabilities such as laser and stereovision on a rotating head.
Organization: Halmstad University, Center for Applied
Intelligent Systems Research (CAISR)
Short Name: HH
Role: Contributor to technical WP, focus on WP 5-6
Reference: Dr. Roland Philippsen
The Swedish University of Halmstad, Högskolan i Halmstad (in short, HH), are the main responsible for the development of an autonomous path generation and path following software.
HH’s involvement have been extremely effective in the cooperation with the mapping and localization software as well as in the integration into the prototypal Automated Truck.
The Centre for Applied Intelligent Systems Research (CAISR) is a long-term research program on intelligent systems funded by the University and the Swedish Knowledge Foundation (KK-Stiftelsen - KKS) with support from Swedish Industry. The focus of CAISR is on systems that are human aware, situation aware, and to some extent self-aware. Its areas of expertise are signal analysis, machine learning, mechatronics, and robotics, with an emphasis on cooperating systems.
Several industrial partners, including established multinational players alongside technology start-ups, are collaborating with CAISR in joint projects. The two key application areas are intelligent vehicles and health care technology.
Organization: ICT Automatisiering n.V.
Short Name: ICT
Role: WP6 leader
Reference: Dr. R. Goossens
The Dutch engineering company ICT Automatisering N.V. (in short, ICT) has been identified as leader for the software integration tasks characterizing WP6.
ICT’s Automotive Vertical is an independent ‘single source’ engineering service provider, fully specialized in automotive applications as embedded software for automotive systems in the domains infotainment, multimedia and navigation and the more traditional ones like body, chassis and engine. Keywords for these domains are ‘close to hardware’, real time and safety critical.
ICT Automotive provides for all software engineering services, ranging from single modules to
complete ECU developments through to ‘fusion’ of such products in the architecture and
infrastructure of the vehicle up to the fully ‘connected’ vehicles of tomorrow. ICT Automotive also has specific know-how on incorporating functional safety for automotive control systems.
The company strength in people and technologies always allowed ICT to deliver solutions tailored to clients’ needs, using standard and state-of-the-art technology.
Webpage
The external webpage presented in Fig. 6 (http://www.cargo-ants.eu/) has been created as a divulgation tool, where to upload the project goals, the partner’s challenges and the achieved results to external parties, as stakeholders and end-users. Moreover, the external webpage is also the primary address for the participation of the project final event and demonstration.
Fig. 6 project webpage: home and link to sections
The webpage is divided into 5+2 sections, the last two being purely logistic:
➢About Cargo-ANTs: defining the scope of the project
➢News: presenting the series of events the partners have organized to align the technical developments
➢Consortium: describing the partners fields of expertise
➢Download: illustrating the use cases and the results obtained
➢Dissemination: collecting the scientific publications resulting out of the project activities
➢Final event:
➢Contact us:
Aiming at wide spreading the innovative solutions developed in the project, the sections “Download” and “Dissemination” contains a short explanation of the most intriguing scientific achievements supported by download-abled videos both for the use-cases as well as for the simulation video.
Furthermore, the section lists also the scientific publications elaborated during the project.
The following Fig. 7 and Fig. 8 demonstrate the readability of the exploitation sections of the webpage.
Fig. 7 project webpage: section Download
Fig. 8 project webpage: section Dissemination
Beside the presented papers (listed in the next chapter) the scientific methods developed in Cargo ANTs have been illustrated in the Halmstad University and IRI University webpage, to which the “Dissemination” section is linked.