Final Report Summary - AVITRACK (Aircrafts surroundings, categorised Vehicles & Individuals Tracking for apRon's Activity model interpretation & Check) The AVITRACK project addressed the specific case of automatically supervising commercial aircraft servicing operations at a designated airport. AVITRACK developed an intelligent survey system on the apron, addressing aircraft, vehicles and people's presence and movements for the apron real time management system.On the apron area, aircrafts are parked and serviced by specialised ground support vehicles (fuel, catering, baggage, small maintenance). The efficiency and security of the handling (servicing) process are significant issues for airport authorities, aircraft operators and citizens.The AVITRACK apron's activity model is able to model, interpret and check normal servicing operations and to track objects and persons over the airport parking area. The system is based on sensors used for video-surveillance linked to a smart system. The system outputs are aimed at airport operators, handlers and security services. AVITRACK addressed 3 strategic objectives: decreasing aircraft servicing costs, reduction in take off delays due to turn around time over-run, increase security.To date, no system has been developed to analyse so complex scenes, in real time. The diversity of the operations and the materials implemented around the planes, added to the movements of the personnel to carry out the stopover operations in a limited time, compose a very complex scene.Simultaneous convergence of diverse materials and personnel around the planes to carry out stopover varied operations constitute a very complex unit to observe and analyse automatically. Some applications of video monitoring start to emerge proposing the recognition of simple event, like deposit of a luggage in a corridor or the passage of a person in a given zone. Recent work analyses the behaviour of one or two people since a fixed camera. The work undertaken within AVITRACK constitutes an incontestable projection in the development of the automatic recognition of scenarios.The achievements of the project can be summarised as follows:- operational test site on Toulouse-Blagnac airport; 8 video sensors are installed, on the echo-40 apron, viewing the aircraft;- handling operations analysis allowed formalising a representation for each turnover activities;- 3D numeric model of apron area and involved vehicles and aircraft;- distributed architecture for video streaming, video tracking, event recognition and HCI;- video tracking software for people and vehicle detection around the aircraft;- real-time model based vehicle recognition;- multi-view data fusion for 3D object location;- artificial intelligent modules for events recognition close to the aircraft, in real time;- efficient Beta prototype processing, in real time, video tracking, scene understanding and reporting;- simultaneous video streaming and recording architecture.The first finality of AVITRACK prototype was the optimisation of the supervision of the stopovers, by an information feedback, in real times, of all the operations in progress on the airport platform. The actors of each airport platform interested by this process of optimisation could be: airline companies, managers of airport, handlings, air traffic as well as the services of safety.The objectives are:- for the airline companies: to ensure the punctuality of the departures,- for the airports: to optimise the exploitation of their capacity;- for the people receiving benefits in load of the operations: to minimise times of stopover;- for the air traffic: to ensure the fluidity of the traffic by improving management of the car parks.AVITRACK combines innovative technologies in terms of video tracking, apron activity model and understanding artificial intelligence video sensors are fixed to the airport building, viewing the apron area. The assembly of cameras monitors the aircraft-parking zone, creating digital images in which individuals, objects and vehicles are localised and identified. Advance image processing software detects movement in the scene on a per-frame basis. Detected objects are then classified as individuals, vehicles or mobile objects. Using geometric models the vehicles are categorised by functionality (e.g. loader, truck, tanker, etc.). Multi-sensor data fusion technique is used to create a real-time 3D representation of activities and movements, which can be interpreted by an intelligent computer programme.A geometric model of the apron is formed including the operational function of specific areas (as waiting area, ERA, tanker area, gateway evolution area, GPU area). The aircraft functional model, containing all potential contact points (as refuelling point, passengers' doors, cargo doors, etc.) completes the 'apron scenes and actors database'. This static apron representation has to be combined with dynamic scenarios. A scenario is a scheduled arrangement of basic and combined events. Each event is the result of the semantic description of activities in connection with multiple individuals and vehicles around the aircraft. After the analysis of the handling process, using a dedicated ontology, a scenario database is created. The 'apron activities model' database covers a large part of all the handling operations observed around an aircraft.The video-tracking module captures the activities around the aircraft. The 3D-generated current situation map (people, aircraft and vehicle location) is combined with the 3D models of the apron area. This observed apron situation is compared, in real time, to the 'apron activities model', using artificial intelligence technologies. Every operation and movement around the aircraft, from the simple to the more complex, is analysed and identified. All pertinent events are dispatched for remote control to the apron manager, airport authorities, airlines and security services.To answer to the air traffic grown, airports seek to reinforce their capacity of reception. The improvement of the management of the existing means contributes directly to this objective. The interest of the establishment of such system of supervision is as beneficial as the traffic flow is more tended. The platform airports ('Hub') will withdraw a strong benefit by exploiting such supervision of the stopover operations (Lyon Saint Exupéry, Zurich, Munich, etc.).The deployment of system AVITRACK on the airport platforms will also increase safety near planes. AVITRACK brings an additional control of the behaviour of people near the plane, and the detection of the approach of the vehicles and any objects.The innovations of project AVITRACK interest many other applications. The automatic monitoring principle and recognition of complex events dedicated to the supervision of specific operations meet in many other transport activities (railway, subway, port, etc). The concept of AVITRACK can be declined to recognise the specific scenarios to each context and each metier. Related documents Final Report - AVITRACK (Aircrafts surroundings, categorised Vehicles & Individuals Tracking for apRon's Activity model interpretation & Check)
