Final Report Summary - SOS ((Multi-) Senor Offshore Safety system)
Executive Summary:
A disturbing number of man overboard incidents occur every year in the maritime transport industry. Unfortunately, most of these incidents are fatal. The loss of many lives each year and associated costs rapidly call for a change. SOS ((Multi-) Sensor Offshore Safety System) helps to reduce the loss of lives at sea.
The project partners forming the international R&D consortium of this project has worked on the development of SOS, a new concept and potential solution for the man overboard problem. The new concept has addressed the main shortfalls of existing systems and made extensive use of new technologies to combine commercially available state-of-the-art sensors with bespoke software solutions. The result has become a modular and scalable product which is intended to be a reliable and cost-effective overboard alert system for the maritime transport industry.
In order to design the final product, the project has gone through a thorough design project which has considered the required operational functions, the characteristics of the situation where a person falls overboard and the environmental conditions present in typical and adverse situations. In addition the cost/performance parameters of available and emerging sensor and software technologies has been considered and candidate solutions were assessed for operational performance, ease and cost of installation and also considered against the business strategies of the participating partners.
Based on the design efforts, the project has concurrently gone through several stages of experimentation and verification of sensor components and software solutions by way of in house testing and field trials. This has led to some solutions having been discarded, while new components have been introduced.
The resulting configuration with distributed sensor pods and a central processing system has been manufactured in the form of a prototype, which has been thoroughly tested and was demonstrated in actual scenarios at the end of the project. Furthermore a design for a cost-reduced implementation has been developed and will form the basis for a further development/industrialisation of the system.
After a larger number of tests were executed the certification agency taking part in the consortium (DNV-GL) verified the actual performances and issued a technology certificate verifying the concept and the feasibility of the solution. We furthermore invite any end-user ship or platform operator to take part of further testing of the system.
Foreground IPR of the project has been protected by a solution patent and application patent.
A project demonstration video has been prepared. This video can be found on www.overboardalert.com
Project Context and Objectives:
SOS ((Multi-) Sensor Offshore Safety System) is intended to reduce the loss of lives at sea. It is a new automatic man overboard-alert system, which uses advanced integrated multi-sensor technology to detect any person falling overboard in real time and immediately alarm the crew in such an event.
The project context
Almost every day, someone is going overboard (MoB) from a vessel or an offshore installation. The majority of human loss at sea is typically from larger transport vessels. The cruise industry alone has on average one MoB incident a week, resulting in loss of life. Most cases remain unresolved.
Generally speaking, the main problem in most of the MoB incidents is that nobody notices you fall. And even if the person going overboard is observed by someone (which is rarely the case), alarming the crew is most often accomplished too late. It sometimes takes hours until the crew actually becomes aware of the incident — too late for a successful search and rescue operation and most often too late to locate the victim at all, whether alive or drowned.
The ever-increasing number of fatalities in MoB incidents is of grave concern for the entire maritime industry and action to resolve this is given high priority.
For example, the United State's Cruise Vessel Security and Safety Act (CVSSA) mandates that from 27th January 2012 cruise ships must have a man overboard monitoring system, which shall "integrate technology that can be used for capturing images of passengers or detecting passengers who have fallen overboard, to the extent such technology is available". Many other seafaring nations enforce this act as well. As a consequence, numerous maritime transport companies have recently assessed and validated the available technologies, but have been unable to find a system that is reliable and cost-effective. There is currently no available equipment on the market to meet the requirements of CVSSA.
Timing is critical. Only when a man overboard incident is reported to the crew immediately, will it be possible to locate the drowning person in time for recovery. Therefore, every second counts in alarming and starting the necessary search and rescue operations. Early detection increases the chance of a successful recovery.
Apart from the loss of lives at sea, each incident triggers a complex and costly search and rescue operation. Such an operation typically involves turning around the vessel, launching smaller rescue boats for searching and calling for assistance from shore-based units or nearby ships. Direct costs are significant and typically exceed € 1,000,000, depending on the scope of the operation. In addition, shipping operators face additional costs and challenges, such as legal procedures and loss of reputation.
The longer the time span between the time of the incident and the time at which the crew is alarmed, the more complex and costly the search and rescue operation will become. And above all, with rapidly diminishing chances of success.
The project objectives
The majority of MoB incidents go unnoticed. The project objective was to create a system which aims to detect any person going overboard and to alert the Bridge/Control Room instantaneously.
Due to its instantaneous alerting capabilities, SOS reaches two other objectives. It contributes to reducing both the loss of lives and high expenses in connection with search and rescue operations.
The projects ultimate goal was to demonstrate SOS as the first reliable, cost-effective and real-time man overboard detection and alerting system.
There were many technical aspects that needed to be covered in developing a complete and reliable sensor system like this. The concept needed to be realized and RTD (Research and Technological Development) work needed to be executed. The system then needed to be thoroughly tested, calibrated/tuned, and validated. This effort had to be carried out by complementary organisations in order to exploit the full potential of the base technology.
Following the SOS project kick off, a number of end-user scenarios were developed with help from end-users. These scenarios reflect the most likely user cases of the system, in prioritized order. These scenarios were thereafter converted to a detailed list of functional requirements, which finally resulted in the chosen system architecture. A significant part of this process was the understanding of the relative merit and the actual technical feasibility of different possible adjunct sensors supporting the overall SOS system. This was accomplished by addressing the various user scenarios and functional requirements in particular the more demanding requirements such as all-weather/all-day operation. This led to the multi-sensor approach.
The overall performance metrics of the system have been defined to give an accurate measure of the system´s operational capabilities with respect to actual man-overboard scenarios. The main performance metrics for the system are: detection probability (correct positives), false alarm rates (false positives) and overall system response time. Fundamentally these performance metrics are difficult to adequately measure due to the following facts:
- Detection probabilities are required to reach very high values. In order to give accurate values for detection probability a very large number of tests need to be performed.
- Compounding to this issue, the actual test situations depend on numerous environmental factors as well as test scenarios, making the required test volume for high significance evaluation very high.
A further issue is that the test situation does not fully verify the issue of false negatives (missing detections). In an operating environment false negatives can occur due to numerous factors, also for those not anticipated in the test plan. (A number human ‘look-alikes’ were deployed during the testing).
Verification of the requirement of extremely low false alarm rates (false positives), in the order of only few instances per e.g. 1 year, would necessitate prohibitively long test periods.
The unique feature of the SOS system is the multi-domain approach using disperse sensors and sequential event processing. For the test system a basic set of sensors with a limited selection of operations have been implemented, in a final system a more comprehensive approach may be used that will gain improved metrics relative to the one under testing.
As a solution to these aspects, the project evaluated system metrics using a modelling tool working in parallel to the test procedures to evaluate the final performance based on single sensor and single event testing. Extrapolation from the single sensor to compound system performance is based on multivariate estimation and individual sensor performance extrapolation that is furthermore based on signal to noise values and distribution characteristics. Additionally, estimation of the sequential detection performance and overall time lag and latency is done using conditional estimation procedures.
A large number of meetings with representatives from SMEs and the R&D partners have been held during the project period with the objective to define the system that best suits operational requirements within the constraints of the project, to identify the best locations for installing the various components of the system, define all mechanical and electrical interfaces, cabling requirements and a time schedule. Cost reduction exercises have been conducted to optimize cabling and interfaces. An acceptable solution that presents the best compromise between cost and development time has been found.
There has been some discussion of the number of sensors and their positions during the project. Due to the relatively high cost for a prototype installation, the number of sensors had to be reduced to a number that was within the budget of the project, but at the same time sufficient to illustrate the concept of the system. Ideally a 360-degrees coverage of the ship is desired, but a sector of the vessel was selected for concept demonstration. There’s also some issues to post install such a system in a way that does not affect the normal operation of the ship and protect the equipment from parts of the ship where use of cranes, passengers, etc. may appear and possible can damage a post installed system. Also there are some issues to minimize the blind zones of the system. This issue has been solved by installing two or more camera pods, each containing cameras looking both forward and aft, where each pod monitoring the same area but at different angles as do the pod(s) adjacent to it.
The SOS technology developed in this project was subjected to a technology qualification process. Technology qualification (TQ) is the process of providing evidence that the technology will function within specified limits with an acceptable level of confidence. Its purpose is to enable industry to cost effectively put technology into safe use.
The TQ process was conducted in accordance with DNV GL Recommended Practice DNV RP-A203, Qualification Procedures for New Technology. This recognized industry standard provides a systematic approach to the technology qualification process and is thus a tool to guide the development of the SOS technology so that it will function reliably within the specified limits.
The SOS project conducted land- and harbour-based testing (Phase 1 and Phase 2 testing, respectively), but the degree of testing was insufficient to conclude the Technology Assessment and qualify the SOS technology.
Results of the Phase 2 (harbour-based) testing were promising. The tests indicated that a sensor package including optical, thermal and infrared cameras, radars and laser scanners can detect objects going overboard a vessel under normal environmental conditions (daylight, darkness, rain). Furthermore, the detection and classification program demonstrated that for individual sensors, it can make correct classifications between objects fitting a human profile versus those that do not.
The data fusion feature of the SOS technology had not been sufficiently tested and analysed by project end to be considered in the technology qualification process, so real-time detection classification has not been proved (only near real-time detection). Any of the several types of classification algorithms would likely reduce the false alarm rate and improve the classification success, however it was not possible to conclude whether the system will meet the classification success criteria set in the Technology Qualification Basis (see SOS-TQBR-D6.1). This must be the subject of future Phase 2 and 3 testing.
Given the limited time frame for the SOS project under the FP7 grant agreement, it was not feasible to conduct Phase 3 testing prior to scheduled project end. Phase 3 involves mounting the equipment on a vessel and testing it at sea under a full spectrum of conditions. This includes ensuring that a sufficient number of tests under each test condition are conducted so that the results will be statistically significant. Successful completion of Phase 3 testing is essential in order to properly qualify the technology and fulfil the TQ process expectations contained in DNV-RP-A203 in general and the project TQP in particular.
In light of progress demonstrated at project end, DNV GL considers the SOS technology conceptually feasible as defined in DNV-DSS-401, Technology Qualification Management, and thereby suited for further development and qualification according to DNV-RP-A203. DNV GL has issued a Statement of Feasibility to this effect.
Limitation – The Statement of Feasibility by DNV GL is not a Product Certificate.
DNV GL has not certified, approved or accepted the Technology.
DNV GL recommends therefore that the principal consortium members (the SMEs) consider options for continuing the system testing and technology qualification after the formal end of the current project in May 2016.
Project Results:
The project´s results relate to 4 main categories:
1. Instantaneous detection and alerting
SOS integrates several advanced sensor technologies to instantaneously detect where and when a person is falling overboard. Its communication infrastructure and software solutions ensure that no unnecessary time is lost from when an incident occurs until the crew (and eventually 3rd parties) are alerted.
The type of alerts and the way they are presented is optimally adapted to the situational awareness on the bridge. In this way, the surveillance system has a maximum chance of successfully alarming the right crew members in time.
This capability for instantaneous detection (with extremely high detection probability and low false alarm rate) and instantaneous alerting are the main advantages of SOS.
2. Reliable situational awareness
When SOS is installed on a vessel, the shipmaster will be provided with an accurate 360-degrees overview of the surrounding environment. Due to the integration of different types of sensor technologies, the surveillance system is robust and reliable with minimum false alarms. SOS thus gives the shipmaster optimal situational awareness. SOS is particularly designed to detect falling objects close to the ship's hull.
3. All-weather system
SOS is based on several sensor technologies that have complementary characteristics. This not only means the quality of the detection system is greatly improved, but it also means that the system is less influenced by weather and environmental conditions. For example, some sensor technologies work better during the day with overcast weather conditions, while others perform better during darkness with no dependence on light. The combination and integration of several sensor systems enable a robust, all-weather reliable system.
4. A complete system solution
The SOS technology provides a complete system to detect instantaneously any person falling overboard from a ship or any other maritime platform. It complies with and will be certified for existing international maritime regulation.
The foreground created within the project has been formulated in patent claims substantiated with detailed technical descriptions:
1. A system for the detection and handling of a man-of-overboard situation comprising:
- a plurality of sensor systems configured to observe the environment surrounding the vessels and monitor the presence of objects (human, natural and manmade)
- characterized by having a central processing system that continuously compares the processes and sequence of a man-overboard situation with available sensor data and verifies the possible event with an intelligent agent to make a decisive evaluation of the man-overboard situation
2. The system of point 1, wherein the sequence of events comprise at least the stages early warning, jump event detection, fall detection, splash detection, and object in water detection.
3. The system of point 2, wherein the intelligent agent is comprised of a man-in the loop for high intelligence detection.
4. The system of point 2, wherein the intelligent agent is comprised of a computer system making similar operations as the man-in the loop for high intelligence detection
5. The system of any of the preceding claims, wherein the early warning detection is made considering screening of locations and/or individual persons based on risk profiles, external data input or other actionable information.
6. The system of any of the preceding claims, wherein the early warning detection is derived from CCTV imaging and video analytics data to judge if an individual has intent of performing an accidental, forced or suicidal event
7. The system of any of the preceding claims, wherein the jump event detection is performed using an optical scanner to detect objects falling through a plane.
8. The system of any of the preceding claims, wherein the jump event detection is performed using an RFID tag on persons and a RFID reader deployed to cover the outside of the ship side
9. The system of any of the preceding claims, wherein the jump event detection is performed using a radiating cable sensor deployed on the outside perimeter of the ship
10. The system of any of the preceding claims, wherein the fall detection is performed using a radar system that is optimized to detect falling objects
11. The system of any of the preceding claims, wherein the fall detection is performed using infrared camera to detect warm falling objects outside the ship
12 The system of any of the preceding claims, wherein the splash detection is performed using acoustic means or radar signatures
13 The system of any of the preceding claims, wherein the splash detection is performed using radar detection or video analytics
14. A method for detection a man-overboard situation, the method comprising:
-implementation and deployment of sensor systems according to claims 1-13
-implementation of a central processing system to continuously perform operations on sensor systems deployed according to claims 1-13
-implementation of an intelligent agent according to claims 2 or 3 to generate high probability of detection of man-overboard situations with controllable and acceptable false alarm rate events
15. A computer program comprising a program code for performing the method of claim 14 when executed on a computer.
Ownership of IPR foreground
Foreground comprising the total foreground being generated by the individual or joint partners from the project is owned by the 3 leading SMEs. In order to be able to prove ownership (as well as the date of generation) of foreground, it has been explicitly required that all participants maintained documents showing the development of the generation of knowledge and/or results, e.g. testing notebooks, in accordance with proper standards. This has helped avoid or resolve disputes between participants about the origin of certain results and any related IPR.
In addition, partners have been made to ensure (and have been requested to ensure) that, where necessary, they reach an agreement with their employees and other personnel if the latter are entitled to claim rights to foreground (including third parties such as subcontractors, students, etc.) in order for the participant to be able to meet its contractual obligations.
The three SMEs have arranged the ownership of Foreground in advance of project start. They have transferred all Foreground to a separate entity where the ownership structure, in the form of the total share capital, is distributed as follows: Offshore Monitoring Ltd 42.33%, HJELMSTAD AS 37.85%, Global Maritime Services Ltd 19.82%.
The transfer of all resources provided in the project to one entity is favourable for the further industrial and commercial routes. It is anticipated that the majority of the income from the first commercial sales will be directly allocated to further empower this entity with resources to ramp up production, distribution and installation efforts since in the end of the project the entity will be reliant upon the production capability of PRE and their distribution network. Even though their production facilities are state-of the- art and their network extensive, it will be considered to go into a joint venture with one of the large equipment providers in order to take advantage of their (more extensive) distribution network and service/ maintenance/support services. Moreover, it must be considered if production/manufacturing/assembly of the system can be performed more cost-effectively elsewhere. Another advantage with a separate entity with shared responsibilities among the three leading SMEs is that the entity is less vulnerable to any changes in each of the shareholding SMEs.
In respect to the latter, it implies that the owner of the IPR is the new entity. With respect to the terms of exercising the ownership, the joint ownership arrangement has clarified management issues such as the sharing of the costs arising from legal protection procedures (e.g. patent filing and examination fees, renewal fees, prior state of the art searches, infringement actions, etc.) and the exploitation of the jointly owned foreground (e.g. sharing of any revenues or profits). This joint ownership arrangement also takes into account the different national joint ownership regimes to avoid their potential pitfalls.
The three leading SMEs are free to transfer Foreground (all or in parts) to the new entity at the time of convenience of the SMEs without the need for prior notice nor approval by any other party (nor the European Commission/Research Executive Agency).
Protection of the foreground
Valuable foreground has been protected. Protection is mandatory in all cases.
Where foreground is capable of industrial or commercial application (even if it requires further research and development, and/or private investment), it have been protected in an adequate and effective manner in conformity with the relevant legal provisions, having due regard to the legitimate interests of the 3 leading SME participants, particularly their commercial interests. The 3 SMEs will collectively reflect on the best strategy to protect in view of the use of the foreground both in further research and in the development of related commercial products, processes or services.
The reference to industrial or commercial applicability and to the legitimate interest of the 3 leading SMEs means that Intellectual Property protection is mandatory in all cases.
Although the SMEs do not have to formally consult the other participants when deciding to protect (or not to protect for reasons specified above) a specific piece of foreground they own, the other participant has been informed, also after protective measures have been taken.
Main S&T activities
The main S&T activity was to design a sensor system for detecting and alerting man overboard incidents, implement the sensor system, install the system on a commercial platform, validate the system and its installation/integration, and demonstrate the main applications of the system.
The overall preparation for the demonstration was successful and unique for this project. All sensor units were installed on the platform where the demonstration took place. During the test phase, the MOB sensors were completely installed and fully operational, the MoB system had all sub-systems working and system performance was in line with the chosen capability for the demonstration event. The test cases included the use of test targets (a dummy as analogue for humans, and various non-human objects) dropped from the ship to demonstrate the detection capabilities of objects falling overboard.
A major part of the project has been the dissemination of results and the leveraging of project results for the benefit of the participating companies. Early in the project a project website was established to provide an information platform. This site was continuously maintained and improved during the project. The project also recorded and produced a video documentary of the demonstration event. This video can be found on http://www.overboardalert.com.
In the second reporting period, one of the major activities was securing Intellectual Property rights from the project. In the preparation for the formal patent application process, the project SMEs provided a screening of the project work to identify the unique aspects of the project and the technical solutions, along with a broad search of existing patents and publications. In order to secure the broadest possible IPR, a search was made for professional patent agents with relevant experience and good track record. After a competitive process the patent lawyers got selected and authorized for the patent filing process.
It was further considered whether other parties had explored the SOS concept and published relevant papers. These findings proved the usefulness of the overall project and application also for other applications and regions (not only EU). However it forced a rethink of the patenting strategy, resulting in a bigger than expected effort to scope and structure the patents to appropriately protect the project results, also in regions outside EU. This was a useful exercise and the result was the filing of patents inside and outside of EU countries.
The SOS project developed a new sensor technology with man overboard detection for maritime shipping and offshore energy applications. To facilitate the exploration of the already defined applications for ‘new-to-the project’ maritime companies and also to investigate alternative and further applications, a one-day workshop was arranged that took place in the end of May 2016. The workshop provided a good basis for constructive exchange of ideas and suggestions.
A final and significant part of the project is the production of a promotion video. The intention of the video was to capture the final demonstration for the EU Commission and to provide a promotional tool for the system and solution for customer discussion and business development.
List of used sensors in the project which constitute the SOS basic system:
Laser scanner
A Laser Scanner is involved in the SOS tests. This device is scanning the 180° half-plane in front of it. The data acquired are distance and angle to all detected objects.
The scanner relies on measuring the round-trip time of distinct laser pulses, from emitter to target and back, at evenly spaced angular resolution. To achieve this, the laser and sensor is mounted on a rotating head. The maximum rotational speed of the laser head is 100Hz, which provide sufficient confidence in catching a human sized falling object by means of illuminating it repeatedly with a number of laser dots. The laser half-plane thus represents a light curtain falling objects have to pass through.
Relying on a directed, non-dispersed laser, it is also able to overcome both darkness and direct sunlight ambient illumination. It is therefore able to operate in all weather conditions, including extreme temperatures.
Infrared cameras
The second kind of sensors are the long wave infrared cameras. These cameras observe heat differences in the image. LWIR cameras do not rely on active illumination, they observe the temperature of each pixel in the field of view. Three different long wave infrared cameras are involved. One ‘household brand’ type has been tested in lab, at sea and at a swimming pool. This sensor is giving good results and should be kept in the process as reference.
Two later additions were added to the pool of long wave infrared cameras, each with different manufacturer, wavelength, size and performance.
Two sensors provide analogue video output only, and require a digitizer while one is a dual mode IP camera that offers video over Ethernet in addition to plain analogue video out.
The latter two cameras are representative of each end of the LWIR market. One has a Vanadium Oxide sensor and is equipped with a traditional shutter, while the other an Amorphous Silicon Pixel Detector implemented as a shutter-less system.
Visual cameras
The third kind of optical sensor is the visual camera. These will mostly be utilized for verification purposes after the event, by providing the officer on bridge with a video playback of the event. However, this video stream might be subject to the same pattern recognition algorithm that are applied to the LWIR cameras, provided sufficient illumination is present.
Extreme low-light capable cameras are entering the marked at the time of writing. While interesting to the project, these arrive too late to be included in the demonstration setup. However, two traditional HD optical cameras were used in the tests. Information and power are sent through a single Ethernet cable.
IT equipment
The IT equipment is composed of a central switch, a high end laptop for fusion SW (SoftWare) and GUI (Graphical User Interface), and an AXIS digitizer for the infrared cameras (which provide analogue data streams).
The system is connected together with different types of cables and network equipment, e.g. Ethernet, power and coax.
By consulting experts within the consortium a system description was established containing a number of new innovations needed in order to both fulfil the majority of functional requirements and the anticipated production costs of the system. After a number of technical specification iterations, the potentially best components were ordered. In parallel, the main data processing software architecture was developed, in order to be ready when the sensor hardware when that got finalized.
Potential Impact:
The progress to the end of the project was impressive. The plan for component cost savings compared to DoW has been accomplished with a good margin. The total system production cost will therefore be as anticipated in the DoW (but having additional sensor components).
A first prototype was tested to gain the first experimental observations. Consequent modification resulted in a first mock-up. Following that, the first SOS prototype was assembled and installed for offshore testing against the initial functional requirements. Following these tests the project did a number of tests that showed the potential of the system for sought for functional achievements.
Based on these functional achievements the project established a specification for the future industrialisation of the system. A number of prototypes have been produced for further testing and modifications for a period before the industrialisation process will start. Existing end-users/clients of the project partners have already expressed their interest.
Any user/client in need of an automated man overboard detection system, sampling in space and time, is a wished for client. Thus the project target a larger socio-economic impact than what is foreseen in the project’s Description of Work.
An aggressive plan got adopted to preserve the intellectual property (IPR) generated as a result of the user /technical group interactions and focused design processes. A broad sweeping patent application got filed before the end of the project in addition to additions to similar and relevant existing patent pools of the SMEs.
Dissemination partners
The coordinator of the project has an extensive network in the shipping industry. Among the individuals of the project are shipping founders (e.g. Capt. Reidulf Maalen was among the founders of Crystal Cruises that got voted ‘The best cruise line’ 16-years in a row before he retired from his fleet manager position), shipping operators (e.g. Capt. Otto Nergaard has been captain for several shipping companies), class societies (that effectively and indirectly can market new technologies that increase safety and reduce operational costs), members in the International Maritime Organization (IMO) that can enforce new legislations, etc.
Further, we have collaborations with Protection & Indemnity (P&I) clubs for high-value shipping, and consortium members within the most recognized propeller clubs of the World (US and UK), etc.
All these humans are empowered with knowledge to disseminate and promote the system. All the access to vessels in these groups of people will more than saturate production facilities for the next few years (if product functionality and price is as expected).
All these parties have worked to reduce operational impact and improve safety.
Having these and other partners taking part of realistic demonstrations helps their willingness to work with the project partners.
Communication
Once the patents got secured, the communication of project findings and results have been persuaded through varied channels
• Demonstrations on vessels and port demonstration system to show real-life capabilities to interested parties
• Preparations for further prototype installations on vessels for ‘cost-price’ (i.e. systems offered to a bare minimum production cost) to allow end-users to test/demo the real benefits at low-cost (and also to have them provide operational feedback to the team)
• Formal Safety Approval proposition to IMO for assessing prototype system for safety improvements
• Traditional communication channels (functional videos, web-site, workshop at Nor-Shipping, Seatrade, publications, reports, etc.)
The end-users normally purchase technical equipment through agents at the ship yard that has deals with various vendors. Our communication strategy has been different; it is to allow decision makers of the largest shipping groups to evaluate the product’s functionality by real-life demonstrations and allow them to install a system at ‘cost-price’ as to have themselves to evaluate the true-value of the system.
Dissemination
Our research findings/product is a network multi-sensor system. It can be used to detect small (falling) objects from a ship or platform
Our primary end-users are the high-value shipping market. We involved end-users in our dissemination efforts by allowing them to receive ‘production-cost’ systems for further testing. We have used the partner (OSM, GMS) individuals, organisations and networks to help the dissemination process. The main ways of communication has been through video and workshops at large shipping events (Nor Shipping, Seatrade – see details below) to invite key decision makers of large shipping groups to take part in ‘real-life’ demonstrations showing the system’s real-life capabilities. Obstacles that we faced in the dissemination of the product were related to installation and production costs. We were mitigating those by including the class/certification society and external advisors, and the project is greatly cost reducing the system by assembling low-cost modularity components. We evaluated the dissemination plan by measuring the number of ‘Expression of Interests’ for system installations of prototypes by the project end. At the moment of demonstration all visiting captains were interested in the system. We encouraged feedback from end-users and dissemination partners by involving them in the evaluation and demonstration exercises, allowing more prototypes to be installed for further evaluation and demonstration exercises, and thereby receive continuous feedback from them as well as feeding back evolutions/upgrades to the system being demonstrated depending on the outcome of the demonstration responses.
The following lists describe various activities which were undertaken by the consortium partners during the period of the project:
Partner OSM performed the following dissemination activities:
- Presentation to two European national Government Officials representing coast guards, navy, patrolling vessels and related infrastructure to safeguard national waters
o To allow the operation of a vessel to become more safe, secure, cost efficient and environmental friendly AND to allow the coastal monitoring to become more efficient and cost effective
o The representatives’ conclusion: of interest for their coast guard, maritime authority, flag state, commercial vessels
- Two maritime training centres’ presentation; simulation exercises of MoB incidents related to available technology
- Armed forces of two European governmental officials; support for search and rescue activities, particularly related to search for (illegal) immigration and support for incidents at sea
- Attended the testing and demonstration activities on vessel Veøy in Flåm and Ålesund; 2- campaigns with extensive testing of the combined sensor system
Partner GMS performed the following dissemination activities:
Cruise Shipping Miami, March 2015
- The yearly subject event is without comparison the biggest venue targeting the cruise industry. This industry is the one that will most likely benefit from SOS.
- Over the course of two days, GMS had the opportunity to present the SOS to many cruise line executives, visit booths of all major suppliers of technology to the industry and to ultimately further network contacts for the project.
- GMS conclusion by the end of the visit, is that the industry is very interested in our product. Upon inquiry, none of the technology companies that GMS approached seemed to currently have any projects running that aimed at solving the Man Overboard issue in a similar fashion.
- Several invitations for future follow-on activities, involving testing and demonstrations were received
- Attended the testing and demonstration activities on vessel Veøy in Flåm and Ålesund; 2- campaigns with extensive testing of the combined sensor system
Partner PRE performed the following dissemination activities:
- 2015-06-16-18: Prevas attended the IFSEC exhibition at ExCel London where the project were discussed with several possible vendors of sensors. This helped discovering if other providers are working on the same or similar applications and if there would be other sensor components that the project possibly could consider as relevant for integration
- Attended the testing and demonstration activities on vessel Veøy in Flåm and Ålesund; 2- campaigns with extensive testing of the combined sensor system
Partner SNT performed the following dissemination activities:
- 2015-04-14: presented the project to a group of foreign journalists (South Korea, South Africa, Portugal) as part of the Media tour “Innovations in Safety and Security in The Netherlands” organized by Netherlands Enterprise Agency.
- 2015-04-17: presented the project to the whole company during an internal company wide meeting.
- Various testing and demonstration activities, where end-users were presented with features and applications of the system
o 2015-06-02: First outdoor testing on land
o 2015-06-02: Outdoor testing on sea shore
o 2015-07-30: Testing at an outdoor swimming pool
o 2016-04-18/19: Testing on demonstration vessel
o 2016-05-2/27: Final testing and demonstration on vessel
Partner DNV-GL performed the following dissemination activities:
- Various testing and demonstration activities, where end-users were presented with features and applications of the system
o 2015-06-02: First outdoor testing on land
o 2015-06-02: Outdoor testing on sea shore
o 2016-04-18/19: Testing on demonstration vessel
o 2016-05-2/27: Final testing and demonstration on vessel
- Various internal DNV-GL presentation to increase awareness of the project and its system internally (after IPR was protected by the SMEs).
List of Websites:
The address of the project public website, if applicable as well as relevant contact details.
www.overboardalert.com
Contact: info@overboardalert.com
A disturbing number of man overboard incidents occur every year in the maritime transport industry. Unfortunately, most of these incidents are fatal. The loss of many lives each year and associated costs rapidly call for a change. SOS ((Multi-) Sensor Offshore Safety System) helps to reduce the loss of lives at sea.
The project partners forming the international R&D consortium of this project has worked on the development of SOS, a new concept and potential solution for the man overboard problem. The new concept has addressed the main shortfalls of existing systems and made extensive use of new technologies to combine commercially available state-of-the-art sensors with bespoke software solutions. The result has become a modular and scalable product which is intended to be a reliable and cost-effective overboard alert system for the maritime transport industry.
In order to design the final product, the project has gone through a thorough design project which has considered the required operational functions, the characteristics of the situation where a person falls overboard and the environmental conditions present in typical and adverse situations. In addition the cost/performance parameters of available and emerging sensor and software technologies has been considered and candidate solutions were assessed for operational performance, ease and cost of installation and also considered against the business strategies of the participating partners.
Based on the design efforts, the project has concurrently gone through several stages of experimentation and verification of sensor components and software solutions by way of in house testing and field trials. This has led to some solutions having been discarded, while new components have been introduced.
The resulting configuration with distributed sensor pods and a central processing system has been manufactured in the form of a prototype, which has been thoroughly tested and was demonstrated in actual scenarios at the end of the project. Furthermore a design for a cost-reduced implementation has been developed and will form the basis for a further development/industrialisation of the system.
After a larger number of tests were executed the certification agency taking part in the consortium (DNV-GL) verified the actual performances and issued a technology certificate verifying the concept and the feasibility of the solution. We furthermore invite any end-user ship or platform operator to take part of further testing of the system.
Foreground IPR of the project has been protected by a solution patent and application patent.
A project demonstration video has been prepared. This video can be found on www.overboardalert.com
Project Context and Objectives:
SOS ((Multi-) Sensor Offshore Safety System) is intended to reduce the loss of lives at sea. It is a new automatic man overboard-alert system, which uses advanced integrated multi-sensor technology to detect any person falling overboard in real time and immediately alarm the crew in such an event.
The project context
Almost every day, someone is going overboard (MoB) from a vessel or an offshore installation. The majority of human loss at sea is typically from larger transport vessels. The cruise industry alone has on average one MoB incident a week, resulting in loss of life. Most cases remain unresolved.
Generally speaking, the main problem in most of the MoB incidents is that nobody notices you fall. And even if the person going overboard is observed by someone (which is rarely the case), alarming the crew is most often accomplished too late. It sometimes takes hours until the crew actually becomes aware of the incident — too late for a successful search and rescue operation and most often too late to locate the victim at all, whether alive or drowned.
The ever-increasing number of fatalities in MoB incidents is of grave concern for the entire maritime industry and action to resolve this is given high priority.
For example, the United State's Cruise Vessel Security and Safety Act (CVSSA) mandates that from 27th January 2012 cruise ships must have a man overboard monitoring system, which shall "integrate technology that can be used for capturing images of passengers or detecting passengers who have fallen overboard, to the extent such technology is available". Many other seafaring nations enforce this act as well. As a consequence, numerous maritime transport companies have recently assessed and validated the available technologies, but have been unable to find a system that is reliable and cost-effective. There is currently no available equipment on the market to meet the requirements of CVSSA.
Timing is critical. Only when a man overboard incident is reported to the crew immediately, will it be possible to locate the drowning person in time for recovery. Therefore, every second counts in alarming and starting the necessary search and rescue operations. Early detection increases the chance of a successful recovery.
Apart from the loss of lives at sea, each incident triggers a complex and costly search and rescue operation. Such an operation typically involves turning around the vessel, launching smaller rescue boats for searching and calling for assistance from shore-based units or nearby ships. Direct costs are significant and typically exceed € 1,000,000, depending on the scope of the operation. In addition, shipping operators face additional costs and challenges, such as legal procedures and loss of reputation.
The longer the time span between the time of the incident and the time at which the crew is alarmed, the more complex and costly the search and rescue operation will become. And above all, with rapidly diminishing chances of success.
The project objectives
The majority of MoB incidents go unnoticed. The project objective was to create a system which aims to detect any person going overboard and to alert the Bridge/Control Room instantaneously.
Due to its instantaneous alerting capabilities, SOS reaches two other objectives. It contributes to reducing both the loss of lives and high expenses in connection with search and rescue operations.
The projects ultimate goal was to demonstrate SOS as the first reliable, cost-effective and real-time man overboard detection and alerting system.
There were many technical aspects that needed to be covered in developing a complete and reliable sensor system like this. The concept needed to be realized and RTD (Research and Technological Development) work needed to be executed. The system then needed to be thoroughly tested, calibrated/tuned, and validated. This effort had to be carried out by complementary organisations in order to exploit the full potential of the base technology.
Following the SOS project kick off, a number of end-user scenarios were developed with help from end-users. These scenarios reflect the most likely user cases of the system, in prioritized order. These scenarios were thereafter converted to a detailed list of functional requirements, which finally resulted in the chosen system architecture. A significant part of this process was the understanding of the relative merit and the actual technical feasibility of different possible adjunct sensors supporting the overall SOS system. This was accomplished by addressing the various user scenarios and functional requirements in particular the more demanding requirements such as all-weather/all-day operation. This led to the multi-sensor approach.
The overall performance metrics of the system have been defined to give an accurate measure of the system´s operational capabilities with respect to actual man-overboard scenarios. The main performance metrics for the system are: detection probability (correct positives), false alarm rates (false positives) and overall system response time. Fundamentally these performance metrics are difficult to adequately measure due to the following facts:
- Detection probabilities are required to reach very high values. In order to give accurate values for detection probability a very large number of tests need to be performed.
- Compounding to this issue, the actual test situations depend on numerous environmental factors as well as test scenarios, making the required test volume for high significance evaluation very high.
A further issue is that the test situation does not fully verify the issue of false negatives (missing detections). In an operating environment false negatives can occur due to numerous factors, also for those not anticipated in the test plan. (A number human ‘look-alikes’ were deployed during the testing).
Verification of the requirement of extremely low false alarm rates (false positives), in the order of only few instances per e.g. 1 year, would necessitate prohibitively long test periods.
The unique feature of the SOS system is the multi-domain approach using disperse sensors and sequential event processing. For the test system a basic set of sensors with a limited selection of operations have been implemented, in a final system a more comprehensive approach may be used that will gain improved metrics relative to the one under testing.
As a solution to these aspects, the project evaluated system metrics using a modelling tool working in parallel to the test procedures to evaluate the final performance based on single sensor and single event testing. Extrapolation from the single sensor to compound system performance is based on multivariate estimation and individual sensor performance extrapolation that is furthermore based on signal to noise values and distribution characteristics. Additionally, estimation of the sequential detection performance and overall time lag and latency is done using conditional estimation procedures.
A large number of meetings with representatives from SMEs and the R&D partners have been held during the project period with the objective to define the system that best suits operational requirements within the constraints of the project, to identify the best locations for installing the various components of the system, define all mechanical and electrical interfaces, cabling requirements and a time schedule. Cost reduction exercises have been conducted to optimize cabling and interfaces. An acceptable solution that presents the best compromise between cost and development time has been found.
There has been some discussion of the number of sensors and their positions during the project. Due to the relatively high cost for a prototype installation, the number of sensors had to be reduced to a number that was within the budget of the project, but at the same time sufficient to illustrate the concept of the system. Ideally a 360-degrees coverage of the ship is desired, but a sector of the vessel was selected for concept demonstration. There’s also some issues to post install such a system in a way that does not affect the normal operation of the ship and protect the equipment from parts of the ship where use of cranes, passengers, etc. may appear and possible can damage a post installed system. Also there are some issues to minimize the blind zones of the system. This issue has been solved by installing two or more camera pods, each containing cameras looking both forward and aft, where each pod monitoring the same area but at different angles as do the pod(s) adjacent to it.
The SOS technology developed in this project was subjected to a technology qualification process. Technology qualification (TQ) is the process of providing evidence that the technology will function within specified limits with an acceptable level of confidence. Its purpose is to enable industry to cost effectively put technology into safe use.
The TQ process was conducted in accordance with DNV GL Recommended Practice DNV RP-A203, Qualification Procedures for New Technology. This recognized industry standard provides a systematic approach to the technology qualification process and is thus a tool to guide the development of the SOS technology so that it will function reliably within the specified limits.
The SOS project conducted land- and harbour-based testing (Phase 1 and Phase 2 testing, respectively), but the degree of testing was insufficient to conclude the Technology Assessment and qualify the SOS technology.
Results of the Phase 2 (harbour-based) testing were promising. The tests indicated that a sensor package including optical, thermal and infrared cameras, radars and laser scanners can detect objects going overboard a vessel under normal environmental conditions (daylight, darkness, rain). Furthermore, the detection and classification program demonstrated that for individual sensors, it can make correct classifications between objects fitting a human profile versus those that do not.
The data fusion feature of the SOS technology had not been sufficiently tested and analysed by project end to be considered in the technology qualification process, so real-time detection classification has not been proved (only near real-time detection). Any of the several types of classification algorithms would likely reduce the false alarm rate and improve the classification success, however it was not possible to conclude whether the system will meet the classification success criteria set in the Technology Qualification Basis (see SOS-TQBR-D6.1). This must be the subject of future Phase 2 and 3 testing.
Given the limited time frame for the SOS project under the FP7 grant agreement, it was not feasible to conduct Phase 3 testing prior to scheduled project end. Phase 3 involves mounting the equipment on a vessel and testing it at sea under a full spectrum of conditions. This includes ensuring that a sufficient number of tests under each test condition are conducted so that the results will be statistically significant. Successful completion of Phase 3 testing is essential in order to properly qualify the technology and fulfil the TQ process expectations contained in DNV-RP-A203 in general and the project TQP in particular.
In light of progress demonstrated at project end, DNV GL considers the SOS technology conceptually feasible as defined in DNV-DSS-401, Technology Qualification Management, and thereby suited for further development and qualification according to DNV-RP-A203. DNV GL has issued a Statement of Feasibility to this effect.
Limitation – The Statement of Feasibility by DNV GL is not a Product Certificate.
DNV GL has not certified, approved or accepted the Technology.
DNV GL recommends therefore that the principal consortium members (the SMEs) consider options for continuing the system testing and technology qualification after the formal end of the current project in May 2016.
Project Results:
The project´s results relate to 4 main categories:
1. Instantaneous detection and alerting
SOS integrates several advanced sensor technologies to instantaneously detect where and when a person is falling overboard. Its communication infrastructure and software solutions ensure that no unnecessary time is lost from when an incident occurs until the crew (and eventually 3rd parties) are alerted.
The type of alerts and the way they are presented is optimally adapted to the situational awareness on the bridge. In this way, the surveillance system has a maximum chance of successfully alarming the right crew members in time.
This capability for instantaneous detection (with extremely high detection probability and low false alarm rate) and instantaneous alerting are the main advantages of SOS.
2. Reliable situational awareness
When SOS is installed on a vessel, the shipmaster will be provided with an accurate 360-degrees overview of the surrounding environment. Due to the integration of different types of sensor technologies, the surveillance system is robust and reliable with minimum false alarms. SOS thus gives the shipmaster optimal situational awareness. SOS is particularly designed to detect falling objects close to the ship's hull.
3. All-weather system
SOS is based on several sensor technologies that have complementary characteristics. This not only means the quality of the detection system is greatly improved, but it also means that the system is less influenced by weather and environmental conditions. For example, some sensor technologies work better during the day with overcast weather conditions, while others perform better during darkness with no dependence on light. The combination and integration of several sensor systems enable a robust, all-weather reliable system.
4. A complete system solution
The SOS technology provides a complete system to detect instantaneously any person falling overboard from a ship or any other maritime platform. It complies with and will be certified for existing international maritime regulation.
The foreground created within the project has been formulated in patent claims substantiated with detailed technical descriptions:
1. A system for the detection and handling of a man-of-overboard situation comprising:
- a plurality of sensor systems configured to observe the environment surrounding the vessels and monitor the presence of objects (human, natural and manmade)
- characterized by having a central processing system that continuously compares the processes and sequence of a man-overboard situation with available sensor data and verifies the possible event with an intelligent agent to make a decisive evaluation of the man-overboard situation
2. The system of point 1, wherein the sequence of events comprise at least the stages early warning, jump event detection, fall detection, splash detection, and object in water detection.
3. The system of point 2, wherein the intelligent agent is comprised of a man-in the loop for high intelligence detection.
4. The system of point 2, wherein the intelligent agent is comprised of a computer system making similar operations as the man-in the loop for high intelligence detection
5. The system of any of the preceding claims, wherein the early warning detection is made considering screening of locations and/or individual persons based on risk profiles, external data input or other actionable information.
6. The system of any of the preceding claims, wherein the early warning detection is derived from CCTV imaging and video analytics data to judge if an individual has intent of performing an accidental, forced or suicidal event
7. The system of any of the preceding claims, wherein the jump event detection is performed using an optical scanner to detect objects falling through a plane.
8. The system of any of the preceding claims, wherein the jump event detection is performed using an RFID tag on persons and a RFID reader deployed to cover the outside of the ship side
9. The system of any of the preceding claims, wherein the jump event detection is performed using a radiating cable sensor deployed on the outside perimeter of the ship
10. The system of any of the preceding claims, wherein the fall detection is performed using a radar system that is optimized to detect falling objects
11. The system of any of the preceding claims, wherein the fall detection is performed using infrared camera to detect warm falling objects outside the ship
12 The system of any of the preceding claims, wherein the splash detection is performed using acoustic means or radar signatures
13 The system of any of the preceding claims, wherein the splash detection is performed using radar detection or video analytics
14. A method for detection a man-overboard situation, the method comprising:
-implementation and deployment of sensor systems according to claims 1-13
-implementation of a central processing system to continuously perform operations on sensor systems deployed according to claims 1-13
-implementation of an intelligent agent according to claims 2 or 3 to generate high probability of detection of man-overboard situations with controllable and acceptable false alarm rate events
15. A computer program comprising a program code for performing the method of claim 14 when executed on a computer.
Ownership of IPR foreground
Foreground comprising the total foreground being generated by the individual or joint partners from the project is owned by the 3 leading SMEs. In order to be able to prove ownership (as well as the date of generation) of foreground, it has been explicitly required that all participants maintained documents showing the development of the generation of knowledge and/or results, e.g. testing notebooks, in accordance with proper standards. This has helped avoid or resolve disputes between participants about the origin of certain results and any related IPR.
In addition, partners have been made to ensure (and have been requested to ensure) that, where necessary, they reach an agreement with their employees and other personnel if the latter are entitled to claim rights to foreground (including third parties such as subcontractors, students, etc.) in order for the participant to be able to meet its contractual obligations.
The three SMEs have arranged the ownership of Foreground in advance of project start. They have transferred all Foreground to a separate entity where the ownership structure, in the form of the total share capital, is distributed as follows: Offshore Monitoring Ltd 42.33%, HJELMSTAD AS 37.85%, Global Maritime Services Ltd 19.82%.
The transfer of all resources provided in the project to one entity is favourable for the further industrial and commercial routes. It is anticipated that the majority of the income from the first commercial sales will be directly allocated to further empower this entity with resources to ramp up production, distribution and installation efforts since in the end of the project the entity will be reliant upon the production capability of PRE and their distribution network. Even though their production facilities are state-of the- art and their network extensive, it will be considered to go into a joint venture with one of the large equipment providers in order to take advantage of their (more extensive) distribution network and service/ maintenance/support services. Moreover, it must be considered if production/manufacturing/assembly of the system can be performed more cost-effectively elsewhere. Another advantage with a separate entity with shared responsibilities among the three leading SMEs is that the entity is less vulnerable to any changes in each of the shareholding SMEs.
In respect to the latter, it implies that the owner of the IPR is the new entity. With respect to the terms of exercising the ownership, the joint ownership arrangement has clarified management issues such as the sharing of the costs arising from legal protection procedures (e.g. patent filing and examination fees, renewal fees, prior state of the art searches, infringement actions, etc.) and the exploitation of the jointly owned foreground (e.g. sharing of any revenues or profits). This joint ownership arrangement also takes into account the different national joint ownership regimes to avoid their potential pitfalls.
The three leading SMEs are free to transfer Foreground (all or in parts) to the new entity at the time of convenience of the SMEs without the need for prior notice nor approval by any other party (nor the European Commission/Research Executive Agency).
Protection of the foreground
Valuable foreground has been protected. Protection is mandatory in all cases.
Where foreground is capable of industrial or commercial application (even if it requires further research and development, and/or private investment), it have been protected in an adequate and effective manner in conformity with the relevant legal provisions, having due regard to the legitimate interests of the 3 leading SME participants, particularly their commercial interests. The 3 SMEs will collectively reflect on the best strategy to protect in view of the use of the foreground both in further research and in the development of related commercial products, processes or services.
The reference to industrial or commercial applicability and to the legitimate interest of the 3 leading SMEs means that Intellectual Property protection is mandatory in all cases.
Although the SMEs do not have to formally consult the other participants when deciding to protect (or not to protect for reasons specified above) a specific piece of foreground they own, the other participant has been informed, also after protective measures have been taken.
Main S&T activities
The main S&T activity was to design a sensor system for detecting and alerting man overboard incidents, implement the sensor system, install the system on a commercial platform, validate the system and its installation/integration, and demonstrate the main applications of the system.
The overall preparation for the demonstration was successful and unique for this project. All sensor units were installed on the platform where the demonstration took place. During the test phase, the MOB sensors were completely installed and fully operational, the MoB system had all sub-systems working and system performance was in line with the chosen capability for the demonstration event. The test cases included the use of test targets (a dummy as analogue for humans, and various non-human objects) dropped from the ship to demonstrate the detection capabilities of objects falling overboard.
A major part of the project has been the dissemination of results and the leveraging of project results for the benefit of the participating companies. Early in the project a project website was established to provide an information platform. This site was continuously maintained and improved during the project. The project also recorded and produced a video documentary of the demonstration event. This video can be found on http://www.overboardalert.com.
In the second reporting period, one of the major activities was securing Intellectual Property rights from the project. In the preparation for the formal patent application process, the project SMEs provided a screening of the project work to identify the unique aspects of the project and the technical solutions, along with a broad search of existing patents and publications. In order to secure the broadest possible IPR, a search was made for professional patent agents with relevant experience and good track record. After a competitive process the patent lawyers got selected and authorized for the patent filing process.
It was further considered whether other parties had explored the SOS concept and published relevant papers. These findings proved the usefulness of the overall project and application also for other applications and regions (not only EU). However it forced a rethink of the patenting strategy, resulting in a bigger than expected effort to scope and structure the patents to appropriately protect the project results, also in regions outside EU. This was a useful exercise and the result was the filing of patents inside and outside of EU countries.
The SOS project developed a new sensor technology with man overboard detection for maritime shipping and offshore energy applications. To facilitate the exploration of the already defined applications for ‘new-to-the project’ maritime companies and also to investigate alternative and further applications, a one-day workshop was arranged that took place in the end of May 2016. The workshop provided a good basis for constructive exchange of ideas and suggestions.
A final and significant part of the project is the production of a promotion video. The intention of the video was to capture the final demonstration for the EU Commission and to provide a promotional tool for the system and solution for customer discussion and business development.
List of used sensors in the project which constitute the SOS basic system:
Laser scanner
A Laser Scanner is involved in the SOS tests. This device is scanning the 180° half-plane in front of it. The data acquired are distance and angle to all detected objects.
The scanner relies on measuring the round-trip time of distinct laser pulses, from emitter to target and back, at evenly spaced angular resolution. To achieve this, the laser and sensor is mounted on a rotating head. The maximum rotational speed of the laser head is 100Hz, which provide sufficient confidence in catching a human sized falling object by means of illuminating it repeatedly with a number of laser dots. The laser half-plane thus represents a light curtain falling objects have to pass through.
Relying on a directed, non-dispersed laser, it is also able to overcome both darkness and direct sunlight ambient illumination. It is therefore able to operate in all weather conditions, including extreme temperatures.
Infrared cameras
The second kind of sensors are the long wave infrared cameras. These cameras observe heat differences in the image. LWIR cameras do not rely on active illumination, they observe the temperature of each pixel in the field of view. Three different long wave infrared cameras are involved. One ‘household brand’ type has been tested in lab, at sea and at a swimming pool. This sensor is giving good results and should be kept in the process as reference.
Two later additions were added to the pool of long wave infrared cameras, each with different manufacturer, wavelength, size and performance.
Two sensors provide analogue video output only, and require a digitizer while one is a dual mode IP camera that offers video over Ethernet in addition to plain analogue video out.
The latter two cameras are representative of each end of the LWIR market. One has a Vanadium Oxide sensor and is equipped with a traditional shutter, while the other an Amorphous Silicon Pixel Detector implemented as a shutter-less system.
Visual cameras
The third kind of optical sensor is the visual camera. These will mostly be utilized for verification purposes after the event, by providing the officer on bridge with a video playback of the event. However, this video stream might be subject to the same pattern recognition algorithm that are applied to the LWIR cameras, provided sufficient illumination is present.
Extreme low-light capable cameras are entering the marked at the time of writing. While interesting to the project, these arrive too late to be included in the demonstration setup. However, two traditional HD optical cameras were used in the tests. Information and power are sent through a single Ethernet cable.
IT equipment
The IT equipment is composed of a central switch, a high end laptop for fusion SW (SoftWare) and GUI (Graphical User Interface), and an AXIS digitizer for the infrared cameras (which provide analogue data streams).
The system is connected together with different types of cables and network equipment, e.g. Ethernet, power and coax.
By consulting experts within the consortium a system description was established containing a number of new innovations needed in order to both fulfil the majority of functional requirements and the anticipated production costs of the system. After a number of technical specification iterations, the potentially best components were ordered. In parallel, the main data processing software architecture was developed, in order to be ready when the sensor hardware when that got finalized.
Potential Impact:
The progress to the end of the project was impressive. The plan for component cost savings compared to DoW has been accomplished with a good margin. The total system production cost will therefore be as anticipated in the DoW (but having additional sensor components).
A first prototype was tested to gain the first experimental observations. Consequent modification resulted in a first mock-up. Following that, the first SOS prototype was assembled and installed for offshore testing against the initial functional requirements. Following these tests the project did a number of tests that showed the potential of the system for sought for functional achievements.
Based on these functional achievements the project established a specification for the future industrialisation of the system. A number of prototypes have been produced for further testing and modifications for a period before the industrialisation process will start. Existing end-users/clients of the project partners have already expressed their interest.
Any user/client in need of an automated man overboard detection system, sampling in space and time, is a wished for client. Thus the project target a larger socio-economic impact than what is foreseen in the project’s Description of Work.
An aggressive plan got adopted to preserve the intellectual property (IPR) generated as a result of the user /technical group interactions and focused design processes. A broad sweeping patent application got filed before the end of the project in addition to additions to similar and relevant existing patent pools of the SMEs.
Dissemination partners
The coordinator of the project has an extensive network in the shipping industry. Among the individuals of the project are shipping founders (e.g. Capt. Reidulf Maalen was among the founders of Crystal Cruises that got voted ‘The best cruise line’ 16-years in a row before he retired from his fleet manager position), shipping operators (e.g. Capt. Otto Nergaard has been captain for several shipping companies), class societies (that effectively and indirectly can market new technologies that increase safety and reduce operational costs), members in the International Maritime Organization (IMO) that can enforce new legislations, etc.
Further, we have collaborations with Protection & Indemnity (P&I) clubs for high-value shipping, and consortium members within the most recognized propeller clubs of the World (US and UK), etc.
All these humans are empowered with knowledge to disseminate and promote the system. All the access to vessels in these groups of people will more than saturate production facilities for the next few years (if product functionality and price is as expected).
All these parties have worked to reduce operational impact and improve safety.
Having these and other partners taking part of realistic demonstrations helps their willingness to work with the project partners.
Communication
Once the patents got secured, the communication of project findings and results have been persuaded through varied channels
• Demonstrations on vessels and port demonstration system to show real-life capabilities to interested parties
• Preparations for further prototype installations on vessels for ‘cost-price’ (i.e. systems offered to a bare minimum production cost) to allow end-users to test/demo the real benefits at low-cost (and also to have them provide operational feedback to the team)
• Formal Safety Approval proposition to IMO for assessing prototype system for safety improvements
• Traditional communication channels (functional videos, web-site, workshop at Nor-Shipping, Seatrade, publications, reports, etc.)
The end-users normally purchase technical equipment through agents at the ship yard that has deals with various vendors. Our communication strategy has been different; it is to allow decision makers of the largest shipping groups to evaluate the product’s functionality by real-life demonstrations and allow them to install a system at ‘cost-price’ as to have themselves to evaluate the true-value of the system.
Dissemination
Our research findings/product is a network multi-sensor system. It can be used to detect small (falling) objects from a ship or platform
Our primary end-users are the high-value shipping market. We involved end-users in our dissemination efforts by allowing them to receive ‘production-cost’ systems for further testing. We have used the partner (OSM, GMS) individuals, organisations and networks to help the dissemination process. The main ways of communication has been through video and workshops at large shipping events (Nor Shipping, Seatrade – see details below) to invite key decision makers of large shipping groups to take part in ‘real-life’ demonstrations showing the system’s real-life capabilities. Obstacles that we faced in the dissemination of the product were related to installation and production costs. We were mitigating those by including the class/certification society and external advisors, and the project is greatly cost reducing the system by assembling low-cost modularity components. We evaluated the dissemination plan by measuring the number of ‘Expression of Interests’ for system installations of prototypes by the project end. At the moment of demonstration all visiting captains were interested in the system. We encouraged feedback from end-users and dissemination partners by involving them in the evaluation and demonstration exercises, allowing more prototypes to be installed for further evaluation and demonstration exercises, and thereby receive continuous feedback from them as well as feeding back evolutions/upgrades to the system being demonstrated depending on the outcome of the demonstration responses.
The following lists describe various activities which were undertaken by the consortium partners during the period of the project:
Partner OSM performed the following dissemination activities:
- Presentation to two European national Government Officials representing coast guards, navy, patrolling vessels and related infrastructure to safeguard national waters
o To allow the operation of a vessel to become more safe, secure, cost efficient and environmental friendly AND to allow the coastal monitoring to become more efficient and cost effective
o The representatives’ conclusion: of interest for their coast guard, maritime authority, flag state, commercial vessels
- Two maritime training centres’ presentation; simulation exercises of MoB incidents related to available technology
- Armed forces of two European governmental officials; support for search and rescue activities, particularly related to search for (illegal) immigration and support for incidents at sea
- Attended the testing and demonstration activities on vessel Veøy in Flåm and Ålesund; 2- campaigns with extensive testing of the combined sensor system
Partner GMS performed the following dissemination activities:
Cruise Shipping Miami, March 2015
- The yearly subject event is without comparison the biggest venue targeting the cruise industry. This industry is the one that will most likely benefit from SOS.
- Over the course of two days, GMS had the opportunity to present the SOS to many cruise line executives, visit booths of all major suppliers of technology to the industry and to ultimately further network contacts for the project.
- GMS conclusion by the end of the visit, is that the industry is very interested in our product. Upon inquiry, none of the technology companies that GMS approached seemed to currently have any projects running that aimed at solving the Man Overboard issue in a similar fashion.
- Several invitations for future follow-on activities, involving testing and demonstrations were received
- Attended the testing and demonstration activities on vessel Veøy in Flåm and Ålesund; 2- campaigns with extensive testing of the combined sensor system
Partner PRE performed the following dissemination activities:
- 2015-06-16-18: Prevas attended the IFSEC exhibition at ExCel London where the project were discussed with several possible vendors of sensors. This helped discovering if other providers are working on the same or similar applications and if there would be other sensor components that the project possibly could consider as relevant for integration
- Attended the testing and demonstration activities on vessel Veøy in Flåm and Ålesund; 2- campaigns with extensive testing of the combined sensor system
Partner SNT performed the following dissemination activities:
- 2015-04-14: presented the project to a group of foreign journalists (South Korea, South Africa, Portugal) as part of the Media tour “Innovations in Safety and Security in The Netherlands” organized by Netherlands Enterprise Agency.
- 2015-04-17: presented the project to the whole company during an internal company wide meeting.
- Various testing and demonstration activities, where end-users were presented with features and applications of the system
o 2015-06-02: First outdoor testing on land
o 2015-06-02: Outdoor testing on sea shore
o 2015-07-30: Testing at an outdoor swimming pool
o 2016-04-18/19: Testing on demonstration vessel
o 2016-05-2/27: Final testing and demonstration on vessel
Partner DNV-GL performed the following dissemination activities:
- Various testing and demonstration activities, where end-users were presented with features and applications of the system
o 2015-06-02: First outdoor testing on land
o 2015-06-02: Outdoor testing on sea shore
o 2016-04-18/19: Testing on demonstration vessel
o 2016-05-2/27: Final testing and demonstration on vessel
- Various internal DNV-GL presentation to increase awareness of the project and its system internally (after IPR was protected by the SMEs).
List of Websites:
The address of the project public website, if applicable as well as relevant contact details.
www.overboardalert.com
Contact: info@overboardalert.com
