Forschungs- & Entwicklungsinformationsdienst der Gemeinschaft - CORDIS


CYCLADES Berichtzusammenfassung

Project ID: 313972
Gefördert unter: FP7-TRANSPORT
Land: Germany

Final Report Summary - CYCLADES (Crew-centered Design and Operations of ships and ship systems)

Executive Summary:
Accident investigations show that the majority (up to 80%) of ship accidents are caused by human error. The causes are manifold, including aspects such as difficult working environment, crew qualifications, as well as lack of human centered design of equipment, of working spaces and/or of human machine interfaces.

The research project Cyclades (“Crew-centered Design and Operations of Ships and Ship Systems”) started in October 2012 and was finalized in September 2015. It aimed to increase the safety of shipping by improving the current lack of implementation of human-centered design (HCD) principles in workspace and equipment design. A multi-disciplinary team of operators, representatives from research institutes and classification societies focusses on all the key steps in the lifecycle of ship design. The project investigated where the barriers to human element integration occur, and how to best locate, produce, disseminate, and apply human element knowledge within the context of shipping.

In the first phase of the project causes of maritime accidents and incidents were investigated. An accident database was established and populated with data available from accident reports, incident reports, interviews and a questionnaire survey. The database allowed to analyse the events regarding their type, their location, the operators involved and the user task and equipment used. The evaluation showed that 67 % of the investigated incidents involved the human-machine interface. Further on the TRACER taxonomy was adapted and was applied to break down each accident into a series of critical task errors.
Fatigue and work overload are other important causes for ship accidents; therefore eye-tracking and movement sensors have been employed and enhanced to measure the current stress-level of the bridge personnel. Experiments performed in a bridge simulator showed a correlation between mental workload and movement activity of the test persons.

A lot of human factors and ergonomics recommendations exist (e.g. in class rules) and ought to be applied in ship design process. These were compiled and analysed regarding their applicability in the second phase of the project. Another aim of this phase was to discuss how usability can be evaluated by regulators and how usability requirements can be provided that enables the equipment and system designer to develop products with better usability and to allow the regulator to measure usability. As a result it can be stated that the majority of available guidance do not explicitly address usability in the sense of ensuring effectiveness, efficiency and user satisfaction. Many requirements are formulated in a more general unspecific way. In CyClaDes a usability evaluation approach has been developed that allows a quantification of a product’s usability.

The focus of the third project phase was to support maritime stakeholders with applicable HCD material. In this phase an E-learning platform and a web-based toolbox (“CyClaDes Framework”) have been developed. Furthermore a new edition of the Book “Improving ship operational design” has been published. These resources provide a comprehensive source of information for all stakeholders in the maritime domain, from the ship and system designers to operators and the crew on-board. Some of its information has been generated in the project while others has been collected from a variety of sources such as IMO, IACS, and ISO. Framework and E-learning platform are publicly available for all stakeholder groups and it is planned to further extend their content. By providing these tools, CyClaDes contributes to the spreading of HCD knowledge and awareness in the maritime industry and will thus create more human-oriented work environments and help to reduce accidents.

Project Context and Objectives:
The CyClaDes project was designed to promote the increased impact of the human element in shipping across the design and operational lifecycle. The project brought together a multi‐disciplinary team to focus on all the key steps in the lifecycle; the stakeholders; where the barriers to human element integration occur; and how to best locate, produce, disseminate, and apply human element knowledge within the overall context of shipping. The advantage was realized by supporting the integration of the human element in the design and operational life‐cycle from appreciation, to concept, to design, to application, to evaluation and approval, to maintenance. The outcome directly addresses pressing needs identified in the shipping industry.

Shipping is the foundation of the current economy. Roughly 90% of the world’s trade is transported by ships. Efficiency and effectiveness of sea transportation has a large impact on world economy and not least on the global environment. The latter is particularly relevant in European waters, where a large number of main shipping routes cross Particularly Sensitive Sea Areas (PSSA): Western European PSSA, Baltic Sea PSSA, Wadden Sea PSSA, Canary Islands PSSA, other sensitive areas, such as the Adriatic sea and Marmara sea, as well as highly frequented sea areas such as the English Channel, the Kiel canal, the strait of Gibraltar, the Dardanelles and the Bosporus. The focus on economy and efficiency can directly compete with, if not outright override, the consideration of the human element. Due to the global competition in the transport sector, processes in ship operations become increasingly streamlined and much more sensitive to disruption in an environment laden with potential disruptions.

As the European Commission describes, “The sea can be a treacherous environment, and despite increasingly stringent safety legislation, many lives are still lost at sea. In 1987, almost 200 people perished when the Herald of Free Enterprise capsized in shallow water shortly after leaving the Belgian port of Zeebrugge. Just a few years later, in 1994, over 850 people were killed when the Estonia went down in heavy seas off the coast of Finland.

The sinking of a ship can also wreak havoc in the environment and the economy; when the oil tanker Erika sank in 1999, 20 000 tonnes of oil were released into the Bay of Biscay, much of which ended up on France’s Breton beaches. As well as killing marine life on a massive scale, the incident was disastrous for the local fishing and tourism industries.” After the Erika disaster the EU set up the European Maritime Safety Agency (EMSA) which aims at reducing the risk of maritime accidents, marine pollution from ships and the loss of human lives at sea. Additional legislative actions try to reduce the risk of ships carrying hazardous cargo. However, large accidents still happen and show the need for further measures to not only address the consequences of accidents but to avoid such events in the first place. Detailed investigations of accidents, such as those performed in the recently completed Formal Safety Assessment on general cargo ships (MSC88/INF.8), indicate that human error is considered a root cause in 55% of all collision accidents. The high‐profile accidents are easy to remember and cry out for attention. But the everyday accidents, incidents, design and operational challenges are just as critical to the overall shipping system’s efficiency, safety culture, and crew quality of life, and therefore cannot be ignored. A human element strategy for shipping therefore needs to take a wide view of the shipping system. This concept is in line with recent IMO initiatives to investigate the human element. The initiatives take root in the IMO work to develop a strategy (MSC-MEPC.7/Circ.4), checklist (MSCMEPC.7/Circ.1), and framework (MSC-MEPC.7/Circ.3) for addressing, and considering the human element and work environment within the whole stakeholder community. This includes evaluating barriers to implementing the guidance and regulations that exist, producing and making better use of existing data (accident and risk), guidance for assessment of the human element (and within e-Navigation HEAP), and identifying and integrating human element research in known problem areas. The IMO list of high priority areas (MSC-MEPC.7/Circ.3) includes;

1. manual valve operation, access, location and orientation;
2. stairs, vertical ladders, ramps, walkways and work platforms;
3. inspection and maintenance considerations;
4. working environment; and
5. the application of ergonomics to design

The concepts for human element integration are there but the challenge that remains is to develop, apply, and evaluate these concepts in a way that produces tangible results for multiple key stakeholders involved in the design and operation of a variety of shipboard areas and processes. The CyClaDes project plan accepts this challenge by introducing a user‐centered perspective for key stakeholders (i.e., designers, operators, authorities, end-users), through a framework that captures, “translates”, and disseminates usable tools, methods, and information to provide maximum support for the human element across all stages of design and operation. The outcome of the project will help to increase the safety for ship, crew, cargo and consequently the environment by:

1. Increasing researchers’ understanding of stakeholders, including when human element input can best be applied, in what format, and what the barriers to integration are. This understanding will help to combat the familiar scenario where good guidelines, requirements, and best practices are not read, understood, implemented, or only given lip-service.
2. Assembling existing applicable knowledge (i.e., guidelines, tools and methodologies) from maritime and other domains into an easy to use format for the end user (i.e., Designer, operators, authorities, shipboard personnel). This assimilation and “translation” process will seed the framework with initial content and best practices.
3. Developing and applying selected methodologies in order to demonstrate their use and impact in the shipping context, develop stakeholder oriented methodologies and tools, provide reference cases, and further populate the aforementioned framework. The goal in selection is to target high-priority problems that will cover a variety of shipboard areas, work types, work environments, and stages in the ship design and deployment life cycle (from requirements to inspection and maintenance).

These results can be applied by European shipyards and owners to build ships that have a competitive advantage with respect to usability; which experimentally results in an increase of safety and may also turn out as an advantage when competing for well-trained crew in the future.

Project Results:
Accident investigations show that the majority (up to 80%) of ship accidents are caused by human error. The causes are manifold, including aspects such as difficult working environment, crew qualifications, as well as lack of human centered design of equipment, of working spaces and/or of human machine interfaces.

The research project Cyclades (“Crew-centered Design and Operations of Ships and Ship Systems”) started in October 2012 and was finalized in September 2015. It aimed to increase the safety for ship, crew, cargo and environment by improving the current lack of implementation of human-centered design (HCD) principles in workspace and equipment design. It is expected that better access to information regarding HCD will improve the design of workspaces on board and thereby reduce the likelihood for human error.

The project can be subdivided into four phases: In the first phase causes of maritime accidents and incidents were investigated. In the second phase the state of the art of human-centered design was compiled and analysed. The focus of the third phase was to support maritime stakeholders with HCD material. In the final phase of CyClaDes the practical applicability of HCD methods was tested.

The following subsections will summarize the work done and describe the project results that have been created during the different phases.

1.3.1 Analysing the causes of maritime incidents and accidents
Living and working on board ships is challenging; the work environment is harsh (e.g. distractions from noise, vibration, changing environmental and climatic conditions) and the workload is high (e.g. due to shift-work, physically demanding tasks, tight schedules, economic pressure, strongly perceived responsibility for the safety of crew, ship and cargo). The crew is separated from families and friends for months on end. Communication between the crewmembers is often hampered by poor knowledge of English amongst multinational crews. Additionally, advancements in information technology and increased automation in the control rooms and on the bridge have extended the tasks of the crew from traditional seamanship towards the operation of complex computerized systems. Furthermore, the increased amount of paperwork is a time consuming factor that distracts the crew from their classical tasks.

The CyClaDes project started to analyse the causes of maritime incidents and accidents from different perspectives. In this phase of the project accidents and incidents were compiled in a database and analysed using the TRACEr (Technique for the Retrospective and predictive analysis) taxonomy. Ethnographic studies were carried out to get a better insight of the current working conditions on board of ships. Measurement techniques (eye tracking and movement sensors) have been applied to investigate the mental workload of seafarers. A methodology for the incorporation of the human factor and the assessment of its contributing factors was developed and practical ways of assessing and applying resilience to maritime operations were explored. Incident and accident database
In the beginning of CyClaDes an accident and incident database (“MaRiSa database”) was developed. The database aims to study aspects of Human Machine Interface (HMI) in accidents to identify priority safety concerns. The accompanying documentation also explores safety culture in shipping. The data coded and entered in the database comprises 427 near misses and incidents from Hellenic shipping companies and publicly available incident reports, 42 publicly available accident investigation reports published by the Maritime accident investigation Branch (MAIB) of the United Kingdom (UK), the Transportation Safety Board of Canada (TSB) and the National Transportation Safety Body (NTSB) of the United States of America and 129 publicly available accident investigation reports covering a three year period from 2009 – 2011 across all maritime accident investigation boards were entered in the database and analysed with the TRACEr (technique for the retrospective and predictive analysis) taxonomy utilised to facilitate the coding and analysis of incident and accident data.

Additionally survey and questionnaire studies for seafarers and shipping companies was prepared, distributed, and finally analysed. The respondents for the survey studies include 46 seafarers and 14 shipping companies. 24 semi-structured qualitative research interviews were conducted to supplement the incident and accident analysis and the survey studies. Of the 24 interviewees, 23 were seafarers and 1 company owner. The survey was completed by the analysis of information provided in alternative data sources of internet forums, social networks and anonymous/confidential reporting scheme and the organisational impact on maritime safety and includes studies performed in other industries – offshore industry, helicopter flight safety, aviation, and medicine and chemical/process industry.

The results of the TRACEr incident analysis show that 34.66% of all incidents were ‘machinery’ related, 23.65% were ‘personal accidents (slips, trips, falls)’ and 15.22% were ‘fires’. With respect to incidents, 176 task errors task errors were coded for the ‘deck’ followed by the ‘engine room’ in 144 cases and the ‘bridge’ in 62 of the incidents. Implying that the bridge is safer than the deck and the engine room, highlighting that measures need to be taken in these parts of the vessel. Furthermore, the main incidents/near misses on board are the personal incidents related to slips, trips and falls. The findings of the incident analysis can be compared and contrasted with the findings of the accident analysis, given below.

The results of the TRACEr analysis show that two-thirds or 67% of the accidents involved the human-machine interface. The majority of accidents could be identified as personal accidents (44%), that is accidents that happened to individuals, followed by collisions (29%) and groundings (15%) in descending order. Most errors could be associated to operations on the bridge (50%) followed by deck (38%) and engine room (12%). The bridge being the command and control centre of the ship featured prominently in the accident analysis, followed by accidents on the deck, which involved cargo operations and mooring operations, followed by the engine room, in descending order. The personnel the accidents were attributed to, were the captain who was identified in 19% of the errors in the reported accidents, followed by the chief officer (14%), able bodied seaman (AB) (9%) and the pilot (8%). The identified tasks which led to the coding of the task errors were navigation (23%), followed by traffic monitoring (18%), cargo work (18%), maintenance work (15%) and mooring operations (4%). The equipment involved in the task errors were identified as radar (n=64), loading devices (n=13), mooring equipment (n= 11), stairs, ladders (n=11), steering panel (n=11), engine room controls (n=10), Very High Frequency (VHF) radio (10) etc. The accident analysis highlights that the interactions between workspaces, operations, personnel and equipment are crucial for safe operations and should be considered when designing ships and shipborne equipment or operations. The accident analysis drilled down from the general to the specific, from the type of accident to the location of the accident, to the operator involved in the accident to the operations being undertaken and the equipment in use at the time.

With respect to the detailed accident analysis of grounding and collision accidents, 96% of the errors were performed on the bridge. The majority of the erroneous acts performed by the operator fall into the task error category “supervision” (28.72%) followed by “traffic Monitoring (19.68%) where the main technical equipment involved is the radar (17.55%).

The incident and accident analysis highlights the shipboard areas, operations and equipment involved in incidents and accidents that could potentially benefit from design support utilising HCD principles, thereby contributing to maritime safety. Ethnographic studies
One long term goal of CyClaDes is to improve the role of the human element in ship design. To this end, information about the current situation is indispensable. Ethnographic studies were carried out to get a better insight of the current working and living conditions and the improvement potential.
Ethnography is a methodology to collect and analyse mainly qualitative data in field studies, techniques used were interviews, observation and archival studies. The general principle of human-centred design underlies the entire Cyclades project; in that expanding on their Hu-man Factors knowledge is something most stakeholders in the domain need to do. The results include a number of ways to represent and access information about the context of use and the seafarers as users on board. Photos, movies and text excerpt are available as a resource during design stage or as a possibility to learn more during studies or career.
The findings were compiled in a mind map to be a repository of easily accessed and easily understood information about what life and work on-board is like. The intention is that it can be used in teaching at student level but also when at work as a ship designer or naval architect. The results of the study were included into the CyClaDes framework that will be described later in this document. Assessment of the crewmembers effectiveness and fatigue
This part of the project was aimed at developing knowledge and defining the usefulness and applicability of a novel monitoring system of the operator’s behaviour and cognition, assessing fatigue and alertness by movement sensors and eye tracking techniques. Eye tracking
The construct of fatigue is still very much present in the maritime domain and has been seldom investigated in relation to mental workload. The indices of ocular activity are among the class of measures used for studying operator mental workload, particularly because the recent availability of less intrusive and less expensive eye-tracking systems. The eye tracking technique has been used in a consistent number of studies to investigate the operator functional state in several stress-related work domains. However, the studies carried out in the maritime context are too few compared to the aviation and the automotive contexts. Likely, this is due to the fact that the maritime domain has only recently been adopting technological changes that need the mental workload to be addressed.
The research in this task of CyClaDes focussed the attention on the operator functional state in relation to eye-movements and how ocular data may assess the operator mental workload in the maritime domain, has been carried out in a series of validation studies.
In order to understand the functioning of several systems on the bridge, the Officers approach to the instrumentation, preliminary observations carried were out on the “MN Splendid” (docked) followed by the data collection (also using wearable eye-tracking glasses) aboard of the “MN La Suprema” (along the Genoa- Palermo-Genoa route). Goal of this activity was also to identify the instrumentation to be monitored in the future research phases. Among the instruments, the ARPA radar appeared to be the piece of equipment most consulted by officers, especially in poor visibility conditions, suggesting that this is the part of instrumentation we should have taken into consideration in the following phases of the research.

In the laboratory experiments, a visio-motor task with three levels of complexity was carried out in order to investigate the sensitivity of the dispersion of eye fixations to variation in workload, also investigating the relation between subtle motor activity and mental workload using an eye-tracker device with capacity sensors.
A Matlab software package (ASTEF II) was specifically coded for computing the Nearest Neighbour Index (NNI), an index of spatial dispersion that has been repeatedly found to be correlated to mental workload. Results of the validation study provided additional evidence for the distribution of eye fixations as an indicator of mental load by showing a significant effect of taskload on the NASA-TLX weighted scores, on the distribution of eye fixations, and on the average excursion of movements over the chair.
A successive comparison of the operator functional state when the participant received (adaptive) automation support (based on the changes in the NNI) and when he/she was on his/her own, showed that time spent within the tolerance limits was significantly longer when adaptive automation support was provided.
The experimentation at the World Maritime University’s (WMU) simulator was carried out in order to investigate the sensitivity of an algorithm based on the analysis of the fixations distribution to variations in mental load during different navigation conditions and results have confirmed the usefulness of eye-tracking as a measure of mental workload in the realistic simulation environment.

These studies, together with the idea underlying them and the main scientific results, are reported in the following research papers:
• Di Nocera, F., Proietti Colonna, S., Dessì, F., Capobianco, C., Mastrangelo, S., Steinhage, A. (2013). Keep Calm and Don’t Move A Muscle: Motor restlessness as an indicator of mental workload. In De Waard, D., Brookhuis, K., Wiczorek, R., Di Nocera, F., Barham, P., Weikert, C., Kluge, A., Gerbino, W., and Toffetti, A., (Eds.) (2014), Proceedings of the Human Factors and Ergonomics Society Europe Chapter 2013 Annual Conference. Available as open source download. ISSN 2333-4959 (online).
• Proietti Colonna, S., Capobianco, C., Mastrangelo, S., Di Nocera, F. (2014). Implementing dynamic changes in automation support using ocular-based metrics of mental workload: a laboratory study. In D. de Waard, J. Sauer, S. Röttger, A. Kluge, D. Manzey, C. Weikert, A. Toffetti, R. Wiczorek, K. Brookhuis, and H. Hoonhout (Eds.) (2015). Proceedings of the Human Factors and Ergonomics Society Europe Chapter 2014 Annual Conference. ISSN 2333-4959 (online).
• Di Nocera, F., Mastrangelo, S. (2015). Not only fatigue: Why designers should take mental workload into account. In: Improving Ship Operational Design (38-39). London: The Nautical Institute.
• Di Nocera, F., Mastrangelo, S., Proietti Colonna, S., Steinhage, S., Baldauf, M., Kataria, A. (forthcoming). Dynamic assessment of mental workload using a wearable eye tracking tool and capacitive movement sensors. In: Human Factors & User Experience in everyday life, medicine, and work. Proceedings of the Human Factors and Ergonomics Society Europe Chapter 2015 Annual Conference.

Summarizing what has been thoroughly presented in the above mentioned Papers (and in D2.2 and D3.5), we provided additional and newer evidence concerning usefulness of eye-movements metrics (as an indicator of mental load) and their compatibility with simulation settings and for future adaptive automation solutions. This latter is an original result, that was never achieved in the literature before. Furthermore the sensitivity of fidgeting to task variability and therefore to taskload, highlighted a completely new way for the growth of future low-cost strategies for operator functional state sensor monitoring systems in operational settings. Movement sensors
Human error, lowered attentiveness and distraction based on stress and work overload is one of the major causes for ship accidents. One approach to address this problem is to investigate in how far an enhancement of the design of the static workspace, e.g. the bridge or the engine room, can improve the attentiveness and lower the workload of the crewmembers. However, there are also many dynamic factors such as the current traffic situation, which have a strong influence on the crew’s workload and concentration. A very simple sort of sensor for attentiveness is the so-called “dead man’s switch” which has to be pressed by the Officer on watch on a regular basis. However, the necessity to interact with this appliance can be distracting and annoying such that the stress level is even enlarged.
Consequently, one research direction within CyClaDes was to measure automatically the current stress-level of individual crew members such that countermeasures like the temporal replacement of overloaded personnel can be undertaken in time before the situation becomes dangerous.
As studies suggest, the average movement of participants during mentally demanding tasks is a good indicator for workload. Therefore, it was decided to develop a movement measurement and classification system for this purpose within the CyClaDes project.
The system used is based on the large-area capacitive sensing technology of CyClaDes-partner Future-Shape GmbH. This sensor system consists of a 3 mm thin textile composite with built-in electronics. It is capable of measuring the proximity of body parts even through rigid materials such as conventional floor covering. Consequently, the bridge simulator room of CyClaDes-partner WMU was equipped with 20 m² of this sensor underlay by placing it invisibly underneath the laminate floor covering. During simulator sessions, it was now possible to observe and record the trajectories of walking crewmembers with a spatial precision of around 10 cm and a temporal resolution of 10 Hz. To capture not only walking movement but also the variations of a crewmember’s seat position e.g. during navigation tasks, the original sensor principle has been refined such that the capacitive sensor is capable to measure mechanical pressure variations. This has been accomplished by laminating an additional textile compound consisting of a conductive layer and a spacer textile on top of the capacitive underlay. By sliding this sensor mat under the seat- and backrest cushion of a conventional captain’s chair, it was now possible to detect and record variations in seat position with the same spatial and temporal resolution as in the floor. For the sampling, labelling and evaluation of the data a comprehensive MATLAB® software was developed which also serves as a real-time data visualisation tool during the simulator experiments.
The suitability of the seat sensor system for a raw classification of the workload level has been investigated in a control experiment where participants had to solve hand-eye-coordination tasks of varying complexity.
During a first simulator session with 15 test persons, floor- and seat-movement data were recorded while the participants had to solve navigation tasks of two different complexity levels. Subsequent analysis of these data revealed a clear dependency between the average movement activity on the seat (motor-restlessness) and the complexity of the task. As predicted by previous research works in the literature, the level of the participants’ movement activity on the chair changes significantly, when the demands of the task and thus the workload- or stress-level increased. Therefore, the real-time analysis of the average movement activity seems to be appropriate to serve as a rough classification of the current workload.
For a second experiment with 20 participants, the workload phases which consisted of navigating through waters with varying density of ship traffic were refined. Not only movement data from the seat sensors but also from the floor sensors were captured and correlated with the actual workload level. The result showed a clear relation between workload and movement activity. The majority of the participants displayed an increasing movement activity when the workload increased. A small number of participants showed a negative relation which means that they became calmer in stress situations. Those participants displayed a lower level of movement activity in general.
Based on these findings we believe that it is possible to develop a device which can be installed on a ship’s bridge and which indicates phases of increased workload. We believe that by means of such a device, the risks of accidents caused by fatigue due to increased workload could be reduced. Risk assessment and crew performance prediction
The goal of this task of CyClaDes was to investigate the quantifiable influence of the human operators on the risk associated with an operation.

In the context of the quantitative risk analysis, a methodology for the incorporation of the human factor and the assessment of its contributing factors was developed to more adequately include the user in risk assessments. This approach is based on the elaboration of a model with two interacting segments; a Bayesian Network (BN) model and an Event Tree like approach (ET); i.e. a “dynamic” and a static one. The BN model is embedded with integrated elements from the Technique for Retrospective and Predictive Analysis of Cognitive Errors (TRACEr) and the Standardized Plant Analysis Risk Human Reliability Analysis (SPAR-H) and focuses on the calculation of the collision accident probability due to human error. This probability is inserted into the ET model which allows the calculation of the human, economic and environmental risk (in the format well used from various Formal Safety Assessments (FSAs)). The model takes into account the human performance in normal, abnormal and critical operational conditions and implements specific tasks which are derived from the performed analysis of the task errors leading to the collision (CN) accident category. A sensitivity analysis was performed to identify the most contributory factors to ship collisions. Finally, the verification of the model is based on statistical data from the Dover strait.

Given the produced results, important conclusions can be elicited: it is apparent, that the collision probability is greatly dependent on the human factor, which plays a paramount role in a possible collision scenario. Factors directly connected to the human element, such as assessment, detection and performance exceed by far as the most important factors. Other important factors including machine based factors (radar detection) and interaction factors (external communications), contribute, but in a lesser level. Furthermore, the implemented probabilities from the BN model into the ET model and the verification of the outcome using traffic data of a high density traffic area leads to CN frequency similar to CN frequencies determined by the FSA on RoPax or by the Goal based Damage Stability (GOALDS) project. Subsequently, the enhanced consideration of the human factor allows the assessment of measures for the reduction of the collision accident probabilities and the derived consequences by adopting specific design alterations with respect to Human Centered Design (HCD); Resilience engineering
Safety is more than the absence of accidents. Recent research on ‘Safety 2.0’ and Resilience Engineering has attempted to identify the positive characteristics of safety and the role of the operator in ‘making safety’. This task of CyClaDes examined the topic to identify practical ways of assessing and applying resilience. The topic is still really in the research phase, and suffers from a lack of clarity. The concepts in general and academic use are frequently new and ill-defined, so clarification is necessary. Accordingly, this task set out and attempted to clarify the concepts associated with resilience and Resilience Engineering. The concepts proposed here are based on an extensive examination of the literature. A fairly rich set of concepts with some degree of clarity and coverage has been assembled. The application of Safety 2.0 to ship operation with its relatively low standards of safety must be seen as experimental. The extent to which processes from High Reliability Organisations can be transferred to mainstream organisations is unclear. However, tools and measures from Resilience Engineering have the potential to improve safety in a way that complements Safety Management Systems.
Assessment measures were identified; some of these have potential for application in the shipping context, with some tailoring. The recently developed Organisational Resilience Health Check looks to be a promising way of measuring organisational resilience and could be of value to ship operation. At a business level, there is a case for studying resilience in the context of greater complexity and coupling in shipping. The question of whether shipping is becoming more vulnerable to systemic accidents is an open one, but potentially important. Resilience offers a broader focus than traditional system safety engineering.

Finding ways of modelling “hard” resilience is still at the research stage. The extent to which it offers a new theoretical basis is still unclear, but the broader focus offers the potential for a more cost-effective approach in practice. Certainly, the alternative of greater system complexity and redundancy has distinct disadvantages. There is a real shortage of measures of ‘hard’ resilience that might be used to drive design. Some approaches to modelling ‘hard’ resilience were examined with a view to future application. Three approaches to modelling were examined in detail. They indicated the potential success in practical application.

• A workshop exercise to examine the resilience of ship operations has been developed. The adaptation of workshop methods to shipping appears to offer a viable tool for discussion “what-if’” questions from unexpected combinations of events to develop mindfulness.
• The examination of bowtie modelling and use of barriers showed that this approach can be useful in the analysis of a real ship hazard situation.
• The Functional Resonance Analysis Method (FRAM) modelling showed that it is possible to follow all the steps of the FRAM modelling. The FRAM model for the evolution of a fire on board ship highlights the importance of functions in describing the whole process. The application of the risk based hypothetical scenario for the fire on board indicated that the FRAM can support estimates of, and increase resilience at a ship level in an effective and illustrative way.

1.3.2 Identification of state of the art and weaknesses of current processes

Human-centered design is already a concern in other industries (aviation, space, rail, etc.) and particularly in some research and development fields of the maritime industry. In order to utilise this knowledge and make it more accessible to designers, operators and authorities, this task aimed to compile those best practices, methods and knowledge about common errors across the design and operation life cycle.
During this project phase it was further investigated whether human-centered design is already used by ship yards and ship operators and to what extent. Based upon these results, the potential use of such techniques should be determined.
A lot of human factors and ergonomics recommendations exist (e.g. in class rules) and ought to be applied in ship design process. These were compiled and analysed regarding their applicability. Another aim of this phase was to discuss how usability can be evaluated by regulators and how usability requirements can be provided that enables the equipment and system designer to develop products with better usability and to allow the regulator to measure usability. First Stakeholder workshop
One of the first steps in the CyClaDes project was the analysis of the current practices and the innovation potential in the HCD domain. During this first project phase it was investigated whether HCD is already used by ship yards and ship operators and to what extent. Based upon these results, the potential use of such techniques should be determined.
A first workshop was held to gather information from stakeholders and to collect initial requirements for the development of tools and methods to promote HCD in the shipbuilding and shipping industry. Despite the project partners, representatives of the three major stakeholder groups attended the workshop, including system design engineers (equipment designers), naval architects (ship designers), and owners and operators. They provided their view on the topic and especially discussed what the envisaged CyClaDes framework should look like in order to promote HCD in a wider range of applications. The workflow, inputs, and potential places to impact the process with more human element input were also discussed and open questions identified. These findings were gathered and portrayed such as in Figure 1 for System Design Engineers to better understand how the project could integrate with the stakeholders’ processes. The conclusion was that the goal of CyClaDes to develop an HCD framework is highly interesting for the maritime industry. It would directly address an issue that currently lacks a convincing solution.

Figure 1: Workshop Results (System Design Engineers)

Based upon the workshop findings, development of an incident and accident database, a framework for promoting HCD material and the development of training material E- was initiated. After the developed software had reached a mature status, further workshops were held in order to collect user feedback and to further improve the results. Finally, a public presentation at the very end of the project was given to an interested public,
Briefly described, the first workshop yielded some remarkable results:
• All participants regarded HCD to be important.
• Nearly all participants claimed that they already use HCD. However, usage is poorly formalized and only applied partially.
• Interest in the topic and the envisaged framework was high. Participants expressed a significant demand to be supported by methods, tools and guidelines.
• The participants agreed that the most promising way for increased HCD use would be the introduction of rules that would force the maritime industry to consider HCD aspects.

The conclusion was that the goal of CyClaDes to develop an HCD framework is highly interesting for the maritime industry. Together with an incident and an accident database to provide the rationale and with an E-learning platform to introduce HCD to different stakeholder groups, a comprehensive software environment was established that was evaluated in sub-sequent workshops. The solution found was regarded as valuable toolbox that could, once made available to the public, be a convincing means to support interested parties in learning and applying HCD in shipbuilding. Therefore, the outcome of this workshop provided the rationale of the developments in the subsequent work packages. Best practices in the maritime industry
In the first part of this task, regulatory material (i.e. from international organisations, national administrations and classification societies) was identified and described. Some mandatory requirements were found but these are often very unspecific, making them very difficult to apply and to verify compliance with. Although many additional advisories, recommendations or guidelines have been found, these are not mandatory and compliance is only on a voluntary basis. Challenges for the regulators to develop more specific HCD-related rules and regulations were also discussed. Two alternative approaches could be followed: requirements for the product (i.e. the result of the design process) or requirements on the design method itself. The technical pros and cons of both approaches were discussed but the main challenge appears to be the need for a driver for developing such new rules and regulations and compelling compliance.

The second part of the research highlighted the very limited attention paid today to the end-users needs and work, apart from what is directly required by rules and regulations, by people designing, building, manufacturing and selling maritime human-computer systems. This may be driven by the fact that such people are primarily engineers with neither the natural inclination towards the ‘softer’ sciences nor the training and education needed to understand, appreciate and successfully apply human factors knowledge. Models of Usability Capability Maturity have been provided to illustrate that moving ahead is a cultural change which requires dedication and investment, and contrasted this to the salient point that the various stakeholders do not appear to have any natural inclination towards changing the present practice, in lieu of a more human-centric approach, especially when ruled by an economic reality as the maritime industry is. Having said that, this part positively concluded that there is ample room for improvement in the design of maritime systems, and that the CyClaDes project, as expressed especially by the CyClaDes framework, is an opportunity to support a positive change. To support this change guidance is provided for developing materials that would resonate with the typical engineer’s preferences for thinking and learning.

Finally, ten maritime design projects where HCD has been applied were described. Although these descriptions give the impression that HCD is gaining both momentum and traction, at least within the marine equipment design community, this impression might not be entirely accurate as it remains to be seen if these companies take measures to include HCD outside of these specific, high-profile, projects. Indicators of increased adoption might take the form of hiring or training individuals in the company staff in HCD, more consistent use of HCD consultants during design and the overall reaction to the new e-Navigation standard concerning HCD, usability testing and software quality assurance. Best practices in other industries
The goal of this task was to evaluate best practices from other industries and identify human-centered design information that could be applied to the maritime industry.

HCD appears to be explicitly applied to some level in all of the four high-hazard industries reviewed (oil and gas; railway; aviation; healthcare). There is also evidence that each industry has tried to look towards other industries to learn about best practice and learn lessons about the design of applications in high risk environments. This is clearly in line with what the Cyclades project has tried to achieve in setting out this piece of work. However, it is apparent that the practice of HCD does not always comply with the “good practices” advocated for the industry concerned. Specifically, although there are guidance documents that have been produced on the topic in each of the industries, such documents are not embedded in standard industry practice and thus are not consistently referred to by designers. Further, not all designers appear to consistently consider HCD to begin with (i.e. it is not an established “way of working”). Despite the inconsistent application, progress has clearly been made in each of the industries towards incorporating HCD; each recognizing the value of HCD and the rationale for its use is largely similar across the industries. Overall, it appears that the oil and gas, rail, aviation and healthcare industries acknowledge that the main advantages of HCD (and Human Factors Engineering—HFE) are a saving on costs from minimizing the need for significant changes to devices or facilities after they have been built. However, HCD is not fully embedded in these industries as standard practice, with various stakeholders (including clients who commission design projects) not fully understanding the value that HCD can bring. Regulatory regimes that do not require adherence to HCD may also weaken the impetus to implement HCD by clients and designers.

In the second part of this task a detailed review of the available materials from other industries outside of commercial maritime was performed. An extensive table was created to document the materials available in the transportation, industrial, medical, computer, communications, and military industries. The results of the review made it clear that there is a significant amount of relevant information that is available throughout the literature and online resources to aid the various components of ship design but it requires a good deal of effort to identify it, time to read and understand it, and time and knowledge to adapt and apply it. Any steps that that CyClaDes project can accomplish to reduce these constraints should significantly increase the stakeholders’ awareness and willingness to use the design criteria and some of the methods in their work.

The result of these activities was a detailed look into how HCD was being applied and encouraged in other domains, as well as identifying best practices, resources, criteria, and potential tools that could be then applied to the maritime domain. These materials were prepared for dissemination to the stakeholder audience in a usable format via the Framework, e-learning materials, “Improving ship operational design” book, naval architect training materials, guidance documents, and project case studies. The usability of HCD guidelines for maritime workspaces
A lot of human factors and ergonomics recommendations exist and ought to be applied in ship design process. However, there is often a distance between the designers, the regulators and the operators perspectives. The main questions of this task are to evaluate if current guides and recommendations, if followed, would really fulfil operator’s needs and discuss how existing guides and recommendation are applied in a ship-design process.

The plan for this part of CyClaDes was to assess the usability of HCD guidelines. The task completed an audit of rules and international or national regulations and guidelines on maritime workspaces from the Human Factors perspective. In addition, two examples focus on applying those rules and regulations to the machinery spaces of ships.

The work consisted first in a literature review of relevant Human Factor areas, especially ergonomics (Human Machine Interface or HMI), Occupational Health and Safety and current state of the art in the maritime industry as well as other relevant industries.

In order to verify that the existing literature matches the actual need on board, experts in operations observations were used. Information was collected through interviews with Bureau Veritas (BV) employees (in offices and in the field), visit in ships’ engine rooms and machinery spaces and interviews with captains and other seafarers. Additional sources were also used to complete these observations.

Afterwards, correspondence between the issues raised and the existing recommendations regarding familiarity, occupational health, ergonomics and risk issues have been examined.
Two aspects can be derived from this analysis:

a) One of the most important outputs from the user-feedback is that existing prescriptive guidelines and rules address a large part of the real observed HCD problems in machinery spaces. Part of the rules are functional requirements (with high level requirement without specifying how to meet it, e.g. words as “suitably”, “adequately” “easy access” etc.).
b) The remainder of the user feed-back items were not directly addressed by rules and should be better analyzed, possibly using HCD. These are for example specific alarm processes, handling of heavy equipment or parts, clearance and access.

The first aspect a) questions the effectiveness of the application of the existing recommendations. There are different factors that could explain this:

• Non usable recommendations/guidelines:
o Prescriptive requirements are either too high level (functional requirements) or too specific (to be used at early stage)
o Many guidelines and recommendation exist, but they may be poorly known, used and understood by designers HCD studies are too long and costly to be fully applied at design stage, while commercial pressure and shorter delivery times is often the governing decision parameter for the owner
o According to interviews, most of the time the human element issues are detected after construction, when the ship is either visited (construction or 1st sea trial), used (first years of ship life), and maintained.
• In reality, enhancement of the design is rather performed after building
o Some of the design issues can be modified on-board during ship life (typically location of valves, ladders/steps for better access/visibility). But addressing many issues may not be feasible.
o Sister ships can and should benefit from the first prototypes errors.
o To progress from this situation, one Captain interviewed suggested tests for lifting and transporting heavy parts to be performed at delivery, with lifting capacity to be checked as well as space to transport the element. This is typical from HCD analyses see next section.
• Verification of application of the existing recommendations:
o Verification of the degree and compliance of an HCD at early design stage with the sole availability of layout/drawings appears to be much more complex and difficult than usual class work about naval architecture.
o Except on main safety issues (for example Alarms), many of the human element related recommendations are not mandatory recommendations.
• Application of the existing recommendations
o Application of HCD guidelines is rather a choice/ a conviction of the owner, and sometimes of the Authorities.
o There are too few incentives for applying the existing recommendations or HCD guidelines and recommendations.
These results therefore highlight:

• The need for actual application of existing recommendations
• The need for practical design tools to direct the owner/designer/class on specific recommendations (Cyclades Framework).
• Promote generic HCD analysis performed by experts groups to provide generic analyses or pre-analyzed equipment, or arrangements.
• The need for earlier detection of HCD errors and enhancement at design stage and on prototypes.
• A verification scheme to check that the design actually reaches ergonomic or HCD functional requirements.
• Consider improvement of the design towards Human Element in as built conditions and not only at design stage.
• Train/publicise the existing guidelines/recommendations to class, designer and owner.
• Work on incentives for more HCD analysis at design stage (class/designer/owner).

According to a) and b), a combination of use of existing recommendations and dedicated HCD analysis is therefore suggested. In that sense any specific HCD analysis should make reference as much as possible to these existing recommendations. This should be done in the general HCD analysis process. However, specific design features as well as new equipment or operations could be considered using more performance based technics, although they also include feasibility drawbacks. Handling of usability requirements from a regulatory perspective
The aim of this task was to discuss how usability can be evaluated by regulators and how usability requirements can be provided to enable the equipment and system designer to develop products with better usability and to allow the regulator to measure this usability.

Today, usability evaluation by the regulator is either based on mandatory requirements addressing usability aspects, or based on non-mandatory recommendations. The evaluation is characterised as a verification of compliance to these requirements by the regulator. These requirements are not explicitly addressing usability in the sense of ensuring effectiveness, efficiency and satisfaction when using the equipment. Usability is contained in the requirements more implicitly, by providing requirements addressing aspects of usability, mainly ergonomics and human-machine interface aspects, but also some aspects relating to HCD. For several shipborne equipment, especially equipment implemented on ships’ bridges, performance standards exist. Some of them contain methods for equipment tests and also specifications of the test results. However, these requirements predominantly address technical performance and have less focus on usability aspects. Many of the requirements that address usability aspects are formulated in a general and unspecific manner (e.g. equipment shall be “easy to use”, designed for “ease of understanding”). They leave room for interpretation by both the designer and regulator.

By applying the existing requirements to the design of an equipment or system, usability is actually not measured in a quantitative or semi-quantitative way, e.g. by applying usability test methods. Usability, or better, usability aspects as far as contained in the requirements are evaluated in a more qualitative manner. The motivation of improving the above described current situation is to push for a more consistent and holistic consideration of usability in the design of equipment and systems and to enable regulators to evaluate usability more consistently and to provide better information regarding what aspects are influencing the usability and therefore need to be considered in design and evaluation.

Two general approaches for improving the situation have been identified:

The first approach is aimed at improving the existing requirements in the current international regulatory instruments by clarifying what they mean. Currently several of these requirements are formulated generically and they are not specific, leaving much room for interpretation. This may be done by amending the relevant instruments and providing interpretations for each of the requirements. It is noteworthy that the current situation with the existing requirements may change if specific user centered design knowledge and competences are demanded to be present and participating in the design team and process, and if evidence of usability is demanded.

The other approach is aimed at establishing a practice of more holistically considering usability during the design of equipment and systems. This practice should be based on the principles of a HCD process and include usability testing. In accordance with these principles, the evaluation of usability should also have a stronger focus on the users, their tasks and their goals when they use the equipment and/or system, as well as the context of use, the working environment and the conditions under which the user carries out the tasks with the equipment. The usability evaluation would focus on to what extent and/or quality usability aspects have been considered during design and also on the performed usability testing carried out for the equipment. The approach differs from the first one mainly in the holistic nature of the design and evaluation process.
It distinguishes four stages of a HCD process, i.e. pre-design, concept design, detailed design and evaluation. In each stages of this process dedicated activities need to be carried out in order to achieve equipment design with high usability. The activities are documented in deliverables, which are the basis for the regulator to carry out the usability evaluation. The evaluation is carried out by applying evaluation criteria on how poor or well this activity has been carried out, and/or on how poor or well relevant usability aspects have been considered in equipment design. A usability scoring concept has been developed.
This approach may create the basis for a voluntary certification scheme for equipment usability. Hence it could help in pushing for the application of human-centred design and usability aspects in the maritime industry. Equipment that has achieved and has been certified a certain usability score or level may have a competitive advantage on the market as the certification lends to the expectation of a purchaser that the certified equipment can be expected to have a higher level of usability than an equipment without usability certificate.
Based on the above mentioned advantages the second more holistic approach was developed within the project. A detailed concept has been developed containing requirements during the design process and a scoring system allowing the evaluation of the design process and the usability testing. Thus, a specific equipment can be certified to have reached a certain usability level. The concept has sufficient detailed to be used as the basis for the development of future class guidelines specifying the usability certification process.

1.3.3 Improving the availability of information regarding HCD

As described above a lot of knowledge and information is available on how HCD can be achieved, both in the maritime industry and in other industries, but this knowledge is not widely applied. Therefore CyClaDes published three resources to support maritime stakeholders:

1. a web-based Toolbox „CyClades Framework“,
2. an E-learning platform and
3. the 2nd edition of the book „Improving ship operational design“ The CyClaDes Framework

The results of the first CyClaDes stakeholder workshop indicated that reasons for not applying HCD knowledge include:

a. existing knowledge and methods are not readily accessible to stakeholders (both physically and in the way the information is presented),
b. crew-centered design does not have a high priority on stakeholders’ agendas.

To the first reason, a web-based database (“CyClaDes Framework”) has been developed to organise existing guidance, knowledge and experience to support a more user-centred perspective in the design and approval of ships and ship systems.

The framework addresses the following main goals:

• To determine relevant requirements for design and operation that focus on the needs of the crew.
• To provide a selection of applicable guidance and method(s) that enable designers, operators and approval authorities to assess systems and work spaces aboard ships with respect to crew-friendly systems.
• To provide further reading and information about living and working on board of ships that help designers to better understand the environment and conditions under which their products will be used.

The CyClaDes Framework covers user requirements to the design and Implementation of systems and processes that are specific to particular type of task or work space.

The framework was developed following a human-centred design process; In the beginning the requirements of different users of the future framework were elicited in a stakeholder analysis that built upon the results generated by a stakeholder workshop. The following stakeholders were addressed by this work and these are considered the users of the framework:

• system designers (users of tools, methods and best practices for the human-centered design and analysis)
• ship designers (users of tools, methods and best practices for the human-centred design and analysis of workspaces)
• owners and operators (land-based personnel, typically former users) who are involved in requirements specification for newbuildings and planning of work processes)
• authorities and rule makers (users of means for evaluation of designs with respect to human element influences)

Following the stakeholder analysis, information about human centred design (including tools, methods, guidelines, legal requirements, best practices and design examples) was collected in accordance with the requirements of each stakeholder group.

The collected material includes literature, information about living and working on board ships, best practices in the maritime and other industries, results of the ethnographic study, examples of good and bad design as well as new methods that were developed in the CyClaDes project. Technical solutions were developed for archiving this material in the framework and making it accessible to stakeholders.

Based on the requirements of the identified stakeholders a publicly available web based database has been developed that allows to store and access information about human centred design of ships. Users are able to contribute to the Framework by sending own examples.

The structure of the framework is organized according to the subsystems used in shipbuilding and represents all ship equipment, systems and (control) areas on a ship where human-centred design aspects can be addressed. It has been started to populate it with human-centred design information focussed on those equipment/system developments that were required by tasks in other Work packages of CyClaDes and other high-priority areas that were identified. Populating the framework with information covering the entire ship was not possible within the project’s budget and timeline but a working proof of concept was created. The framework was presented to stakeholders in three public workshops and the feedback of the participants was included in updates. If the framework is successfully adopted by the industry, the population of the framework will be continued beyond the timeframe of Cyclades. The framework in its present state addresses primarily the ship and system designers by presenting rules, guidelines, information about the end users and their context, examples and methods to better include human factor into their designs. The CyClaDes E-Learning platform

The E-Learning solution is built around Moodle, a learning platform designed to provide educators, administrators and learners with a single robust, secure and integrated system to create personalized learning environments. Because it is open-source, Moodle can be customized in any way and tailored to individual needs. Its modular set up and interoperable design allows developers to create plugins and integrate external applications to achieve specific functionalities. Moodle can be extended by using freely available plugins and add-ons.
The training packages developed aim to enable the different parties to make use of the tools developed in the project to increase the overall safety of operation within the maritime domain. The training consists of two major parts: first a course presenting common elements (Introductory Training Course) to be used in all training and awareness raising measures, secondly elements of particular need for the specific party (end-users, ship-owners, naval architects, rule makers and authorities), in order for all parties to understand their roles and the resources available in the CyClaDes-framework on HCD, which has been developed by the consortium throughout the project.
The layout of the modules has been created for the student experience; the learning path is easy to follow and can be seen throughout the platform wherever the student goes. The modular layout enables teachers to add content in an easy and consistent way. The module layout is developed around a content focus. All content can be hidden by the student to enable focus on the content he or she is viewing at the given moment. The content is also divided into tabs to give a streamlined learning experience going from left to right as the student progresses.
The e-learning platform is available online at since the end of March 2015 and was launched publicly at the CyClaDes-Bremen workshop in May 2015. In addition, the courses have been presented to representatives from academia, industry and authorities in the setting of multiple conferences, workshops and meetings. So far the platform has 41 registered users from 15 countries (amongst others Greece, Sweden, Germany, Belgium, UK and USA) that have accessed the courses. Common Training Elements
The introductory training course provides an overview of the scope of the CyClaDes-project and is directed to all stakeholder groups. The task has been a cooperation between WMU, NTUA, Lyngsoe Marine, Kalmar Maritime College (Linnéus University), and DNV GL. It aims to introduce the core concepts of HCD and show why HCD is a concern for all stakeholder groups identified (designers, operators, end users, rule-makers and authorities) by the CyClaDes-project.
This training package represents the basis for the specific training packages and is available to all stakeholder groups addressed by the project. It introduces the CyClaDes Framework, the theoretical concepts related to work conducted within the project’s work packages, shows limitations of current approaches, and demonstrates the overall need for HCD. Training for Designers
The second training package introduces HCD to the community of naval architects and equipment designers. It aims to raise awareness of how the human element can be integrated better in the design of vessels. The focus is on design and evaluation methods which can be used during the development. Training for Operators
The target group for the training course developed here is ship operators, namely staff within a company concerned with the purchase of ships and equipment and their subsequent operational use, training, and upkeep. The focus of this course is to present the CyClaDes-framework and show the benefits of applying HCD in the design of vessels and equipment. The training shows examples from field visits and interviews with crew members and company representatives centered around the questions on why and how companies can benefit from applying HCD whenever a new vessel or equipment is purchased, as well as when vessels are refitted.
The training package attempts to exemplify how a company can use the output of CyClaDes to reduce the negative consequence of “inappropriate” HCD. It starts with presenting examples for design not matching the working conditions of the mariners, and the consequences of not being crew-centered. This is augmented with an overview of priority activities and areas where crew accidents and injuries result in insurance claims and some cost-effective remedies that could reduce these. This is followed by an introduction into the CyClaDes framework and an example of successful case of the re-design of a vessel making use of a set of tools for integrating human element knowledge into the design activities. The course finishes by presenting successful examples of HCD and the benefits of applying this type of knowledge in the design of ships and procedures for work on-board. Training for End-users
The Training for End-users focuses on the potential end-users of equipment, vessels and procedures. It aims to enable the trainee to become proactive in providing feedback on design and procedures to the company, as well as it should provide means on how to express user needs and wishes as part of an equipment/vessel design process. Training for Authorities
This training package focuses on the training for authorities and rule makers. It aims to provide knowledge on HCD to flag states and authorities to highlight how and why design of vessels should take the crew into concern during the design phase. There are three primary objectives which are that the trainees understand HCD, its application in shipping as well as its assessment. The task aims to provide means and measures for authorities to assess whether a product/vessel/procedures has been developed based on HCD. Further workshops
After the framework, the E-learning platform and the incident/accident database had been finished, further stakeholder workshops were held. They collected feedback for further improvements of the different tools. However, another purpose was the collection of opinions regarding the usefulness and applicability of HCD in order to support the promotion of this method in the shipbuilding industry.
As conclusion it can be stated that the results are justifying the work of the CyClaDes project by providing a sound rationale for using HCD in shipbuilding. The analysis of the incident and accident database as well as the ethnographic studies clearly show the opportunity to reduce accidents and save human life as well as the assets by designing ships and systems that suit the needs of the crew. The accompanying workshops collected additional requirements in the very beginning of the project and also were part of the evaluation process in the end. The results of these different tasks were mainly used for the framework development but also in the design work, the preparation of the HCD book and in the creation of the e-learning environment. HCD courses for naval architecture students
To deliver fit for purpose ships and their system to seafarers, the designers need to understand the human factors issues that relate to delivering both performance and safety. Therefore there needs to be a process of education and training to deliver HCD to designers, e.g. naval architecture students.

Chalmers Technical University (CTU) developed and delivered the course: ‘Introduction of human factors & human-centered design’, in the Naval Architects Master program including lectures, ship visits and design projects, e.g. the installation of an offshore wind farm.

Human-Centered design and usability have been well-established notions in human factors engineering, general design research and interaction design since before the 1990’s, aiming to enhance effectiveness, efficiency and user satisfaction, thereby facilitating retention of personnel, decreasing costs, the need for training and the likelihood of use error. Nevertheless, it has not always been an easy implementation within design practice. For a complex network of stakeholders such as that of the maritime sector, change often meets resistance. This project, to examine the effectiveness of the HCD course, identified a number of challenges experienced when attempting to synthesize a human-centered approach within the process of a Marine Design Project and investigated solutions to overcome them.

Successful integration was found to be dependent on discovering common ground, thereby creating a mutual understanding between all stakeholders involved and a shared environment to facilitate communication. The iterative ship design spiral process, and the visual representations of the vessel it produces via Computer Aided Design (CAD) software tools, in fact provide several suitable opportunities to do so. Though time and resources were limiting factors, the main obstacle to be overcome proved to be the necessity of seeking to understand the task at hand from the perspective of the other stakeholders. Additionally, the impact of a human-centered approach may be significantly increased if one of its primary benefits were to receive more emphasis, namely its contribution to safe and efficient operations.

Example exploitation:
de Vriesa, L., Costa, N., Hogströmb, P. and Mallam, S. (2015) Overcoming the challenges of integration of Human-centred Design within the Naval Architecture ship design process. Proceedings 19th Triennial Congress of the IEA, Melbourne 9-14 August 2015

The National Technical University of Athens (NTUA) developed and delivered the course: ‘The human element – Introduction to human reliability for maritime transport’, to naval architecture students including an exercise to redesign bridge and engine spaces.

Some design requirements in the maritime industry, and in other industries, exist to cover operational and maintenance aspects, but typically fail to be applied in a human-centered approach. Such poor design creates environments where operation and maintenance is performed with a higher risk than necessary (e.g. crowded working environment,). By introducing Human Factors (HF) at the ship design phase, the designer has the ability, across the whole design process, to identify the needs of the crew members, to integrate their needs and to evaluate his work towards this objective.

In the context of the course “The human element – Introduction to human reliability for maritime transport” at NTUA a project assignment was submitted regarding human element integration in the design phase. The aim of this task is to implement HCD principles by taking into account existing design requirements and to highlight the importance of the Human Factor (HF) and Ergonomics on the design, construction and day-to-day operation of a seagoing vessel’s Bridge, Engine Control Room (ECR) and Steering Gear Room.
This approach is divided into 3 parts. At first, a brief description of the specific compartment of a ship in terms of equipment and basic layouts, the objectives of the compartment, as well as the tasks that are carried out there, is presented. The second part includes the actions undergone by the dedicated personnel and the methodology of the Hierarchical Task Analysis is used to illustrate multiple real-life situations met by the ship’s crew. These two parts are necessary in order to provide a general idea of the compartment utility and function and cast light upon potential inadequacies and defects. The final part contains from the one hand the identification of design weaknesses and discrepancies of the preselected areas of the ship (e.g. bridge, Engine Room) (always from the Human Centred Design point of view) in accordance with the Regulations/Guidelines (mainly guidelines) of Classification Societies and hetero design guidelines (e.g. ASTM) and from the other hand the redesign of the area and the arrangement of equipment by applying human-centered design characteristics and principles; this way the likelihood of the human error will be reduced and therefore the human performance will be strengthened and upgraded. The aforementioned approach is based on the analysis of multiple case studies of real ship accidents which are attributed to human error (with a special focus on HCD topics or operations associated with HCD issues) for the preselected ship locations.

Finally, the application of Digital Human Modelling (DHM) by redesigning the place of navigation of a ship’s bridge in the direction of making it more user-friendly is evaluated. DHM is the digital representation of the human in a software-based environment, used to generate prediction of human performance and/or safety. It involves advanced visualizations of the human body (segments, joints, muscles, etc.) with scientific models running on the background. Various disciplines are integrated into the platform, such as biomechanics, kinematics, physiology research, etc. The aim is to simulate the response of a worker to a given task and work environment, thus eliminating the need of in situ experimentation and data collection. This approach is based on the selection of the best place of navigation on the bridge and the installation of the required equipment which lead to the increment of the situation awareness and to the reduction of the risk due to fatigue Book “Improving ship operational design”
The book “Improving ship operational design” (2nd edition, ISBN 978 1 906915 285) is aimed at making maritime operations safer and more effective – all through the work of the naval architect. It provides some practical help to improve design, whether you are just starting out or are an experienced naval architect.
Naval Architects have a lot of power at their fingertips because bad ship design kills people.
Just to give one example: the entry to some enclosed spaces does not allow for people to go in wearing protective gear or to be rescued properly. How long would this be tolerated in mines on land?
So the maritime industry has some catching up to do and this book is full of many other instances giving practical examples to help naval architects make people’s lives more bearable and productive onboard vessels.
It is unfortunate that naval architects rarely get an opportunity to go to sea, and there are few avenues for seafarers to make their views known to designers, so we have brought shipboard life to naval architects.
The authors believe that the more sea time naval architects can experience, the better their design will be. Ideally, sea time should be included in the naval architects’ curriculum and be assisted by the shipowner community. The Royal Institution of Naval Architects (RINA) and RINA members have played a major part in bringing naval architects and users together – and in producing this book.
Human-centred design is the watchword. When designers do not take this into account, users do adapt to the workplace when forced to – but this is a sign that the design should have been better.
We hope that the practical experiences outlined in the book will help readers to interpret seafarers’ needs and translate them into design. Shipowners will thank naval architects too, as invariably safer and more efficient vessels will save them money. By working together at the design stage, naval architects and seafarers will avoid missing out on some simple operational features that will improve safety and efficiency.
The book (ISBN 978 1 906915 28 5) is available at the Nautical Institute (NI) shop since November 2015.

1.3.4 Test cases
In this phase of CyClaDes the applicability of HCD guidelines and processes was tested. Different ship systems were re-designed with focus on user-centeredness. Following test cases were selected:
• a conning display as a typical example for a system on the bridge,
• a winch control panel as a more physical control located on deck,
• the communication and collaboration between bridge and Engine Control Room (ECR). Conning display
The CyClaDes project was devised to explore a number of ways and means to address a persistent paradox in the maritime industry: On the one hand, there is a comprehensive and accessible knowledge base of maritime human factors, and on the other hand, the usability of maritime designs at large leaves much to be desired. To the point, and building on the understanding that human factors knowledge and human-centric design methods needs to be better targeted towards maritime design engineering, the CyClaDes project was designed to provide a tool which would better suit this target audience, and thus could provide timely human factors guidance in the forms and shapes preferred by the design engineering profession: the envisioned CyClaDes ‘Framework’. To support the design of the CyClaDes ‘Framework’ this task adopted a formative methodology, using maritime human factors expertise as a ‘living framework’, in order to determine the most appropriate and effective information to go into the CyClaDes ‘Framework’, to this end, this work package succeeded in producing a large amount of data which is judged to have appropriate validity and reliability.

One major work package result is that the design engineering team tasked with the design of a conning display did in fact adopt human-centric thinking and methods, which not only is a success in its own right, but which is also seen as evidence of the effectiveness of the efforts and insight of all involved, including the methodology of the ‘living framework’. Other data provided by the work package confirms earlier research on at least two counts:

• For the archetypical design engineering team, there are barriers to the adoption of a human-centric mindset; an issue which also involves the concept of usability capability maturity;
• Maritime human-centric methods should match the format and ease-of-use required by design engineering, i.e. be practical, pragmatic, targeted to particular design quests, and more or less directly applicable ‘out-of-the-box’.

A central theme in the results presented is the decisive and crucial importance of direct contact between the design engineering team and human factors expertise in-the-flesh, permitting the latter to provide advice on adapting and applying human-centric processes and methods, as well as product usability. Indeed, without this interaction, a design engineering team may not be able to successfully become human-centric. The challenge that the need for direct human interaction presents to an IT-based resource like the CyClaDes ‘Framework’ is clear; it is moreover compounded by the finding that the ‘Framework’ should promote and facilitate the early and direct involvement of representative end-users – the paradox however being that there presently are no reliable mechanisms to find and involve representative end-users at any scale. In response, the CyClaDes project has launched an initiative which affectionately has been nicknamed the CyClaDes ‘dating service’, which is a forum where design teams, end-users and human factors experts can reach out to each other, to agree and form up the multidisciplinary environments required for the successful application of human-centric thinking and methodology.

Other task results have demonstrated a serious need for human-centric training and knowledge transfer, possibly as a supplement to having direct access to maritime human factors competence. The CyClaDes e-Learning platform is seen to answer to this need; indeed, it is suggested that the once monolithic CyClaDes ‘Framework’ essentially consists of at least three legs: The CyClaDes ‘Framework’ itself, the CyClaDes eLearning Platform and the CyClaDes initiated, Comité International Radio-Maritime (CIRM)-based, ‘dating service’. Adopting a more long-term perspective, it is recommended that this dependency is not only recognized, but also that plans for their continued availability are considered and eventually secured in some fashion: If any of these components are missing then the long-term value of the CyClaDes project could be seriously reduced. Limitations of Near-field Simulator-based Test Methods
Performance assessment of ship operations and instruments are to a high degree based on real-time simulation results, but it is important to understand the differences between simulator environment and the reality, and thus the capability to generalize from simulator-based results. One of the most critical issues in close quarter and confined water navigation is the visual information (view out of the windows). It is well known that methods to visualise a 3D environment have limitations. In particular for ship bridge simulation the brightness, contrast levels and field of view are difficult or impossible to simulate correctly. Moreover 2D synthetisation of a 3D environment results in limitations regarding the ability to estimate distances and angular relations. Although these limitations have strong impact on human performance the applicable research in this field is very sparse.

The aim of this research was to
• achieve a better understanding of limitation in 3D visualisation techniques
• test and suggest methods and recommendations to mitigate some of limitations of 3D visualisation
• test and evaluate modern and innovative methods for improved ability to estimate distances and angular relations

Centred round simulator-based testing of the Conning Information Display, the work on this task aimed to provide recommendations and proposed method improvements with respect to the this particular type of simulator testing. The video uptake and analysis of precision manoeuvring at the bridge wing in reality and in simulator environment showed that a feasible method can be achieved for recognising different behaviour in different environments. The literature study also gave support to the work package with more understanding of the implications of simulator studies due to angular declination, optic flow, depth perception and motion parallax. Winch control panel
In contrast to the task described above, this task has been concerned with the usability of physical controls. Based on the accident analysis and the scenarios formulated based on the results of the accident and incident analysis of CyClaDes and with further consideration of the challenging environmental conditions, a mooring winch control panel was selected for the design exercise in this task.
Mooring operations are more or less similar even for different ship types and mooring equipment and procedures have not changed over the last decades. In addition, accident statistics suggest that the risk of mooring operations is high and therefore improved Human-centered designs may lead to a risk reduction. These aspects also supported the selection of the mooring operation for the design exercise.
For the design exercise an existing 100 m long containership has been chosen as a test case. The main concern of the crew was that the winch control panels on the afterdeck are located in the snap back zones of the lines. Due to limited space, relocation is not possible and therefore both a fixed and a portable winch control panel have been proposed.
Typical mooring equipment and operations were investigated in detail; most critical steps during mooring were identified. The researchers visited the ship, followed mooring operations and talked to the crew. An analysis of physical and cognitive tasks of operator was performed. The existing winch control panels were analysed with regards to their usability.
A literature review identified available methods, techniques, guidance and international and industry based design standards. It revealed very few methods or tools (e.g. established step-by-step processes that designers/engineers can be guided by) that were readily available for designing a user centered control panel. However, there were some useful pieces of guidance, and elements of International, national and industry based standards for ergonomic design that provided assistance.
Design specifications of a new fixed and a portable winch control panel were proposed based on guidance documents and international and industry design standards and operator’s comments and suggestions.
Finally, both design solutions were evaluated by the winch operator, by a manufacturer with experience in designing mooring winches and related control panels and regarding their compliance with existing standards and guidelines (rule-based evaluation).

This- at first glance – simple design exercise has shown that human centered design aspects are worth to be included in the design of physical controls, such as fixed and portable winch control panels. It was found that operators have difficulties in the formulation of their requirements and on the evaluation of preliminary design solutions without a proper functional prototype. Moreover, although background information exists and was used for the design of the fixed control panel, similar guidance was not available for portable devices.
Nevertheless the involvement of the ship owner and the winch operator in the design process has raised their awareness for user centred design with regard to ship equipment in general and to the winch control panel in particular, which is an important objective of the project. Communication and collaboration between bridge and ECR
This part of the project was tasked with a creating a case study in designing for novelty in the maritime domain. The plan was to apply methods to investigate communication and collaboration processes, using the communication and collaboration between bridge and engine control room (ECR) personnel as a case study. The goal of the design activities were to address the traditional void in understanding between the bridge and engine room with a focus on shared understanding of the critical information required, the flow and presentation of this information, the tasks supported, as well as the decision making and reasoning behind actions that have an impact on the other group.

At a more methodological level, the aim of this task was to systematically examine the communication dynamic between the two control centers to determine what information is required, for what tasks, when it is required, and in what format it would optimally be shared. The activity consisted of two loops, an inner loop where the actual human centered design (HCD) was undertaken, and an outer loop, where the applicability of the method was studied, based on the work, challenges and results of the inner loop. The deliverable document, 3.4, describes in detail the activities conducted by the inner loop and provides outer loop insight into the applicability of these methods, with an eye on their use by HCD-novice firms.

A variety of HCD methods were applied by the inner loop and are described in detail in CyClaDes deliverable 3.4. These included HCD methods applied to understand the context of use, priority scenarios, problems experienced, identify potential solutions and describe a solution concept. These methods included: observation, interviews, applied cognitive work analysis, extensive discussions with Subject Matter Experts (SMEs) on the project team, surveys, review and prioritization meetings with the project team, design iterations, and a demonstration walk-through of the solution concept using existing ECDIS equipment to support a shared planning session. The output of each of these methods is provided and several, such as the scenarios created and interview and observation conclusions, can serve as stand-alone support to future designers working on related design problems to better understand the tasks, context of use, and challenges for bridge and ECR personnel.

The overall goal of this task was to provide a case study to future researchers conducting research in similar domains or intending to integrate more human element consideration or HCD or Human Factors (HF) methodologies into their design. Therefore, an outer loop analyses was carried out as well to provide insight into the methodologies and how they were applied, as well as any problems or challenges that were experienced. This type of commentary and analysis was provided throughout the deliverable document with a focus on providing advice to those that may not have as extensive experience with HCD or HF. Short extracts of the case study methods were also prepared for presentation in the framework to provide concrete examples for framework users who wanted to learn more about a method and how to apply it in practice.

Overall the task was very successful and has provided a case study as well as ideas for solution concepts that can be pursued in reality, as well as others that can undergo further research and development and testing. Several HCD methods were identified as particularly useful for HCD-novice firms, such as reviewing training materials to identify tasks, observation to understand how those tasks are really performed, and interviews to fill in the details and answer key questions. Guidance on conducting these activities is provided as is guidance for other possible methods and activities. Bridge – Engine Control Room Shared Planning System

Based on the results of the bridge-ECR Case Study and to support the development of safe and effective end-user tasks, an evaluation was conducted of a touch-display table demonstrator of a bridge and Engine Control Room (ECR) shared planning and timeline tool. The evaluation was focused on usability and utility of the tool as well as the importance of including the table component and the need to meet in person instead of using remote options. The evaluation involved an interactive scenario followed by ratings and discussion.

The results were positive for conducting an in-person shared planning meeting, and in particular the engineers saw a significant advantage of doing this digitally through the table display. Clear advantages were seen for improving situation awareness of the overall voyage and specific voyage events, as well as the task and context of the other department. Several recommendations were made for improvements and additional features and functionalities. Overall it was felt that the system would have to be integrated into one of the regulated systems such as ECDIS rather than providing another redundant system. One con that was noted was that some ships will not use such a system based on their non-communicative and non-sharing management styles.

In summary, benefits can be found for both a regular procedure of shared planning, as well as for conducting this in a format that provides take away reminders and possibly other benefits back in the ECR, such as with a large screen electronic chart table. Additional discussions were held with the manufacturer partner on how such a system idea could be introduced. In addition the evaluation provided another example of how to apply HCD methods to conduct a product evaluation and how to identify potential metrics. Enclosed Space Management tool

Enclosed space working is one of the known hazardous tasks undertaken onboard ships.
Therefore there is a requirement to protect the human in this environment. A review of an Enclosed Space Management tool was conducted with maintainers to assess both utility and usability of the tool to support vessel maintenance activities (based on a whole system overview) and planning and decision-making (based on a scenario involving a riding gang receiving a report output by the tool). Following the evaluation the system and its component features received high overall ratings.

One issue highlighted was that the content of the tool’s output report was not organised in an optimal format and suggestions were provided on ways to make the important content more obvious and usable. For example, when first looking at the information one of the participants noted that there is too much information and asked, “what am I supposed to do with this?” It was felt that a person would not look through so many details unless there was a specific query ,and there is therefore a high risk that they miss the essence due to the volume of detail. The recommendation was that the summary report should focus on the result highlights and any special precautions required. It was felt that the tool could produce a checklist of these special procedures or precautions, such as measure ’X’, 2 people are required for recovery, etc.

Further recommendations included the consideration of ways to integrate the Enclosed Spaces Management (ESM) tool further with the other on-board systems, namely the relevant existing checklists already available in the Safety Management System and the content in the overall Maintenance Management System.
The results of the evaluation were distributed to the tool’s manufacturer for integration into future development. This supports the goal of providing an even more user-friendly means to reduce a known on-board hazard, as well as raising awareness that such a tool exists. In addition the evaluation provided another example of how to apply HCD methods to conduct a product evaluation and how to identify potential metrics.

1.3.5 Conclusion
Summarised, the CyClaDes project has provided the following major results:
• A HCD framework that contains a comprehensive set of information about HCD and its application in the maritime industry. It comprises rules, examples for good and bad practices, insides into the work environment of seafarers and further reading for a variety of elements on-board ships. The framework is publicly available ( will be enhanced over the following years.
• An E-learning platform which serves as an education environment for different types of stakeholders such as designers, operators and seafarers. It is also publicly available (after a registration, ) and will enable the different stakeholder groups to use HCD in their work, thus promoting the broad use of this methodology in the maritime environment.
• A similar target will be reached by the HCD book which has been written as part of WP 4. While the E-learning platform goes into details and aims at an as comprehensive education as possible, the book provides an overview of the HCD aspects and thus complements the Internet platforms.
• In addition, several results complete the outcome of the CyClaDes projects, namely
o The incident and accident database which provides the rationale for applying HCD
o Several example designs which show the feasibility and value of HCD and also serve as best practice products in the framework
o The analysis of eye tracking and movement sensors as applications for measuring mental workload of ship personnel
o The development of a method suitable for the quantification of the usability of maritime equipment in the context of type approval processes.
o The development of a model to consider the influence of human performance on accidents.
Altogether these results establish a novel toolbox filled with training material, guidance, methods and background material that is able to cover major aspects of HCD in shipbuilding and ship operation.

Potential Impact:
The topic “SST.2012.4.1‐1. Human element factors in shipping safety” is part of the “SAFE AND SEAMLESS MOBILITY” challenge (Challenge 2 of the Transport Programme) which focuses on the optimisation of the global efficiency and safety, making efficient use of infrastructure and network capacity. It aims at offering safe and seamless transport and mobility. CyClaDes specifically supports the safety relevant aspects by orienting the design of ships, including workspaces and control equipment, towards the user, trying to create user interfaces that help to avoid errors, injuries, and reduces stress on the crew. As a side effect, the operation will become more efficient which will help to reduce operating costs as well.

The goal of activity “7.2.4. IMPROVING SAFETY AND SECURITY” as part of Challenge 2 is:

• To develop technologies and intelligent systems to protect vulnerable persons such as drivers, riders, passengers, crew, and pedestrians.
• Advanced engineering systems and risk analysis methodologies will be developed for the design and operation of vehicles, vessels and infrastructures.
• Safety will be considered as an inherent component of the total transport system embracing infrastructures, freight (goods and containers), transport users and operators, vehicles and vessels and measures at policy and legislative levels, including decision support and validation tools; security will be addressed wherever it is an inherent requirement to the transport system.

The results of the CyClaDes project underline the importance to change the perspective in ship design from a technology orientated view towards human centered design. They also point out that different measures will be needed to introduce HCD into the ship design process, e.g. rule collections, training tools and also example designs.
One of the most important findings of CyClaDes was that designers of ships and ship systems are interested in the idea of HCD but many of them are not educated in that domain, existing rules and guidelines are difficult to find and apply. With the current training regime, and without strong-willed on-site encouragement, engineers are unlikely to be in a position to adopt a human-centred approach. Unconvinced engineers are reluctant to spend company money on standards. In CyClaDes, a team of unconvinced engineers gradually became more capable and mature in terms of HCD, especially when they left their offices and started to interact with potential users. Indeed, this experience turned out to be a turning point realizing that discussing design sketches and ideas with end-users appeared to improve the design, to save time and not to be the limit on creativity they feared it would be. CyClaDes also found that most technical ergonomic guidance needed expert interpretation and was not readily usable by (unconvinced) engineers.

Furthermore a significant share of designers does not have practical experience in seafaring and therefore does not fully understand the conditions their products are used under. Therefore the CyClaDes results will have an impact on improved ship design by better educating the different stakeholder groups but also by providing a central source of information about HCD in shipbuilding.
In order to increase the impact, most of the results will be made available to the public, especially the CyClaDes framework and the E-learning lessons. Further measures were the publication of the example designs for a conning display and a winch control panel which show that the HCD approach works and will lead to safer and better to use products.
With the Framework, the E-Learning platform, the NI Book, and with the planned Networking agency CyClades establishes several ways for designers to approach Human-centered design methods in an applicable way. Once the tools are accepted by the industry, this will result in improved ship (system) designs and consequently improve safety and result in fewer accidents.
The lectures (mini courses) developed for universities will help to spark naval architecture student’s interest in HCD methods and will help to spread the methods into design offices.
The processes proposed for the assessment of usability by regulatory bodies will help to increase the pressure on designers to integrate the human element into their designs.

Several papers and conference presentations as well as the IMO presentations are meant to raise the awareness of HCD in the entire maritime domain and hopefully convince ship owners to understand that investment in HCD is investment in safety and will pay off.

As a conclusion it can be stated that the results of CyClaDes – especially the published tools (framework, E-learning, Book) and the courses developed for students – will raise the awareness of the importance on human-centered design of ships and their equipment.
On the long term the consideration of HCD aspects will lead to an improvement of the working and living conditions onboard ships and the overall safety of shipping. Occupational accidents and illnesses as well as the frequency of human error will be reduced. As human error is the main cause for maritime accidents, the frequency of accidents will decrease subsequently.

List of Websites:
Project website:

CyClaDes framework:

CyClaDes E-Learning platform:

1.6 List of beneficiaries and contact persons

Name Short name Contact person (email)

World Maritime University WMU Jens-Uwe Schröder-Hinrichs

Lyngso Marine AS LM Erik Styhr Petersen

National Technical University of Athens NTUA Nikolaos P. Ventikos

Fraunhofer Gesellschaft zur Förderung der angewandten Forschung e.V FKIE Eric Holder

Chalmers Tekniska Hoegskola AB CTU Trevor David Dobbins (

SSPA Sweden AB SSPA Lars Markström

BALance Technology Consulting GmbH BAL Stephan Wurst

Bureau Veritas BV Philippe Corrignan (

Process Contracting Ltd PCL Brian Sherwood Jones (

ErgoProject SRL EP Simon Mastrangelo (

The Nautical Institute LBG NI David Patraiko

Instituto Superior Tecnico IST Ângelo Palos Teixeira (

Future Shape GmbH FS Axel Steinhage

Verwandte Informationen

Reported by



Scientific Research
Folgen Sie uns auf: RSS Facebook Twitter YouTube Verwaltet vom Amt für Veröffentlichungen der EU Nach oben