Final Report Summary - RECOBIA (REduction of COgnitive BIAses in Intelligence Analysis) Executive Summary:The RECOBIA project, which stands for Reduction of Cognitive Biases in Intelligence Analysis, is a European project funded in the framework of the FP7 Programme. Its objective is to improve the work of intelligence officers by reducing the negative effect of cognitive biases, through the application of mitigation strategies that have been developed in the framework of the project. After three years, RECOBIA came to a close at the end of January 2015. The findings are now available to interested parties. One of the key results of the project is the identification, the understanding, and the representation of the human dimension in intelligence. The human dimension implies that unconscious automated mechanisms can influence the intelligence officer. Those mechanisms are also known as cognitive biases. They can be defined as psychological distortions produced by the use of rules of thumbs which fasten our cognitive processing of information. Cognitive biases occure unconsciously, involuntarily and can affect every human being - whatever the role in the organisation, years of experience, gender or cultural background.The RECOBIA project produced results in intelligence and in mitigation of cognitive biases that constitue a break-through in the field. The project team deconstructed the activities of intelligence officers and clustered them into Key Intelligence Tasks (KITs). Those KITs describe for the first time what intelligence officers actually do – both from an intelligence as well as from a psychological perspective. The KITs were used to identify the situation and activities in which cognitive biases are likely to impact intelligence officers. Finally, the project team developed mitigation methodologies and strategies, which are easy to implement and will boost the quality of intelligence. The methodologies have been designed so they can be used by individual intelligence analysts or at the organisational level to develop customized mitigation strategies that respect the specific organisational and cultural environment.All results of the project were validated by representatives of the European intelligence community. Indeed, throughout the project, the consortium organised a series of six workshops, which attracted more than 100 participants representing 21 national intelligence agencies and 5 European Institutions. Those workshops were organised to identify the needs and requirements of the intelligence community and to validate the developed solutions. The close cooperation with intelligence professionals ensured that the results of the project are easily applicable and respond to the requirements and constraints of the intelligence community.The intelligence community is not the only domain that could benefit from the findings of RECOBIA. As all humans are affected by cognitive biases, the identified solutions have a wide range of applications in several other areas of analysis, particularly in the sectors of journalism, banking, finance, health and consulting. A training programme could be derived off the findings of the project. In addition, RECOBIA looked at cognitive biases from an individual’s perspective, but more research should be done on social cognitive psychology (the relations between individuals, individuals in organisations etc.). The methodologies developed in RECOBIA would serve as a strong basis for such research.The RECOBIA consortium comprised of SMEs (CEIS, Hawk Associates, Zanasi & Partners), universities (University of Konstanz, Technological University of Graz), large companies and RTOs (CEA, Thales, ATOS), worked closely with representatives of intelligence services from across the European Union. The project was coordinated by CEIS – Compagnie Européenne d’Intelligence Stratégique. For more information about the project : www.recobia.eu Project Context and Objectives:Cognitive biases are mental short-cuts deriving from our instinctive desire to simplify decision making by reducing the amount of information and uncertainty we have to deal with. These biases typically operate on the unconscious level and result in cognitive simplifications based on memory, experience, education, cultural background, political or ideological beliefs. In addition, heuristics, also known as rules of thumb, also help us to simplify the decision-making process. Heurisitcs make us apply general rules that are derived from past experiences on the question at hand. However, what is helpful to keep us sane and limit the amount of information we have to deal with in our daily lifes might also have negative effects, particularly in intelligence. How can an intelligence officer deliver a reliable report to the political decision-maker when he or she is not aware of the confirmation bias, an effect that makes the analyst disregard all information that contradicts one’s pre-existing opinion on the subject? The answer is that the intelligence analyst can’t. Thanks to Mr. Heuer and a few others, the intelligence community - as well as governments - realised that cognitive biases play a crucial role in the work of intelligence officers.In this spirit, the European Commission co-funded the RECOBIA project. The objective of the project was to improve the quality of intelligence by reducing the impact of cognitive biases through the development of mitigation strategies. Therefor the partners developed the following research plan:• Examination of the activities of intelligence officers• Assessment of cognitive biases• Identification of the intelligence activities that are susceptible to cognitive biases• Development of mitigation strategies.Since all humans are subjected to cognitive biases, the RECOBIA project focused not only on intelligence analysts, but intelligence officers in general. Thus the findings of RECOBIA do not apply only to intelligence analysts but to all employees of intelligence services.One of the key success of the RECOBIA project was to raise awareness on this aspects. The research conducted during the three years of the project always kept this principle in mind.Project Results:During the three years of the project, partners conducted extensive research. This work was regularly presented to end-users in the framework of workshops. It was the occasion for the intelligence community to validate the research and the results of the RECOBIA project. As a consequence, RECOBIA is now able to propose methodologies and solutions that correspond to the intelligence community’s needs and requirements, and that are truly operational.The main results of the RECOBIA project can be listed as follows:• Identification of cognitive biases relevant to intelligence• The worldwide first methodology to classify cognitive biases• Mapping of the activities of the intelligence officer• Identification the activities that are unique from an intelligence and a psychological point of view, the Key Intelligence Tasks (KITs)• Identification of cognitive biases that intelligence officers might be subjected to when performing their daily job • Description of where, when and how cognitive biases strike• Development of mitigation principles to reduce the negative aspects of the human factor in intelligence• Application of these principles and development of mitigation strategies to the most relevant cognitive biases intelligence officers might be subjected to.These results are extensively developed in the 38 deliverables that were prepared in the course of the project. They are available to the end-users on the project website (www.recobia.eu) along with an extensive library of publications and literature on the field of intelligence and cognitive biases. Finally, the results are also featured on a web-based plateform called RECOPEDIA, accessible to end-users. RECOPEDIA offers users a triple entry point: through intelligence, through psychology and through solutions. Some of the key results of the RECOBIA project are briefly presented below:1. The worldwide first methodology to classify cognitive biasesThrough literature review the project team compiled a list of 288 cognitive biases currently known. Those were then classified and clustered to a manageable number. Two methodologies were used to achieve this: clustering of cognitive biases by using a Formal Concept Analysis (FCA) and the experimental design approach (EDA).The FCA is a formal framework describing concepts and concept hierarchies in mathematical terms. In addition, applying the FCA leads to answering the following questions: are there overlaps between some of these heuristics and biases? Are some of theme synonyms or “conceptually the same”? Which pairs / triples / etc. of heuristics and biases are “similar to each other”, which are not? The result of this FCA was to identify 31 biases (out of initially 288) that are conceptually unique.The EDA on the other hand, is method that involves the manipulation of variables in order to discover causes and effects. By applying this methodology, cognitive biases were described in the same comparable manner: what does the subject seeks to do -the cognitive goal - and the context in which the task being done – the cognitive situation. This definition allowed to identify categories of cognitive biases and thus to cluster them.2. Activities of the intelligence officerA taxonomy of over 1,000 intelligence-related activities was elaborated. This taxonomy, while not claiming to be exhaustive, provides a detailed overview of the many activities an average intelligence professional is likely to undertake in the course of a day. Research showed that intelligence is far more complex than is traditionally assumed. The intelligence cycle offers a valuable framework for study, but it fails to capture the diversity of activities - functional, cognitive or otherwise - that intelligence professionals must undertake in the course of their work. Nor does it reflect the many contextual factors that must be managed to ensure the timely production and deliver of intelligence products.The over 1000 activities performed by intelligence officers were classified in five domains: organisational (tasks pertaining to the wider mission and objectives of the intelligence organisation); operational (tasks pertaining to the execution of an intelligence project); informational (tasks pertaining to the management and prioritisation of one’s information needs); technological (tasks pertaining to the adoption and use of IT in support of intelligence work); cognitive (tasks specific to the mental activities informing intelligence work, such as problem finding, analysis etc.).3. Identification the activities that are unique from an intelligence and a psychological point of view, the Key Intelligence Tasks (KITs)Evaluating 1,000 different tasks would have been a considerable undertaking. Thus, the decision was taken to select and cluster those activities the partners considered most susceptible to cognitive biases using the following criteria:• Those activities deemed essential for the completion of a given intelligence product• Those activities that could be considered representative of a broader set of intelligence activities• Those activities that likely to be affected by cognitive biases according to existing literature on intelligence workThis three-pronged approach allowed the partners to identify the 25 categories of intelligence activities. As it was necessary to focus on those activities most likely to be affected by cognitive bias, it was essential that the final grouping remained coherent from an intelligence perspective. Moreover, the grouping meant to reflect the day-to-day work of intelligence professionals and be labelled in an understandable way for end users. This approach led to the identification of seven Key Intelligence Tasks (KITs). Each KIT can be described as a minimum and non-redundant set of tasks that must be carried out by an intelligence officer in order to complete a given intelligence task (regardless of its nature, provenance, etc.).The identification of seven KITs is one of the groundbreaking result of the RECOBIA project. For the first time, the activities of an intelligence officer are described from an practical and operational rather that purely theoretical perspective. Moreover, these KITs are relevant both from an intelligence as well as from a psychological perspective.4. Identification of cognitive biases that intelligence officers might be subjected to when performing their daily job & description of where, when and how cognitive biases strikeIn a later stage in the project, the consortium partners developed the RECOBIA scenario. The objective of this scenario was to illustrate relevant cognitive biases in an intelligence context in order to make them more easily understandable. More specifically, this scenario is useful to understand the task and situation in which intelligence officers might be susceptible to cognitive biases. These situations reflect the real life and they represent the various roles intelligence officers can have in an intelligence service, within the context of the seven KITs developed earlier in the project.Keeping these elements in mind, the partners created a fictitious operational environment and a victicious intelligence officer. The scenario simulated a geopolitical situation between two imaginary countries. The fictitious intelligence officer will then perform his daily job within this scenario. The scenario thus helps to identify to which cognitive biases intelligence officers might be subjected to when performing their daily job and most importantly where, when and how they might strike.5. Development of mitigation principlesOnce this illustration work was completed, the research team focused on the mitigation strategies per se. In order to do so, it was necessary to identify the underlying mechanisms triggering cognitive biases. Such mechanisms are to be found by studying adaptive rules - self-preservation, social nature etc. - and biological constraints – the design of the human brain. Once these mechanisms were identified, they allowed for the classification of cognitive biases – those relying on contamination mechanisms, those stemming from self-preservation mechanisms and so forth – and then for the definition of a generic solution to counter these biases. Considering that KITs were also designed according to their cognitive properties it was then possible to come up with a KIT-by-KIT diagnosis of the issue and finally with a KIT-by-KIT generic principle to develop a mitigation strategy. 6. Application of these principles to selected cognitive biases: the development of mitigation strategiesBased on the generic principle of solution for each KIT, different mitigation strategies were created and presented to the end users. The principle behind the mitigation strategies, is to move the process of making a judgement (selection, assessment, evaluation) from gut feeling, intuition or expert judgement to reasoning. At the end of the project, the RECOBIA team offered a panorama of the different potential mitigation strategies and solutions in order to mitigate the negative impact of cognitive biases. Three types of mitigation strategies were identified: structured analytic techniques; IT tools and psychological mitigation strategies. These strategies focus on the most relevant cognitive biases for the intelligence officers.Potential Impact:RECOBIA’s value is both immediate and apparent. The project has resulted in a significant advance over the current “state of the art”, as well as it has provided tangible benefits to the organisations and individuals that have followed the project and who are engaged in intelligence or related work.The proposed solutions will contribute to improving the practice of intelligence, the quality of its outputs, and the communication of risks and opportunities to decision makers across the EU. In addition the project’s research and results can serve to conduct additional research in other fields. They can also be developed in training programmes for the identified communities of end-users.• The impact of the RECOBIA project on the intelligence analysis and the intelligence communityEnd-users highlighted the usefulness of the project. In particular, RECOBIA greatly contributed to raising awareness on the working of the human mind, especially on unconscious intellectual mechanisms occurring within the human brain and potentially impacting the quality of intelligence work. End-users also underlined that the research within RECOBIA could have a positive impact on decisionmaking processes and could thus contribute to the improvement of intelligence work. Another key aspect is that end-users have learned that although it was impossible to eliminate cognitive biases, their potential negative impact could nonetheless be mitigated.End-users pinpointed the following additional beneficial outcomes of the RECOBIA project:- A better understanding of unconscious intellectual mechanisms pertaining to intelligence analysis- The discovery of methodologies and mitigation strategies and of their underlying functioning principles- A reflection on intelligence work allowing for its improvement- Readily available results on a single web-based tool, namely RECOPEDIAEnd-users also highlighted the usefulness of the RECOPEDIA website as a tool enabling to raise awareness on cognitive biases and to understand in which way they actually impacted intelligence work thanks to the development of the KITs, the specific situations and the RECOBIA scenario. Another key aspect is the sharing of best practices and of methodologies among the organisations represented at the six RECOBIA workshops. A vast majority of end-users stressed the importance of the networking/exchanges opportunities provided by the three-year project, which resulted in the fostering and enlargement of a European community of intelligence officers interested in human factors in intelligence. Having such a broad representation also allowed the consortium to find out that European intelligence agencies actually faced similar challenges in the field of intelligence analysis. • A strong involvement of the end-user communityThroughout the project, the RECOBIA partners aimed at developing research that would be useful to the end-users. To that extent, six workshops were organised in the three years of the project. These workshops aimed at presenting the results of the projects, and gathering the inputs of the end-users on the progress of work.What these workshops also contributed to is to foster a community of end-users working in intelligence agencies and European institutions across Europe. The tables below present a few figures on these workshops:- 6 workshops organised in Brussels and Madrid- Between 15 and 21 participants at each workshop- 102 participants in total- 21 national agencies ; 4 EU agencies ; 11 Member States- Organisations represented: Ministries of Defence, Intelligence Agencies, Navy Intelligence, Intelligence Academy, Customs, Prime Minister’s Office, Military Intelligence, Ministry of Finance, Ministry of Interior, Ministry of Foreign Affairs, European External Action Service, European Union Military Staff, Europol, Frontex, European Commission.- Member States represented: Belgium, France, Italy, Netherlands, Finland, Hungary, Lithuania, Poland, Romania, Sweden, United Kingdom.• A wide dissemination of the RECOBIA’s results throughout the projectThe partners of RECOBIA were commited throughout the project to disseminate the results and promote the project to a wider audience. Therefore numerous dissemination activities were set up, and several publications were prepared by the partners. The website (www.recobia.eu) was key in the dissemination strategy. It provided not only generic information about the project, its objective and partners, but it was regularly updated with news, publications and event on the topics of cognitive biases and intelligence. A closed section of the website, available upon registration to the end-users also provided access to all deliverables and minutes of the workshops.The RECOBIA project’s news were also regularly disseminated through social media, mainly on Twitter and LinkedIn. This social media awareness campaign enabled the partners to reach out to a wider community, including the industry, civil society, the media, and the scientific community.The project was presented at several exhibitions and conference where its visibility was increased by a dedicated booth, project roll-up and flyers. Among the attended events were the Security Research Conference and CP Expo organised by the European Commission in Genova in 2014; the EEAS High Level conference “a global network of crisis rooms”; the Security and Defence Day conference organised in Brussels; and the DECISIVe conference, attracting a scientific community around the topics of cognitive biases and visualisation tools.Finally, the project’s final conference was organised in January in Brussels. It attracted both end-users and journalists. The main findings of the project were presented and discussions on the “after” RECOBIA took place. This event was publicised in multiple articles across the European Union.• The impact of the RECOBIA project on the consortium partnersThe RECOBIA project had a clear positive impact on its community of end-users, and it contributed to enriching the research in the field of intelligence and cognitive biases. The partners of the project themselves also largely benefited from their involvement in RECOBIA. Indeed, the RECOBIA project contributed to facilitating the collaboration between partners from different desciplines and between different types of organisations. The close cooperation of the industry, the scientific community and SMEs will contribute to strengthening a wide research community at the European level. European projects enable organisations that are not naturally drawn together, but work on similar topics of interests to work together and learn from each others expertise. This is clearly an added value of such European funded projects.At the partner level, the RECOBIA project provided a framework for the organisations to deepen their knowledge and expertise on the topic of intelligence and cognitive biases. This was also reinforced by the continuous exchanges with the community of end-users.• Going beyond the RECOBIA project: the exploitation of resultsIn terms of exploitation of the research and results of the project, end-users emphasized two areas of applications: training of analysts and review of procedures/methodologies. The intelligence community is not the only domain that could benefit from the findings of RECOBIA. As all humans are affected by cognitive biases, the identified solutions have a wide range of applications in several other areas of analysis, particularly in the sectors of journalism, banking, finance, health and consulting. In addition, RECOBIA looked at cognitive biases from an individual’s perspective, but more research should be done on social cognitive psychology (the relations between individuals, individuals in organisations etc.). The methodologies developed in RECOBIA would serve as a strong basis for such research.All in all, as many end-users pointed out, that there is room for further research and follow-up projects in the area of intelligence analysis.List of Websites:www.recobia.eu
