Mapping Chronic Non-Communicable Diseases Research Activities and their Impact

Final Report Summary - MAPPING_NCD (Mapping Chronic Non-Communicable Diseases Research Activities and their Impact)

Executive Summary:
MAPPING_NCD maps European Union (EU) Member State (MS), supranational and private research activity, investment and initiatives, quantifying their impact in five key disease areas: cardiovascular disease (CVD), chronic respiratory disease (CRD), diabetes, cancer and mental health. The project has six objectives:
1. Identify current EU-funded, as well as national, regional, charitable and private research programs and initiatives in the field of chronic non-communicable diseases;
2. Map the scale and scope of research activities in this area, including the research fields addressed and their implementation modalities;
3. Provide adequate comparison of data and results through the use of common definition criteria and methodology;
4. Analyze their funding sources and map the impact of overall investments and outputs to date;
5. Identify potential overlaps, synergies, gaps and opportunities for future collaboration across countries and organisations; and
6. Contribute to the development of evidence-based policies towards supporting coordinated approaches in chronic NCD research.

MAPPING_NCD moves beyond the state of the art in the research area in three ways: (i) by providing accurate mapping of research funding activities at EU and MS level, (ii) by pursuing a bibliometric mapping and analysis of the volume of research outputs in the EU and MS relevant to NCD; (iii) by using input from both the above to provide an analysis of potential overlaps, synergies, gaps, opportunities, overall impact of NCD research funding and help identify the research agenda for the future. In doing so, MAPPING_NCD deploys primary and secondary data collection methods.

The general research strategy for MAPPING_NCDs involves the conduct of an Impact Assessment (IA), a Critical Appraisal (CA) and a Disease Level Synthesis (DLS) across the five disease categories. The IA reports the results of the Survey Tool which collects data on RFO research activity, investment and funding flows for NCDs across the European area. Importantly, the IA does not capture all research funding activity in the relevant disease area. The CA consolidates data from the literature review and emerging data from the bibliometric analysis. The CA adds to the richness of the five disease IAs and generally produces a more robust and comprehensive document. The DLS combines all primary and secondary data collected in the IA and the CA into a single report addressing the project objectives for each NCD category. DLS produces a broadly comparable discussion of the data and results in each of the five disease areas.

Project Context and Objectives:
Mapping NCD maps funding and research impact Non-Communicable Disease (NCD) R&D in Europe with a view to improving the returns on investment in NCD research. In parti¬cular, updated mapping information on the impact of NCD initiatives and re¬search has the potential to identify over¬laps, synergies, gaps and also opportunities for policy learning, exchange and collaboration across the EU. The key objectives of the project were to: 1. Identify current EU-funded, as well as national, regional and charitab¬le research programs and initiatives in the field of chronic non-communicable diseases; 2. Map the scale and scope of research activities in this area, including the research fields addressed and their implementation modalities; 3. Provide adequate comparison of data and re¬sults through the use of common defi¬nition criteria and methodology; 4. Ana¬lyze their funding sources and map the impact of overall investments and out¬puts to date; 5. Identify potential over¬laps, synergies, gaps and opportunities for future collaboration and finally 6. Contribute to the development of evidence-based policies towards supporting coordina¬ted approaches in chronic non-commu¬nicable diseases research.
WP1 Methodology
MAPPING_NCD collected primary and secondary data via qualitative and quantitative research techniques. Primary data involved the conduct of surveys with key governmental and non-governmental funding organizations, together with key clinical and scientific researchers for the purpose of identifying outputs, financial resources committed, overlaps, synergies, gaps and opportunities for collaboration regarding NCD research activities, outputs and implementation. The collection of secondary data involved the review of reports, published research from academic journals and grey material derived from websites. Primary bibliometric data was involved in the analysis of published academic research on NCDs for the purpose of deriving data regarding the funding sources, overall invest¬ments and output of NCD research activities. Papers identified for bibliometric analysis were downloaded and classified according to the International Disease Classification (ICD-10).

The objective of the Work Package (WP1) was to produce methods for the collection and analysis of data through the use of common definition criteria and research methodology. Specifically, it provided methods for the conduct of a systematic survey of NCD research initiatives and activities at MS, Regional and EU level. It outlined systematic methods for assessing the characteristics and collecting information from policy makers and research funding organizations (RFOs) regarding NCD research, research funding and its policy and clinical impact. It also provided methods for the creation of suitable bibliometric filters and sub-filters for the purpose of identifying papers (2002-13) within the Web of Science relevant to each of the disease areas listed by the WHO.
WP2
CVDs are diseases, which involve the heart and/or the blood vessels, and involve a number of conditions outlined under the ICD-10 codes I00 to I99. CVDs are the leading cause of deaths globally, with a rising gap between developed and developing countries. In most cases, CVD are progressive and not reversible, although fatal CVD events can be prevented through lifestyle changes, drug therapy or surgery. CVDs are the main cause of death in the European Union, causing a significant higher mortality in women (42.7%) than in men (35.7 %). Of all CVDs, coronary heart diseases (CHDs) are the most common group for both genders, followed by other cardiovascular diseases and cerebrovascular diseases, which drive the differences in gender prevalence for total CVD burden. There is hardly any European country that does not fund CVD research at some level. Our map identified active 132 RFOs in the disease area. Supplementing survey responses with website queries, we achieved satisfactory levels of data from 124 RFOs. Like other disease areas, less than 10% of RFOs funded CVDs exclusively (n=13). Most were active in other disease areas, which complicated efforts to ascertain the proportion of resources allocated to CVDs. Even where these multi-dimensional RFOs were cooperative, they did not keep separate records for each disease area and were only able to estimate the proportion of resources they allocated to CVDs. Overall, the results indicate that the majority of RFOs are not exclusively devoted to CVD research. They invested their funds in different research fields, as for example in other NCDs or non-medical fields. Given this diversity it was difficult to separate the funds solely allocated to CVDs-research, so that most of the RFOs could only estimate the amount of funding they had spent for CVDs research along the provided survey structure. In a few countries, however, we identified RFOs dedicated solely to CVD research – particularly in Northern Europe (foremost UK and Ireland) where the funding systems revealed to be significantly different from other areas, especially the Eastern European Area, where the research system is very centralized and funding investments belong to ministries or national science centres.
WP3
Chronic Respiratory Diseases (CRDs) are a group of illnesses that produce respiratory abnormalities under which breathing becomes slowed or forced. Although serious and often life threatening, most CRDs are largely preventable through simple clinical strategies that are both inexpensive and easy to administer. CRDs include major conditions like: chronic obstructive pulmonary disease (COPD), respiratory allergies, pulmonary hypertension, chronic bronchitis, asthma, emphysema and occupational lung diseases. According to the Global Burden of Disease Study (GBD), CRDs are the fifth leading cause of lost Disability Adjusted Life Years (DALYs) across Europe in terms of the major non-communicable disease (NCDs) categories. Breaking down the CRD category into its major diseases, COPD and asthma are the first and second largest cause of lost DALYs across the European area. There are at least 131 RFOs for CRDs across the European area. Many of these RFOs are active funders of research in other NCD areas (n=119). At the top end of the scale, 38 RFOs have an annual research spend of over € 1.0 million. At the lower end, 13 RFOs have an annual spend below the € 0.1 million, the majority of which were from smaller European countries. On average, the CRD research projects that these RFOs fund have a life span of about 28 months. The projects generally focus on therapeutic and drug development in relation to the major diseases categories within CRDs. Most currently funded research focuses on asthma (32%) with COPD (24%) and cystic fibrosis (14%) following behind. Typically, RFOs fund research conducted at universities or university research centers. They do not usually fund private sector organizations, or even large consortiums of public sector research centers. In fact, the European Commission (EC) is perhaps the only RFO that sponsors large cross-border collaborations.
WP4
DIAB is a chronic disease whose prevalence is about 9% in the adult population worldwide. This condition accounts for 11% of all deaths in the age 20-79 years in the EU. Unlike other NCDs, the trend has increased over the last decades and projections show it will be even more burdensome in the future, due to population-ageing. Across the EU, there are a total of 120 RFOs investing in DIAB research. A total of 14 diabetes-specific RFOs were then identified. The majority were Government/Public organizations (58%) or private-not-for-profit (PNP) organizations (40%). Overall, more than two thirds of the RFOs operate at national or sub-national level and only 12% has an international reach. Across the EU, DIAB-relevant RFOs also display major divergences. Eastern Europe tends to have a centralized funding systems provided by public institutions unlike Central Europe, where the majority of RFOs are PNP. From 2003-2013, annual spend for DIAB research funding experienced an overall steady increase, with an average annual growth rate of 5% and a sharp increase between 2008 and 2009 of 41%. The overall annual funding tracked for the last three years in our historical series was about €140million. When the trend is broken down into geographical areas, the Southern European MSs seem to keep their annual levels of spend almost static. The highest amount of funding for DIAB research seems to come from the UK and the Netherlands.
WP5
“Cancer” is a generic term for a large group of diseases that can affect any organs or tissues of the body (e.g. breast, lung, skin or bone marrow). There are more than 100 types of cancer, which are usually named for the organs or tissues where the cancers form. Other terms used are malignant tumours or malignant neoplasms. One defining feature of cancer is the rapid creation of abnormal cells that grow beyond their usual boundaries, which can invade surrounding tissue and can metastasize to distant sites. Cancer is the leading cause of death worldwide, but there are marked geographical differences, particularly when considering specific tumour types rather than cancer as a whole. There are a total of 169 Research Funding Organizations (RFOs) investing in cancer research. Some of them RFOs (n=78) are devoted exclusively to cancer research, whereas the overwhelming majority make research investments in other NCD disease areas. The study identified 87 RFOs with an annual spend of above € 1.0 million. It also identified 14 RFOs with an annual spend below the € 0.1 million threshold. In Europe, the funding figures are substantial, with over a billion euro invested in the disease area per annum (from the year 2008 onward). Over the study period (2002-2013), the total amount of funding reaches over €11 billion for the UE-28. When considering Europe overall (31 countries), the total spending allocated to cancer research is over €14 billion. Over the same period, Central Europe´s contribution to the overall investment in cancer research is considerable, representing € 6.6 billion. Generally, most identified RFOs were not ‘Cancer-specific’ (n=91). Instead, they funded research under the broader category of NCDs, and were not themselves concerned even to distinguish between disease categories. Government was the main source of funding for 89 RFOs (43 from Central Europe, 8 from Northern Europe, 27 from Southern Europe and 11 from Eastern Europe), whereas the contribution of the private sector, fundraising activities, donors and other sources, is minor, as it is mainly present in Southern (n=14) and Northern Europe (n=12). Cancer research receives funding from the public sector mainly in Eastern (100% of total funding) and Southern Europe (55% of total funding), whereas in Central and Northern Europe there are other sources of cancer research funding (e.g. fundraising campaigns, contributions by members, and others) contributing to the overall investment of cancer research.
WP6
Mental health disorders (MHDs) encompass a broad range of conditions, including depression, anxiety, bipolar disorder, schizophrenia, alcohol and substance disorders, ADHD, suicide and self-harm, eating disorders and Alzheimer’s disease, which together constitute a major share of the disease burden in Europe. Compared to other major NCDs, MHDs are difficult to diagnose and manage due in part to a lack of diagnostic and risk stratification tools. There are at least 137 MHD RFOs in 20 countries, most of which also fund research in other NCD disease areas. The majority of the MHD RFOs was public and received government funding. The level of involvement of charitable organizations in MHD research is low compared to other NCDs, such as cancer. Analysis of the top-funded MHD projects identified in the five largest European Union (EU) Member States (MSs) showed that Alzheimer’s disease and schizophrenia received the bulk of the funding, followed by depression and to a much lesser extent anxiety. Considered in light of the disease burden for these four MHDs, depression and anxiety research received a much lower share of project funding than their burden would suggest. Nonetheless, prevalence should not be the only metric considered in setting research priorities. Other factors, including treatment gaps, cost of illness and potential return on investment, should be part of the decision process.
WP7
Bibliometric analysis establishes establish impact and outputs in terms of number of papers published in the Web of Science (WoS) relative to research priorities and levels of investment. We also determined the numbers of papers world-wide for the five NCDs, and the numbers of papers in biomedical research overall, both in the EUR31 countries and world-wide. The EUR31 countries as a group publish some 40% of biomedical research, and four of the NCDs, but as much as 56% in RESPI. The five NCDs represent very different amounts of EUR31 biomedical research: ONCOL 11.5%, CARDI 8.7%, MENTH 5.7%, but DIABE just 1.7% and RESPI only 0.8%. In comparison with the European disease burden, cancer appears over-researched relative to the other NCDs. For each NCD, we compared each European country's output of research with its wealth, as measured by GDP, as there is normally a much better correlation than with population. It appears that the Scandinavian countries (other than Norway (NO)) are performing rather well, with both Finland (FI) and Sweden (SE) mentioned four times, as is the Netherlands (NL). The UK is mentioned three times, and Greece (GR) and Croatia (HR) twice each. On the other hand, France (FR) is mentioned four times in the low category and Norway, a very rich country, together with Romania (RO), a poor country, score low in three NCDs. The mean figure of the overall average cost of a paper was close to that of the middle-income countries, at €255 k. This gives estimates for the total R&D spend on the five NCDs in the EUR31 countries as: CARDI €4594 M, DIABE €885 M, MENTH €3280 M, ONCOL €6163 M and RESPI €419 M. Papers in DIABE, MENTH and ONCOL were only better cited than the world average in the latter years, and papers in MENTH in the SSCI were slightly less well cited than the world average because Chinese and other east Asian papers are seriously under-represented in this index, so this average is dominated by US papers.
Project Results:
WP1
The objective of the Work Package (WP1) was to produce methods for the collection and analysis of data through the use of common definition criteria and research methodology. Specifically, it provided methods for the conduct of a systematic survey of NCD research initiatives and activities at MS, Regional and EU level. It outlined systematic methods for assessing the characteristics and collecting information from policy makers and research funding organizations (RFOs) regarding NCD research, research funding and its policy and clinical impact. It also provided methods for the creation of suitable bibliometric filters and sub-filters for the purpose of identifying papers (2002-13) within the Web of Science relevant to each of the disease areas listed by the WHO.

WP1 established methods for four key research tasks, or project milestones across each of the five therapeutic category areas:
• Survey Tool Administration and Data Collection
• Impact Assessment based on Data Collection
• Critical Appraisal
• Bibliometrics
• Synthesis across Disease Areas

WP1 outlined methods for the conduct of a Qualitative Survey designed to provide accurate mapping of NCD research funding activities across the European Union in the five key disease areas. The survey gathered information regarding interventions, NCD-related research and research on clinical pathways, products, programmes and devices. In order to achieve this, the Survey Tool was be constructed, validated and subsequently disseminated in order to collect the information required. In practice, the Survey Tool consisted of five elements:
• Defining NCD Categories
• Drafting and Validating the Questionnaire
• Identifying Research Funding Organizations (RFOs)
• Drafting the Introductory Letter
• Distributing the Questionnaire

WP1 outlined precise definitions for the specific disease conditions within each NCD area. The definitions were necessary for both the bibliometric component of the project and the wider qualitative survey. In adopting definitions for the relevant conditions, WP1 used the International Classification of Diseases (ICD) – 10th Edition, which is the standard diagnostic tool for epidemiology, health management and clinical purposes. For the purposes of the bibliometric study, the five disease filters were given pentagraph codes, which were as follows: cancer = ONCOL, cardiovascular = CARDI, diabetes = DIABE, mental disorders = MENTH and respiratory disease = RESPI. For the purposes of the wider project, the definitions and relevant disease conditions for the five NCD areas are:

Identification of Research Funding Organizations (RFOs) and Conduct of the Survey
The Survey Tool targeted European area RFOs with overall investments in each disease category above a baseline threshold of between 0.5 to 1.0 million Euro. The adjustable threshold allowed consortium members to raise or lower the baseline within specific disease areas to reflect the wider levels of investment. For RFOs in countries where the level of investments in specific disease areas was not immediately available, WP1 established that research partners may opt to apply the same threshold on the overall investment made by the RFO. WP1 established that these organizations included key EU level, national and supranational funding bodies (e.g. national research councils and foundations, EU-funded research through DG Research and other related DGs, such as Sanco, Enterprise), public-private partnerships operating at national or supra-national level (e.g. the Innovative Medicines Initiative - IMI), and national and supra-national NGOs. For territorial areas in which RFOs are not immediately visible or exist in a fragmented form, researchers will proceed along the lines of ‘snowball sampling’, gradually building a list of relevant RFOs.

WP1 outlined methods by which questionnaires were sent to participating RFOs with a view to obtaining self-reported data on the resources spent on research and their intended purposes. Consortium members collected data on all NCD categories within their allocated MS. Having collected the data, consortium members distributed the data to the leaders of the relevant Work Package. WP1 established that data collection proceeded on a territorially specific basis; and data analysis proceeded on a disease specific basis,

Impact Assessment
WP1 outlined methods for the conduct of an Impact Assessment designed to measure the impact of health programs and research activities. This element of the project identified how RFOs determined impact and in what ways sponsors, policymakers and researchers distinguish between levels of impact. With the survey having outlined the precise content of the major EU and MS initiatives in the disease area, the impact assessment considered the impact of investments and research initiatives from a clinical and policy practice in different MS settings. In practice, the impact assessment consisted of three tasks:
• Literature Review;
• Collection of Clinical Guidelines; and the
• Collection of Media Stories.

The impact assessment reviewed of the impact of research initiatives on clinical approaches and guidelines, policy development and the media in different MSs and regions. It ascertained whether the identified research programmes have had any impact in informing policy recommendations and/or clinical guidelines within the disease, with a review of the sample initiatives determining how these impacts have contributed to developments at the clinical and policy development level. Follow-up contacts and communications with policy makers, stake holders and researcher interviews were conducted to obtain further insights relating to issues not fully captured in the questionnaire, such as political, demographic and contexts and wider public health policy

Critical Appraisal
WP1 outlined that the Critical Appraisal consisted of four self-contained pieces of evidence:
• Summary of European research projects
• Description of Private sector research pipelines
• Stakeholder interviews
• Bibliometric Analysis of Research Funding Impact

In conducting the Critical Appraisal, WP1 established that consortium members should: 1/ discuss and consider the portents of each evidence base individually. 2/ discuss and consider the portents of each evidence base collectively. In the synthesis component of the project, the four evidence bases outlined in the Critical Appraisal will be consolidated with two evidence bases outlined in the Impact Assessment (literature review and RFO survey)

European research Projects
WP1 established requirements for a “Purposive Sample” of research programs/ projects that the largest RFOs are funding in the relevant disease area. It was neither feasible nor desirable to provide a comprehensive sample of funded research projects in the disease area across EU31. Alternatively, WP1 specified that a “Purposive Sample” provided policy makers with a general description of the types of projects in the relevant disease area for which RFOs across the EU provide funding.

Private Sector Pipelines
Investment in research and development is a key industry activity across the pharmaceutical and biotechnology sector. Throughout the 20th century, the pharmaceutical industry has played a significant and indispensable role in the advancement of science through the production and publication of high quality research. WP1 specified methods by which the Critical Appraisal described the research pipeline for private sector pharmaceutical technologies and medical devices in the area of NCDS.

Interviews with Key Stakeholders
WP1 produced methods for stakeholder interviews. After receiving the completed questionnaire, consortium members elicited the views of senior personnel within RFOs on impact, policy and broader research funding issues regarding NCD treatment. Interviews with Stakeholders will be undertaken on a disease level basis. Researchers made contact with RFOs to discuss these issues with key individuals. These contacts took place by phone or internet and will follow up on issues raised in the questionnaire utilizing a Strengths, Weaknesses, Opportunities and Threats (SWOT) analysis. Researchers will create a representative sample of RFOs interviewees per country to obtain information about impact and priorities for future research. WP1 specified that the sample included a selection of RFOs from each of the listed categories, but with the precise number of contacts being left to the judgment of consortium members.

Bibliometrics Impact of CRD Research Funding
WP established methods for the collection and sue of bibliometric data. Measuring the impact of research is a complex task. Often, health improvements depend on a host of different research discoveries, which are made at different times and in different places. The pathway from the conduct and publication of research to better health is usually indirect. In addition, the results of research contribute to better health in different ways, from the improved diagnosis and treatment of patients to the prevention of illness or the reduction incidence. Among these many nodes and linkages, ‘government policy’ occupies a central position and has a several linkages to other nodes. Moreover, the ‘reduction of illness incidence’ also depends on a large number of inputs, including: environmental pollution, individual health behaviours, wealth, education and the effectiveness of public health campaigns. Thus, WP1 observed that research impacts can be observed at all these nodes, many of which are not specific to individual disease areas. Similarly, different types of research can also deliver advances in individual disease areas. And for these reasons, the norms for measuring both the effectiveness of research and its quality can also differ.

Synthesis
WP1 established methods for the Synthesis component of Mapping NCDs which combined the evidence produced in the Impact Assessment and the Critical Appraisal a consolidated disease focused report for the purpose of delivering an adequately comparable discussion of the data and communication of the results across the five disease areas. Synthesis of the results consolidated all collected material in order to establish recommendations for future investments and research initiatives regarding NCDs. It included an analysis of research investments, research activities, policy developments, outputs and outcomes in terms of whether or not research investments and activities have achieved expected and desirable outcomes. Assessing the combined evidence bases, it specified that researchers would establish where existing investments had succeeded and failed, where deficiencies remained within the disease area and suggest priorities for the future.
Research Strategy
In particular, WP1 established how the Synthesis would identify research funding gaps, opportunities for both collaboration and synergies among different actors, stakeholders and entities with a view to improving the understanding of the different NCDs, their treatment and the overall policies that impact individual health but also health systems. The Synthesis considered the impact of major research initiatives within the disease on policy programmes, clinical approaches, criteria and processes and the media across MS. It ascertained whether the selected research programmes have had any impact in informing policy recommendations and/or clinical guidelines within the disease. It commented upon how each of the impact categories previously identified have contributed to actual impacts at the clinical and policy development level.
WP2
CVDs are diseases, which involve the heart and/or the blood vessels, and involve a number of conditions outlined under the ICD-10 codes I00 to I99. CVDs are the leading cause of deaths globally, with a rising gap between developed and developing countries. In most cases, CVD are progressive and not reversible, although fatal CVD events can be prevented through lifestyle changes, drug therapy or surgery.

CVDs are the main cause of death in the European Union, causing a significant higher mortality in women (42.7%) than in men (35.7 %). Of all CVDs, coronary heart diseases (CHDs) are the most common group for both genders, followed by other cardiovascular diseases and cerebrovascular diseases, which drive the differences in gender prevalence for total CVD burden. There is also a wide variation of CVD mortality rates across EU member states. In 2010, eight countries were able to push CVD below cancer mortality: Belgium, Denmark, France, Luxembourg, Netherlands, Portugal, Spain and Slovenia (Nichols et al., 2014). This is contrasted by the high CVD mortality in Eastern European member states: Bulgaria and Romania reported a three times higher, the Czech Republic, Estonia, Hungary and others had a twofold higher CVD than cancer related mortality in men. CVD mortality in women is still higher than cancer across all EU member states, although in the Netherlands and Denmark the gap to cancer mortality has been almost closed in 2010.

There is hardly any European country that does not fund CVD research at some level. Our map identified active 132 RFOs in the disease area. Supplementing survey responses with website queries, we achieved satisfactory levels of data from 124 RFOs. Like other disease areas, less than 10% of RFOs funded CVDs exclusively (n=13). Most were active in other disease areas, which complicated efforts to ascertain the proportion of resources allocated to CVDs. Even where these multi-dimensional RFOs were cooperative, they did not keep separate records for each disease area and were only able to estimate the proportion of resources they allocated to CVDs.

Overall, the results indicate that the majority of RFOs are not exclusively devoted to CVD research. They invested their funds in different research fields, as for example in other NCDs or non-medical fields. Given this diversity it was difficult to separate the funds solely allocated to CVDs-research, so that most of the RFOs could only estimate the amount of funding they had spent for CVDs research along the provided survey structure. In a few countries, however, we identified RFOs dedicated solely to CVD research – particularly in Northern Europe (foremost UK and Ireland) where the funding systems revealed to be significantly different from other areas, especially the Eastern European Area, where the research system is very centralized and funding investments belong to ministries or national science centres. RFOs were grouped into four categories: Government/Public, Private-non-profit, International and Commercial. The large majority of RFOs turned out to be Government/Public organisations (66%) followed by Private-non-profit organisations (31%). No organisations with a clear commercial focus could be identified in either the surveys or the complementary online research.

The UK is by far the largest funder of cardiovascular research in Europe with almost 1 billion EUR spent within the last years, when correlating level of spending to levels of GDP. Switzerland and Netherlands are second and third position in terms of money invested in CVD research. Germany or France, have a higher GDP, but spent less money on CVD research in absolute terms. Looking at the total funding investment by European area, Central Europe has the highest amount of funding to CVD research for the period 2007-2013. Spain and Italy spend a comparatively low amount, which causes a very low overall funding in the Southern European area. From Eastern MSs, Poland and Estonia are among the top-10 funders in Europe. Unfortunately, survey data on other Eastern European member states were scarce, thus this region may be underrepresented in the analysis. Other results have been influenced by a lack of available data. Moreover, the number of RFOs focusing on CVDs is higher in Central Europe than in Northern or Eastern Europe, and the number of RFOs that supplied quantitative data is substantially higher.

The EC is the most important source of funding to CVD research on a European level, spending € 413 million on CVD research between 2006- 2013. By far the most known funding scheme for medical research investment is run by DG Research and Innovation and managed in framework programmes (FP), here FP 6 and 7 are relevant, listing also Horizon 2020 as the successor programme since 2014.

Initially, we found over 5000 individuated CVD research projects. Given the large number of projects funded for the period 2006-2013, we considered only projects that commenced and concluded within the time period, aiming at full project information and representative research of a sample to 100 (N) projects for all European member states. We found that the average life of grant is of 35 months with a minimum time of 8 months and a maximum of 60 months per project. Projects by National RFOs are by a large share conducted by universities (n=82) or public researching bodies (n=10). In only few cases larger consortia made up from private actors and NGOs have been funded for CVD research. Typically, research has been granted to national universities or to national university research clusters by up to five universities or hospitals (n=4). Indeed, over 91% of the projects were funded within the MS of RFOs, and only 9% were developed on a European level. Interestingly enough, these rare European research projects are marked by narrow research questions, e.g. the Epidemiology of coronary heart diseases in the resident population in Luxembourg compared to the unusual high proportion of non-resident population when compared to their dietary habits in collaboration with France. On the other hand, there are only a few established research focused cooperation programmes, such as the EEA grant by Norway, Liechtenstein and Iceland funding research projects in CESEE countries and Greece (n=2). The other two projects are represented by so called researching networks of the BMBF of Germany, clustering up to five universities or hospitals to competence centers for prevention of heart failure and its complications and Atrial Fibrillation, both also leading the project sample in terms of total funding of € 25million and € 18 million respectively.

Research by pharmaceutical companies in this selection for combating CVD is rather moderate. Less than 10% of all R&D investments in the field of NCDs among the top 10 European pharmaceutical companies has been spent on CVD research. Almost half of the last four years research pipeline of European and US pharmaceutical companies was done for oncology-related drugs (n=164 out of 341). The tremendously skewed research efforts towards cancer have been amplified over past years. Despite being the largest pharmaceutical company in Europe, ROCHE has only three CVD relevant molecules in its research pipeline and is heavily focused on cancer related drugs. Over the last years, 21 out of 35 NCD molecules have been cancer-related drugs. Two of the three CVD-related molecules were stopped in 2012 and 2013 respectively. One molecule against the Acute Coronary Syndrome is currently in Phase I.

The Medical Devices Industry provides a mixed picture in relation to investment in CVD research. On the one hand, we identified only 5 companies with CVD-relevant products but on the other hand, the bibliometric output data does not confirm this. Here, all –except of one- companies were involved in CVD-research. Siemens AG e.g. published 299 articles about CVDs, but we could not identify a single device in the searched databases. A deeper conclusion of the medical device industry is, based on this data, therefore difficult.

In general, our bibliometric analysis revealed the European area to be an influential player in CVD related research with 42% of all papers being produced within the European area. This might be partly the result of several countries having increased their CVD output rapidly, for example Poland, Greece, Portugal and Iceland. However, this does not hold true for all countries, since some countries, such as Germany, France, Austria, Ireland, Slovenia, Bulgaria and Estonia decreased their research output. In addition to the substantial quantity of papers produced in the European area, the quality is another important determinant for evaluating the research output. A possibility to operationalize the quality of papers is to look at the mean citation scores. Comparing European papers with papers published in other parts of the world shows that European papers are on average cited more often and that the difference in mean citations has increased throughout the last years.

Looking at the output of each European country revealed an overall significant positive correlation between paper output and their levels of GDP. But there are some countries that deviate substantially from this relationship. Countries like Netherlands and Greece produce more papers relative to their GDP than countries like France, Norway or Romania. Romania in particular is noteworthy since it publishes less than half of what might be expected from its wealth.

WP3
Chronic Respiratory Diseases (CRDs) are a group of illnesses that produce respiratory abnormalities under which breathing becomes slowed or forced. Although serious and often life threatening, most CRDs are largely preventable through simple clinical strategies that are both inexpensive and easy to administer. CRDs include major conditions like: chronic obstructive pulmonary disease (COPD), respiratory allergies, pulmonary hypertension, chronic bronchitis, asthma, emphysema and occupational lung diseases. According to the Global Burden of Disease Study (GBD), CRDs are the fifth leading cause of lost Disability Adjusted Life Years (DALYs) across Europe in terms of the major non-communicable disease (NCDs) categories. Breaking down the CRD category into its major diseases, COPD and asthma are the first and second largest cause of lost DALYs across the European area.

WP3 provides a high resolution map of European Research Funding Organizations (RFOs) for CRDs, and the scope, scale and impact their investments, for the purpose of identifying gaps, overlaps and potential synergies for future funding of the disease area. The chapter produces findings at two levels: (a.) the Present Level, which involves a description of the significant features of the map of CRD research, including observations about the scale, scope and an impact of European investments; and (b.) the Future Level, which involves a non-exhaustive set of recommendations regarding gaps, overlaps and potential synergies for the future funding and potential areas of focus for CRD research in Europe.

The funding map for European investments in CRD research has a several important features, perhaps the first of which is that there are at least 131 RFOs for CRDs across the European area. Many of these RFOs are active funders of research in other NCD areas (n=119). At the top end of the scale, 38 RFOs have an annual research spend of over € 1.0 million. At the lower end, 13 RFOs have an annual spend below the € 0.1 million, the majority of which were from smaller European countries.

On average, the CRD research projects that these RFOs fund have a life span of about 28 months. The projects generally focus on therapeutic and drug development in relation to the major diseases categories within CRDs. Most currently funded research focuses on asthma (32%) with COPD (24%) and cystic fibrosis (14%) following behind. Typically, RFOs fund research conducted at universities or university research centers. They do not usually fund private sector organizations, or even large consortiums of public sector research centers. In fact, the European Commission (EC) is perhaps the only RFO that sponsors large cross-border collaborations.

Funded projects involve basic strategies for the prevention and management of CRDs. Although many of these initiatives have been successful, they are not usually part of a coordinated action plan. However, RFOs generally suggest that these basic strategies for the treatment of CRDs should feed into a coordinated policy program involving emission standards, pharmaceutical therapies, disease management and regulatory controls for tobacco.

The private sector is an often neglected feature of the map of CRD research funding. In Europe, private pharmaceutical companies have generally increased their commitment to R&D over the past four years. However, the sector demonstrates a limited commitment to investment in CRD research. Only five European pharmaceutical companies are currently active in the development of New Molecular Entities (NMEs) for CRDs.

The Medical Devices Industry provides a similarly mixed picture in relation to investment in CRD research. Broadly, industry’s commitment to R&D investment remained steady over the past four years, but advances were limited to individual companies. Most companies in the sector had not developed any CRD relevant medical devices.

Beyond the private sector, the great majority of European RFOs for CRDs are national public sector organizations that make funding investments at the national level (n=60). There are no public sector RFOs operating at the international level, and very few national public sector RFOs make investments at the international level. The overwhelming picture the map presents is that CRD research is being conducted at the national level on the basis of national level priorities. Government is the major funding source for 72% of RFOs across the European area. By comparison, other funding sources, such as the private sector and general fundraising activities, played surprisingly minimal roles.

By comparison with other NCD categories, CRDs is a small subject area. Consistent with its comparatively small size, impacts of CRD research funding are also small. In terms of research output, CRDs, and even in Europe, average only 0.8% of papers published in biomedicine overall. In terms of publications, the representation for major CRD disease areas is very unequal, with asthma representing over 40% of the total, and bronchiectasis barely 1%. COPD measures about 24%.

Assessing the impact of CRD research investment on the basis of these outputs, our map suggests that some European countries are doing well and other less well. For example, the UK has the highest research output for CRD investment, more than twice as high as the second country, France, which is publishing almost twice as much as expected considering their relative levels of national wealth (GDP). Similarly, Sweden and the Netherlands are also publishing about as twice as much as expected. On the other hand, Austria is publishing very little, and Germany, Norway and Switzerland are doing barely half of what might be expected considering their wealth.

Measured on the same basis, the impact of private sector investment in CRDs is even more minimal. With the exception of Novartis, which seems to dominate CRD research, and is responsible for 79.4% of all scientific papers, very few U.S. and European based companies are producing outputs relevant to CRDs at all. In terms of medical devices, output of scientific papers is also limited. Again, Novartis was responsible for the overwhelming majority of papers for medical devices (76%) with Covidien claiming a 5.7% share, Johnson and Johnson 3.0%, Koninklijke Philips NV 2.8% and Siemens 2.6%.

RFOs have very generalized intentions with regard to future strategies for CRD research. When invited to respond, they mentioned nonspecific themes like ‘prevention’, ‘health services research’ and ‘patient management’. RFOs do not seem prefer generalized themes, rather than strategic goals for CRD research. Interestingly, RFOs did not declare any interest in developing strategies that build on the success or failure of earlier strategies and investments. Consequently, CRD research organizations would seem to enjoy a large ambit of autonomy in directing the science within their field, which may, in fact, be a very desirable outcome. Regarding potentialities for future collaborations, the most common current forms of RFO collaborations are between public sector organizations, followed by partnerships between the public sector and the voluntary sector organizations. Interestingly, there are comparatively few partnerships between private sector and public sector organizations with regard to CRDs research.

Interestingly, comparatively few RFOs are anxious to pursue links with the private sector with regard to CRDs. While this finding may be a consequence of the fact that private companies are not very active in the disease are, it probably needs to be considered in relation to findings in the other NCD categories. Overall, results indicate that RFOs investing in CRD research are relatively content with their existing strategies and funding regimes. RFOs plans for the future do not depart in a radical way from the strategies and plans they already have in place. They are content to pursue the development of new therapies and treatments. And they are content to pursue funding collaborations with voluntary and public sector organizations.

Enhancing the resolution of our map with stakeholder interviews, we were able both to contextualize existing areas of the map, and also make projections about the future focus and shape of CRD research. Our informants, mostly scientists, were acutely aware of the future research needs within the field and quite forthcoming about the principles upon which RFOs should be investing in the disease area. Broadly, they were not content that future arrangements for funding in the field should resemble existing arrangements. At the very least, this finding would seem to indicate that any assessment of the potential overlaps, synergies, gaps and potential areas for collaborations in the field of CRDs is likely to develop from within research organizations themselves rather than the RFOs investing in them. Indeed, our map demonstrates that RFOs lack the resources and expertise to make informed assessments regarding overlaps, synergies, gaps. In this case, any assessment of gaps, synergies and overlaps has little option but to privilege the perspective of informants within the research organizations themselves.

Our map reveals five major themes with regard to the future of research in the area of CRDs. There is a generalised recognition of the growing importance of stratified medicine, which several informants considered to be the future of research across the wider spectrum of NCDs. Other informants emphasised the importance of directing research towards tackling individual CRDs such as COPD and asthma. For COPD, informants stressed that the future was equally about non-clinical measures like smoking cessation, public health and health service delivery and also about relieving the symptoms of COPD via pharmaceutical treatments. For asthma, informants suggested there was a greater scope for developing innovative and game-changing pharmaceuticals than for COPD. Across the broader spectrum of CRDs, and indeed NCDs, informants also suggested that there was a need to find new ways of working with private sector. Consistent with the academic literature, stakeholders emphasized that the model for drug discovery has changed and that pharmaceutical companies were less prepared to take risks in regard to allocating resources. Consequently, there was a need for academic institutions to pursue the basic science that pharmaceutical companies were no longer able to conduct for themselves. Other informants suggested that these new research requirements needed to fit within a wider strategic approach to the funding of NCD research that accommodated the both needs of researchers and the requirements of funders to demonstrate the effectiveness of their investments. In the interests of providing readers with direct access to the views of leading scientists in the field, our map purposefully reproduces the voices of individual scientists who are identified numerically in order to distinguish them individually.

WP4
The work carried out in WP4 has provided a broad comprehensive picture on research funding activities and organizations (RFO), both private and public, active in DIAB between 2002 and 2013.

1. Systematic Mapping of DIAB Research Activities at MS, Regional, EU and NGO or private level
DIAB is a chronic disease whose prevalence is about 9% in the adult population worldwide. This condition accounts for 11% of all deaths in the age 20-79 years in the EU. Unlike other NCDs, the trend has increased over the last decades and projections show it will be even more burdensome in the future, due to population-ageing. Across the EU, there are a total of 120 RFOs investing in DIAB research. A total of 14 diabetes-specific RFOs were then identified. The majority were Government/Public organizations (58%) or private-not-for-profit (PNP) organizations (40%). Overall, more than two thirds of the RFOs operate at national or sub-national level and only 12% has an international reach. Across the EU, DIAB-relevant RFOs also display major divergences. Eastern Europe tends to have a centralized funding systems provided by public institutions unlike Central Europe, where the majority of RFOs are PNP.

Across the European area, 36% of RFOs fund research for the purpose of advancing the research and policy agenda. A small proportion (11%) identified a connection between previous funded projects and their future plans. Some RFOs lack explicit processes for developing goals and intentions for the future. This deficiency may well hinder the development of long term plans and increase the potential for discontinuity in research. The most common expected impacts for funded projects are published academic papers, improved access to new therapies and care and achievement of public/political attention. In terms of future strategies, research in the development and evaluation of treatments and detection screening and diagnosis seem the most common objectives.

Research on specific projects funded by the identified list of RFOs was conducted on institutional websites and websites of national associations on diabetes. A vast range of European projects, each receiving over one million Euros of funding, target diabetes and its complications, with a major focus on aetiology and origin of the disease or development and evaluation of new therapeutic interventions. Fewer projects relate to patient or disease management. Four projects aim at improving prevention strategies for Type II diabetes and, unsurprisingly, no projects target the issue of prevention of Type I diabetes. A considerable bulk of research has been focusing on the study of treatment for diabetes. When comparing current funded projects to strategies set in the past, some areas of research seem to have been more explored, while for others there is still need for improvement both in knowledge advancement and in coordination among stakeholders.

As regards private sector investment in NCDs research, considering both Pharmaceutical and Medical Device sectors, we mapped R&D expenditures of these companies to quantify R&D composition and size within DIAB research. We considered the top European and US pharmaceutical companies in terms of R&D investment in 2014, whereas in order to map Medical Device industry R&D investments, we identified a list of top 16 medical device manufacturers worldwide ranked by total revenue. EU companies exhibit 40 molecules under development in DIAB. The majority of molecules developed by US and EU top pharmaceutical firms are aimed at treating type 1 and type 2 diabetes mellitus, while only few companies have developed molecules for the treatment of diabetes complications such as diabetic retinopathy, neuropathy and macular oedema. Among the identified MD companies, only a minority of them have a device for DIAB in their research pipeline.

2. Impact assessment and critical appraisal of major research initiatives in Diabetes
UB team build upon this data and analysed impact of research activities in a first analytical component (Impact Assessment, IA), tracking activity, investment and initiatives within European Union Member States in the area of DIAB. Survey results were complemented by an extensive literature review, epidemiologic and disease burden data.

From 2003-2013, annual spend for DIAB research funding experienced an overall steady increase, with an average annual growth rate of 5% and a sharp increase between 2008 and 2009 of 41%. The overall annual funding tracked for the last three years in our historical series was about €140million. When the trend is broken down into geographical areas, the Southern European MSs seem to keep their annual levels of spend almost static. The highest amount of funding for DIAB research seems to come from the UK and the Netherlands.

The Survey Tool also delivered richer qualitative data regarding the basis on which the above levels of resources were allocated. Leveraging this data from the Survey Tool, the IA found that most EU MSs had introduced policy initiatives and programmes for combatting DIAB. Major efforts are currently directed to improve detection and screening and to develop new therapies. On the contrary, lower effort is dedicated to improved patient management. Prevention strategies for diabetes mellitus are known and not controversial. However, further research would need to span across a variety of scientific specializations, from biological science and genetics to population studies and epidemiology, and even social science.

The Impact Assessment shows that 45% of investments are made on the basis of a ‘call for proposals’, in which submissions are invited from scientific researchers in the field. Around 10% of RFOs have stated that there is no specific process, while about 40 RFOs (37%) did not provide data on this issue, possibly indicating that they have in fact no clear process and criteria for making investment decisions. This result is accompanied by a negligible proportion of investment directed to ‘unfunded’ research (2%) or, to a larger extent, previously funded research (10%). Supporting projects that benefit the research and policy agenda seems the most common approach (33%). It seems useful then to establish a common research and policy agenda, based on a coordinated effort among relevant partners, to which each RFOs could refer to when funding diabetes research.

In a second analytical component of the WP (Critical Appraisal), a cross-comparison of all sources and data, including interviews with key stakeholders and the results of the bibliometric analysis, helped to shed light on areas of overfunding or underfunding in DIAB, or generally challenges in this field.

In particular, the themes emerged from the stakeholders interviews were organized in six major areas: i) challenges in diabetes research, ii) duplication in diabetes research, iii) research gaps, iv) impact of research and priority-setting, v) partnerships and vi) the role of the EU. According to the informants, the challenges faced by researchers and RFOs in diabetes research are financial (i.e. a perceived decrease in targeted funding over the last years, mainly associated with the economic crisis that hit Europe in 2008) and organizational. As the majority of diabetes research funding was coming from private pharmaceutical companies, our informants suggest that industry has implemented conservative management practices for the purpose of increasingly the predictability of drug discovery and the sustainability of returns on capital investment in R&D. Developing the right balance through cooperation between heterogeneous lists of funders is therefore critical to foster the advance in scientific and clinical research. Duplication can only apparently be considered a challenge. Turning redundant funding in opportunities to tackle the most under investigated areas or replicating results in settings where generalizability can be an issue, were common discussion points. A general broader view on the impact of research was supported. Stakeholders recognized the importance of looking beyond academic publications, beyond “scientific excellence” to promote impact on delivery of services and patient health outcomes. This approach involves a relatively higher consideration of translational or implementation research and highlights the importance of general public involvement in all stages of research. Establishing partnerships for the funding and conduct of research seems a successful strategy to be able to co-fund bigger project, to explore research questions on large scale or to achieve dissemination targets otherwise difficult to achieve.

In close collaboration with colleagues at King’s College, the bibliometric impact of DIAB research was studied. Although diabetes is a small field, its presence within biomedical research has increased between 2002 and 2013 from 1.45% to 1.86% worldwide. The European presence has declined from almost 44% of the world total to under 36% because of larger increases elsewhere. The UK has much the highest output, publishing over 50% more than expected, but the three Scandinavian countries, Denmark, Sweden and Finland (but not Norway) are doing even better. On the other hand, Romania and Norway are publishing only half as much as their wealth would suggest.

Papers acknowledging the support of the leading European funders were also examined. The EU is the largest single source of support in terms of the numbers of papers funded, with the second largest being the Danish company Novo Nordisk A/S. The remaining are mainly government agencies, with the exception of the Juvenile Diabetes Research Foundation, the Wellcome Trust and Diabetes UK.

3. Development of future research agenda
A third analytical component (Synthesis Report) consolidated major findings of two previous reports into a synthesis of research activities for DIAB in 2002- 2013 and summarized research and policy recommendations.

When comparing current funded projects to strategies set in the past, some areas of research seem to have been more explored, while for others there is still need for improvement both in knowledge advancement and in coordination among stakeholders. For instance, it still emerges the need for better coordination of research and access to biological samples, registry data and research findings across Europe to avoid duplication and enhance effective use of funding. Communication also remains a critical issue with respect to both better dissemination of advancements in research and effective engagement with diabetic patients and public opinion through activities aimed at raising awareness.

Areas that could be further expanded are those of preventive and treatment measures for complications, given the impact of complications (such as vascular complications) on the social and economic burden for patients and society; biomarkers identification that help predicting individual risk to develop Type II diabetes; studies of geographical, social and environmental information to enhance the development of lifestyle intervention strategies to prevent Type II diabetes and obesity. Another probably unexpected topic suggested for future research was on the social and health related quality of life aspects of people living with the disease, how it is possible to make patients and families more engaged with the treatment and how to get them to use the treatment more effectively. In this regard, a wide set of skills and multidisciplinary research projects are necessary to tackle a disease that is more complex than the genetics behind it. A better integration of activities and collaboration between European institutions, the academia and the industry, particularly the food & beverage sector, is needed not only in terms of research funding but also in terms of policy/regulatory actions. Finally, the role attributed to the EU is one of coordination and leadership. Recommendations from the field to one of the major public government funding body goes from engagement in fruitful discussions with all stakeholders (e.g. inviting charities and private not-for-profit funders to major discussion tables on research initiatives) and better scoping of experts in drafting programs calls.

WP5
“Cancer” is a generic term for a large group of diseases that can affect any organs or tissues of the body (e.g. breast, lung, skin or bone marrow). There are more than 100 types of cancer, which are usually named for the organs or tissues where the cancers form. Other terms used are malignant tumours or malignant neoplasms. One defining feature of cancer is the rapid creation of abnormal cells that grow beyond their usual boundaries, which can invade surrounding tissue and can metastasize to distant sites. Cancer is the leading cause of death worldwide, but there are marked geographical differences, particularly when considering specific tumour types rather than cancer as a whole. With an estimated 3.45 million new cancer cases (excluding non-melanoma skin cancer) and 1.75 million deaths from cancer in Europe in 2012, cancer currently represents the second most important cause of death and morbidity in Europe. Cancer is mostly preventable. The prevention of cancer is one of the most crucial global public policy challenges of this century. Major reductions in the rates of many cancers are achievable. But to be effective, prevention requires a new appreciation of the nature and role of external factors that protect against or are causes of cancer.

The funding map for European investments in Cancer research has a several important features, perhaps the first of which is that there are a total of 169 Research Funding Organizations (RFOs) investing in cancer research. Some of them RFOs (n=78) are devoted exclusively to cancer research, whereas the overwhelming majority make research investments in other NCD disease areas. The study identified 87 RFOs with an annual spend of above € 1.0 million. It also identified 14 RFOs with an annual spend below the € 0.1 million threshold.

We identified and surveyed 169 RFOs active in the disease area. The survey response rate was 55%. However, many of these RFOs were also active funders of NCD research in other disease areas (n=91), which complicated efforts to ascertain the proportion of resources that they allocated to cancer. Even where these multi-dimensional RFOs were cooperative, they did not keep separate records for each disease area and were only able to estimate the proportion of resources they allocated to cancer. By supplementing survey responses and non-responses with website queries, we were able to include 108 RFOs in the quantitative analysis. However, given the non-availability of survey responses from all RFOs, information was completed with website queries for those RFOs that did not respond (n=61).

In Europe, the funding figures are substantial, with over a billion euro invested in the disease area per annum (from the year 2008 onward). Over the study period (2002-2013), the total amount of funding reaches over €11 billion for the UE-28. When considering Europe overall (31 countries), the total spending allocated to cancer research is over €14 billion. Over the same period, Central Europe´s contribution to the overall investment in cancer research is considerable, representing € 6.6 billion.

Generally, most identified RFOs were not ‘Cancer-specific’ (n=91). Instead, they funded research under the broader category of NCDs, and were not themselves concerned even to distinguish between disease categories. Indeed, only a few RFOs were entirely focused on financing cancer research. Further, most western and northern EU MSs structured research funding on a geographical basis, distinguishing between central government funding, and funding allocated by regional and local authorities (e.g. Länder in Germany). By contrast, Eastern European MS centralised research funding, and organized RFOs by wider subject areas, with councils or other governmental bodies focusing on health research in general. In Southern Europe, the research and development conducted on the basis of regions, and dominated by the public sector, with only few private foundations and NGOs actively financing health research.

Government was the main source of funding for 89 RFOs (43 from Central Europe, 8 from Northern Europe, 27 from Southern Europe and 11 from Eastern Europe), whereas the contribution of the private sector, fundraising activities, donors and other sources, is minor, as it is mainly present in Southern (n=14) and Northern Europe (n=12). Cancer research receives funding from the public sector mainly in Eastern (100% of total funding) and Southern Europe (55% of total funding), whereas in Central and Northern Europe there are other sources of cancer research funding (e.g. fundraising campaigns, contributions by members, and others) contributing to the overall investment of cancer research. Interestingly, the private sector is found to be another source of cancer research funding in Central and Southern Europe only.

Currently funded Cancer research projects have an average life span of about 37 months. The total funding allocated to the selected research projects/programs is over €278 million (2.5 million per research project/program). The funding allocated for the majority of the selected projects (68%) was under €3 million whereas for a relatively smaller number of projects (16%) the funding allocated reached €5 million.

The European pharmaceutical sector has five companies in among the world’s top ten pharmaceutical firms. Overall, the European pharmaceutical sector has increased its commitment to R&D over the past four years. GSK is the only top 10 company to record a shrinking commitment to research investment. But the research pipeline for the top 10 European pharmaceutical companies suggests that firms seem to specialize in certain NCD categories. For example, pipeline data shows that NOVO-NORDISK, which prefers to focus on other areas, does not have any cancer relevant molecules under development; other firms like NOVARTIS or ROCHE, however, are developing several.

There are large differences in the channels and flows of funding for cancer research among the countries. Indeed, despite the majority of public funding in cancer research being concentrated in public funding organizations (46% public and 4% private-public sectors), each country has its own funding system. Research systems are generally complex, with some common features depending on the European area: Eastern Europe is characterized by a single funding source (100% of the RFOs are from the public sector; other sectors do not exist), while in Northern Europe charitable and voluntary sectors are the main active funders of cancer research. A more balanced situation is observed in Southern and Central Europe, where all types or funding organizations and funding sources are active.

In terms of the effectiveness of research investment, Cancer funding is among the top three with an average of 38% of paper published for Cancer are of European origin, behind RESPI(56%) and CARDI (42%). According to the results in Europe, Cancer is a big subject area, averaging 11.5 % of the papers in biomedicine overall. Cancer research represents just over one ninth of all European biomedical research output, but one eighth of world biomedical output.

Germany has the highest output in terms of Cancer papers, highlighting that genetics is the dominant research type, followed by chemotherapy, prognosis and surgery. On the other hand, Malta is publishing very little and Germany, Italy and UK are the ones most publishing in cancer research, correlative within the size of GDP.

Within the area of Cancer, the impact of European research is defined first of all into 11 research types, generating different numbers of papers, with genetics giving the most (7.1%), followed by chemotherapy (10.0%), prognosis (9.6%) and surgery (9.4%). Second by 22 different cancer sites, with breast giving the most of cancer papers (10.2 %), followed by colon/rectum (6.9%), leukaemia (5.9%) and lymphoma (4.7%).

Cancer research publications is considered to be one of the major outputs of funding, and is for this reason used as an objective surrogate of overall cancer research activity by the funding organizations. It is therefore used to measure impact of research investment, but is in practice also the organization’s main expected impact. Future research strategies, as reported by 65 organizations, will be detection, screening and diagnosis at a first place, whereas fewer organizations (n=18) were aware of future research focused on policy or public attention, and on health and social health care services(n=36).

WP6
Mental health disorders (MHDs) encompass a broad range of conditions, including depression, anxiety, bipolar disorder, schizophrenia, alcohol and substance disorders, ADHD, suicide and self-harm, eating disorders and Alzheimer’s disease, which together constitute a major share of the disease burden in Europe. Compared to other major NCDs, MHDs are difficult to diagnose and manage due in part to a lack of diagnostic and risk stratification tools. Indeed, the etiology of many MHDs is not fully understood, although a variety of factors are assumed to contribute to the occurrence of mental disorders, including individual attributes, social/economic circumstances and environmental factors.

WP6 contributed to the knowledge base by mapping MHD research funding activities undertaken by public, private and voluntary entities at European and MS levels, exploring MHD research outputs through bibliometric analysis and using these and other key evidence bases to quantify the overall impact of MHD research funding and to identify potential overlaps, synergies, gaps and opportunities in order to support an evidence-based future research agenda for MHDs. WP6 produced findings at two levels: (a.) the Present Level, which involved a description of the significant features of the map of MHDs research, including observations about the scale, scope and an impact of European investments; and (b.) the Future Level, which involved a non-exhaustive set of recommendations regarding gaps, overlaps and potential synergies for the future funding and potential areas of focus for MHDs research in Europe.

Despite the large burden of disease, MHDs typically receive less attention and funding than other non-communicable diseases (NCDs). With the exception of schizophrenia, the amount invested in MHD research is much lower than the overall average of €25 per DALY for all diseases in Europe. Indeed, for mood disorders, the amount is significantly lower: €4 per DALY for depression and €2.9 per DALY for bipolar disorder. However, governments and other public, private and voluntary organizations have begun to recognize and work to address the burden of MHDs.

The funding map for European investments in MHDs research identified 137 MHD RFOs in 20 countries, most of which also fund research in other NCD disease areas. The majority of the MHD RFOs was public and received government funding. The level of involvement of charitable organizations in MHD research is low compared to other NCDs, such as cancer.

Analysis of the top-funded MHD projects identified in the five largest European Union (EU) Member States (MSs) showed that Alzheimer’s disease and schizophrenia received the bulk of the funding, followed by depression and to a much lesser extent anxiety. Considered in light of the disease burden for these four MHDs, depression and anxiety research received a much lower share of project funding than their burden would suggest. Nonetheless, prevalence should not be the only metric considered in setting research priorities. Other factors, including treatment gaps, cost of illness and potential return on investment, should be part of the decision process.

Identified national-level projects were not collaborative, although the largest MSs were highly involved in collaborative projects at the EU level. Smaller MSs have much lower participation in EU projects. The level and extent of research and development (R&D) in drugs targeting MHDs has diminished in recent years, both in Europe and the US, due to the long, expensive and uncertain development process and the lack of identified treatment targets. An exception is Alzheimer’s disease, where the largest pharmaceutical companies had ongoing development activities. Moreover, smaller pharmaceutical companies are active in MHDs, including for MHDs that received low or no investment by the top 20 companies.

R&D investment in MHDs by medical device companies was not extensive, although recent innovations include different forms of transcranial stimulation. In addition, imaging devices are being used to identify treatment targets and for diagnostic purposes, with the development of tracers often undertaken by pharmaceutical companies. Other device innovations are being developed outside of the traditional medical device industry, including mHealth applications, which may provide expanded access and minimize issues of stigma. Additional research is needed in this area to provide a more solid evidence base, and the regulatory status of such applications must be clarified, with a particular focus on ensuring patient confidentiality.

Beyond developments in drug- and device-related therapies, our map considers the role of psychological therapies for MHDs, which may be used alone or as an adjunct to other therapeutic modalities. A solid evidence base must be developed for these therapies, requiring increased investment.

MHD research outputs in terms of scientific publications by researchers in the 31 European countries included in our map account for almost 6% of all biomedical research and 40% of MHD research worldwide. Depression, Alzheimer’s disease and schizophrenia papers comprised 57% of the output for the years 2002-2013 among 10 MHDs. Some relatively high prevalence MHDs, including anxiety and alcohol abuse disorders, receive only a small share of research attention.

Our map suggests that there is a need to encourage and facilitate research collaboration at the European Level. Multiple programs, bodies and platforms have been established at the European level in recent years and have contributed to increased cross-national research collaborations. However, participation by some smaller countries has been very low or non-existent. Moreover, existing structures do not facilitate participation by certain key stakeholders, including patients and caregivers, in setting research priorities. The ROAMER project provided a model for this type of broad participative process. The EC should explore existing structures to determine whether there are barriers to participation that may be addressed and consider whether a neutral platform would encourage broader participation by MSs and the full range of stakeholders.

Our map also suggests that there is a need to support expanded MHD research participation by less traditional participants. Charitable RFOs, smaller pharmaceutical companies and developers of eHeath and mobile devices play important roles in funding research and development of innovative MHDs therapies, particularly for diseases that receive less attention from larger institutional research entities. The EC should explore how best to support and expand such research activity.

Finally, our map identifies some non-exhaustive research priorities for larger RFOs. It suggests that identifying biomarkers for MHDs is essential to improving diagnosis and screening and developing targeted treatments. With the withdrawal of major pharmaceutical firms from MHD drug development, public RFOs should expand their investment in fundamental research. This also implicates cross-national collaboration and coordination to avoid redundancy and to maximize the potential uses of existing data. While the EC is already a key facilitator of European research collaboration, it should consider how to strengthen and further support breakthroughs in this essential aspect of MHD research.

WP7
Bibliometric analysis establishes establish impact and outputs in terms of number of papers published in the Web of Science (WoS) relative to research priorities and levels of investment. In establishing potential research overlaps, synergies, gaps, WHO data on the burden of disease, the latest year for which data are available is 2010, is another a useful tool. Indeed, many NCDs do not lead directly to death; but, rather, to disability (and withdrawal from the labor force), neuropsychiatric conditions (depression, multiple sclerosis, migraine), sense organ disorders (blindness, deafness) and musculoskeletal diseases (arthritis). Using the data, the mapping exercise investigates whether some European countries publish relatively more papers than their particular disease burden would appear to justify, thereby indicating that there may be potential for savings to be made and resources deployed elsewhere. Conversely, where fewer papers are published in high burden areas, there is also some evidence of a research gap. International collaboration can also be evaluated in this way: countries with smaller scientific output usually have a greater need to seek partners abroad, enabling an identification of MSs that collaborate less with other MSs in a given subject area than would be expected, which would suggest the need for the deployment of additional means to encourage such partnerships.

1 Listing of research papers and funding analysis
1.1 Definition of filters and sub-filters
Papers (articles and reviews) were identified in the Web of Science (WoS), both the Science Citation Index Extended and the Social Sciences Citation Index, by means of "filters". These were based on specialist journals and title words, which were developed and calibrated in consultation with our project partners. The calibration factors (precision and recall) were typically above 0.9. The bibliographic details of the papers from 31 European countries (the 28 Member States of the EU plus Iceland, Norway and Switzerland) were then downloaded to a series of text files and converted to five Excel spreadsheets (one each for cardiovascular disease including stroke, CARDI, 211,507 papers; diabetes, DIABE, 40,550; mental disorders, MENTH, 138,666; oncology, ONCOL, 282,055; and respiratory diseases, RESPI, 18,822) by means of a special VBA macro. Sub-filters were also defined, covering disease areas or mental disorders, and research types. These were applied to the spreadsheets of the downloaded papers, and were also based on title words and journal name strings. They were developed in consultation with experts within KCL and UK medical research charities.

1.2 Burden of disease
Data on the burden from the five NCDs and from individual diseases or disorders within them were obtained as the numbers of Disability-Adjusted Life Years (DALYs) from the Global Burden of Disease study published by the University of Washington. This database gives estimates for individual countries and for some 70 diseases within the five NCDs and for six base years from 1990 to 2013. We took the numbers for 2010, being within the time period covered by our study, and downloaded them early in 2015. [However we subsequently found late in 2015 that some of these numbers had been altered. The original numbers were retained as they appeared to agree better with World Health Organization data for 2004 published on their website.] We also determined the total burden for the EUR31 countries and compared the percentages of total DALYs for individual countries and diseases with the European average values.

1.3 Overall outputs of papers
We also determined the numbers of papers world-wide for the five NCDs, and the numbers of papers in biomedical research overall, both in the EUR31 countries and world-wide. The EUR31 countries as a group publish some 40% of biomedical research, and four of the NCDs, but as much as 56% in RESPI. The five NCDs represent very different amounts of EUR31 biomedical research: ONCOL 11.5%, CARDI 8.7%, MENTH 5.7%, but DIABE just 1.7% and RESPI only 0.8%. In comparison with the European disease burden, cancer appears over-researched relative to the other NCDs, but respiratory diseases are seriously under-researched. This is one of the main conclusions of the study.

1.4 Comparison of outputs with wealth
For each NCD, we compared each European country's output of research with its wealth, as measured by GDP, as there is normally a much better correlation than with population. It appears that the Scandinavian countries (other than Norway (NO)) are performing rather well, with both Finland (FI) and Sweden (SE) mentioned four times, as is the Netherlands (NL). The UK is mentioned three times, and Greece (GR) and Croatia (HR) twice each. On the other hand, France (FR) is mentioned four times in the low category and Norway, a very rich country, together with Romania (RO), a poor country, score low in three NCDs.

1.5 The cost of an NCD paper and estimates of total R&D spend in Europe
In order to triangulate the results of the surveys of individual funders carried out by our partners, we sent questionnaires to leading researchers in three groups of countries, selected on the basis of their incomes per caput. The researchers were asked about their total research budgets for the relevant NCD and their responses were compared with their outputs of papers, fractionated according to the numbers of addresses on each. Some of the replies gave unrealistically high or low values for the cost of a paper, but most gave values that accorded with previous estimates. The mean cost worked out at €412 k for the high-income countries (Norway, Switzerland); €260 k for the medium-income countries (Belgium, Finland – but the four large EU Member States, France, Germany, Italy and the UK also fall into this group); and €142 k for the low-income ones (Bulgaria, Czech Republic, Slovakia, Slovenia and Romania – but most of eastern Europe would be in this group). The mean figure of the overall average cost of a paper was close to that of the middle-income countries, at €255 k. This gives estimates for the total R&D spend on the five NCDs in the EUR31 countries as: CARDI €4594 M, DIABE €885 M, MENTH €3280 M, ONCOL €6163 M and RESPI €419 M.

Our analysis of the funding of the DIABE and RESPI papers indicates that 30-40% of NCD papers have no specific funding source, either explicit from the acknowledgements or implicit from the addresses. Such papers would be supported by universities and hospitals. The two major sources of funding are the public sector (both national and regional governments) and the private-non-profit sector, the latter especially prominent in northern and western Europe where there are many hundred collecting charities and endowed foundations. Industry contributes to about 15% of the total public domain research, mainly the pharma and biotech companies, but in DIABE the leader is the Danish company Novo Nordisk A/S. The European Union, with its many programmes, contributes about 3% and is one of the largest single sources of support.

2 Measurement of the impacts of European NCD research
2.1 Academic citations
The conventional measure of research impact is how often the published papers are cited by other ones. We counted citations in a five-year window beginning with the year of each paper's publication, and also determined what percentage of a country's papers received enough citations to put them into the top 5% of all European papers. Both measures were calculated with fractional counts based on addresses. [A paper with one French and two German addresses would count 0.33 and 0.67 for France and Germany.] Values were determined only for the 2002-09 papers.

The main result is that these European papers are at least as well cited as the world average, and in recent years, better because of the increase in Chinese papers which are mostly poorly cited and so depress the world average. Papers in DIABE, MENTH and ONCOL were only better cited than the world average in the latter years, and papers in MENTH in the SSCI were slightly less well cited than the world average because Chinese and other east Asian papers are seriously under-represented in this index, so this average is dominated by US papers. Within Europe, the countries with high average citation scores (Actual Citation Impact, ACI) in each NCD are listed in Table 2.

2.2 The percentage of reviews
This is a relatively new indicator of merit; it measures the esteem in which a country's senior researchers are held by journal editors, who usually invite reviews from leading scientists. The countries scoring higher than the European average were outlined in a table (see main report). Although this measure is usually correlated positively with citation scores, it measures something different.

The right-hand column of the table shows the number of occasions when a country's performance is above the European average, eleven being the maximum as citation scores for MENTH were also determined for SSCI papers (ones not in the SCI made up 22% of the total in 2002-09). Clearly the UK and Belgium score most highly and are above average in almost all the NCDs and both indicators.

2.3 National health advisory committee members
We had intended to search for Member State health policy documents, of which there are many, but found that they contained pretty pictures rather than scientific references to evidence for the policies. Instead we sought out the lists of members of government health advisory committees, to see what research they published – in which subjects, and whether it was co-authored with other European countries, which would indicate that European research was having a cross-border influence.

We were able to find details of members of health advisory committees in 21 of the 31 EUR countries, and we sought their papers in the WoS for the five years, 2009-13. In total, there were 62 committees, 1431 members, and 5713 papers in one (or more) of the five NCDs after they were matched to our five NCD files for these years. The overall pattern of subjects was similar to that of research outputs, but correlated somewhat worse with the disease burden in Europe. The amount of research by the advisers, and hence the advice, on MENTH is about right; CARDI and RESPI lack enough advice but DIABE and ONCOL may have too much. Health policy may be skewed as a result. We also looked at the cancer sites in which the oncologists worked: breast (and prostate) cancer get the most attention and lung (and pancreatic) cancer not enough.

The papers by these advisers in these 21 countries often had international co-authors. Overall, their own-country contribution was 78%, that of other EUR31 countries 14% and that of the Rest of the World just 8%. Since the European contribution to NCD research is typically 40%, with the Rest of the World contributing the other 60%, this means that the advisers' papers were much better linked to other European countries than expected. The expected European presence would have been only 0.4 x 22% = 8.8%, so it was almost 60% higher than expected, and the difference is statistically highly significant.

2.4 The evidence base of clinical guidelines
The second of three new indicators of research impact was obtained from the papers cited on European clinical guidelines (CGs). We limited their selection to those concerned with the 13 diseases responsible for at least 1% of the European disease burden. Because the selection of the CGs depended on their availability, the only analyses possible were of the types of research that were cited, and the countries involved in the cited papers. For cancer, the three treatments (surgery, chemotherapy and radiotherapy) are the most represented in CG references compared with their presence in European research, but genetics is little used in evidence, despite papers in genetics receiving the most academic citations, and surgery papers almost the least.
The provenance of the CG references was heavily biased towards Europe, as might be expected, although papers from the USA were also well cited on the European CGs. In DIABE, for example, the countries whose papers were over-cited the most were the UK and Finland (FI), followed by the Netherlands (NL), Denmark (DK), Austria (AT), Belgium (BE) and Sweden (SE). Those from China (CN) and Japan (JP) were under-cited by a factor of more than four.

2.5 The stories of biomedical research in European newspapers
Newspaper stories of NCD research were investigated by our partners and by about a dozen KCL graduate students from different European countries whom we recruited. They examined the archives of some 29 newspapers from 21 continental European countries, and selected relevant stories. [Stories from two UK papers and one Irish one are still being processed.] Their salient details were noted in a standard format in an Excel spreadsheet, and from these details the cited research articles were looked up in the WoS and their bibliographic data downloaded and copied across to the spreadsheet. This contained a total of 7144 stories and cited papers.

The stories were heavily biased towards cancer, with almost half the stories, and diabetes was also over-reported. Respiratory disease research was not neglected, but the main gap was in stories of cardiovascular research. Within cancer research stories, breast cancer received the most attention followed at some distance by colorectal, lung and prostate cancer, though many stories covered several cancer sites. The cancer stories were mostly about genetics and epidemiology, that is, how people could prevent cancer by changes in lifestyle and how it could be diagnosed. There was relatively little attention to methods of treatment, especially to radiotherapy and surgery (which account for most of the actual curative treatment) which together accounted for barely 5% of the cancer research stories.

Another topic of press interest was mental disorders research, which was covered in almost a quarter of the stories. The disorders attracting most newspaper attention were Alzheimer's and other dementias, and depression. Dementia, and its costs to society, has been much in the news recently but these stories came from newspapers published up to 14 years ago. However, little attention was paid to suicide and self-harm, although it is responsible for one eighth of the mental disorder burden in Europe. European research featured prominently in the newspaper stories, especially that from the UK. Once again, as with the references on European clinical guidelines, research from east Asia was little reported.
Potential Impact:
WP1
The results and impact of WP1 supported the completion of the project. Specifically, the completion of WP1 enabled Mapping NCDs to move beyond the state of the art in the research area in three distinct dimensions: firstly, by providing accurate mapping of NCD research funding activities at EU level and the MS (both institutional and charitable); secondly, by pursuing a bibliometric mapping and analysis of the volume of NCD research outputs in the EU and MSs relevant to NCDs, and, third, by using input from both the above to provide an analysis of potential overlaps, synergies, gaps, opportunities, overall impact of NCD research funding and help identify the research agenda for the future.

WP1 developed methods for the utilization of qualitative surveys of RFOs across the five disease areas, website inquiries, private sector analysis, interviews with stake holders and bibliometric analysis. RFOs are either public and third sector (voluntary, charitable, private non-profit) organizations that provide funds for NCD research. RFO surveys and website queries captured the types of research initiatives actuated in different European countries and regions. They identified which types of research activities are being used to address which types of NCDs in different nation states. They covered both the processes by which decisions to initiate research activities are made and the intended outputs research activities are expected to deliver. They described the reasons why different research initiatives have been sponsored and the different outcomes of interest that the various research activities undertaken in different MS were expected to achieve.

WP1 also provided methods for the analysis of the private sector. Research and development is a key industry activity across the pharmaceutical, medical devices and biotechnology sector. Throughout the 20th century, the private sector has played a significant and indispensable role in the advancement of science for NCDs through the production and publication of high quality products and research. Mapping NCDs necessarily included an analysis the research pipeline for private sector pharmaceutical technologies and medical devices in the area of NCDs.

WP1 also provided methods for the Bibliometric analysis, which established impact and outputs in terms of number of papers published in the Web of Science (WoS) relative to research priorities and levels of investment. In establishing potential research overlaps, synergies, gaps, WHO data on the burden of disease, the latest year for which data are available is 2010, was another a useful tool. Indeed, many NCDs do not lead directly to death; but, rather, to disability (and withdrawal from the labor force), neuropsychiatric conditions (depression, multiple sclerosis, migraine), sense organ disorders (blindness, deafness) and musculoskeletal diseases (arthritis). Using the data, the mapping exercise investigated whether some European countries publish relatively more papers than their particular disease burden would appear to justify, thereby indicating that there may be potential for savings to be made and resources deployed elsewhere. Conversely, where fewer papers are published in high burden areas, there is also some evidence of a research gap. International collaboration can also be evaluated in this way: countries with smaller scientific output usually have a greater need to seek partners abroad, enabling an identification of MSs that collaborate less with other MSs in a given subject area than would be expected, which would suggest the need for the deployment of additional means to encourage such partnerships.

Mapping NCDs necessarily involves a common framework of analysis for establishing levels of disease burden, for mapping of NCD research funding activities of RFOs at the European, regional and country level, for analyzing the private sector research pipelines, for bibliometric mapping and the analysis of the volume of research outputs across the European area. WP1 provide the methodological basis for this framework, upon which the Mapping NCDs project produced five high-resolution maps for each of the disease areas, the results and impact of which are set out below.

WP2
In terms of future plans, the results show an absence of a clear pathway for developing goals and intentions regarding CVD research. The overwhelming majority of RFOs indicated that future plans included cooperation with government and public institutions followed by cooperation with voluntary institutions. Surprisingly little cooperation seems to take place between RFOs and the private sector. Curiously, none of the RFOs declared any connection between already funded projects and the future plans.

While non-scientific researchers must remain cautious about suggesting areas for future research funding, our map also suggest some non-exhaustive areas for future CVD research funding. These areas are necessarily subject to revisions as the field of CVD research shifts and develops.

Research should aim to establish a reliable and solid database for monitoring CVD prevalence in Europe. The current available databases offer a tool for preliminary analysis on CVD mortality and prevalence in Europe. However, the persistent regional variance of CVD mortality in the European Union cannot be fully explained by variations in risk factors, capacities for early risk detection or health system efficiency in member states. Fluctuations in e.g. hypertensive diseases in Eastern European countries, or under representative CHD prevalence in France has been subject to extensive research and revealed practices of wrong coding patterns (so called “garbage coding”) and national biases towards coding CVD categories. This may not cause a major problem at a highly aggregated data level, but limits comparability across EU member states in health system effectiveness. This can be remedied through supplementing mortality data by standardized health care data on CVD morbidity and national patient cohorts that offer insights in more details of treatment outcome and risk potential. Current CVD research has up to date rely on limited patient cohorts across EU member states (as in SCORE; EUROASPIRE) but draw major recommendation from their analysis that become evidence-based standard for 506 million Europeans. This may underestimate population specifics in some risk factors (as genetics), national public health capacities or CVD treatment organization and the political framework of CVD awareness in the population. There is a need to consider success stories in CVD mortality reduction as Slovenia, Spain or the Netherlands in more detail and measure findings against current valid state of the art knowledge.

There should be concern to raise patient awareness for potential of CVD risk management targets. The potential of CVDs risk management is not fully exploited because a) depleting emotionality in patients to change their behavior in order to reduce CVDs risk in primary but especially in secondary prevention and b) clinicians may not find the time for comprehensive and regular risk detection screenings for coronary patients as prescribed by the ESC guidelines. High CVD risk patients may need further professional help as CVDs rehabilitation therapy to break with behavioral patterns, e.g. smoking cessation. Additionally, the persistent higher CVD prevalence in women should also be gain more attention. There is research need to find attractive solutions for the adaption of the ESC prevention guidelines to cultural variations throughout Europe.

There should be more focus on heart failure. Our map shows that research funding is mostly focused on cerebrovascular and hypertensive diseases. Even though, these categories are the main driver of CVD mortality in Europe, heart failure (ICD code I50) is increasing at a slow but steady pace since the 1990s. Chronic heart failure can be treated in only limited extend avoiding a fatal CVD event, including reduction in fluid intake and drug therapy. However, there is research need in new pharmacological tools strengthening the heart muscle.

The EU funding plays a pivotal role for CVD research, posing the second highest investment source after the UK. However, an analysis of funded projects is also concentrated in particular diseases and research settings. The design for research funding by the EU should be further reformed towards more accessibility by lowering formal barriers, sustainability by allowing for follow-up processes as databases and research dissemination in a more systematic manner. Additional, regional concentration bias favor countries as Germany and the UK, that have already access to a multitude of national research investment schemes, whereas Eastern European countries have a persistent CVD burden and comparatively low research efforts. Additionally, public research investment should follow decreasing investment by main private sectors very closely.

Research Translation from bench to bedside: Funders need to consider the long period of CVD treatment innovation to be implemented in daily practice in various phases with different actors and interests. Funders should be advised to employ a broader set of output indicators for their investment. For example, they might become concerned to measure outputs in terms of the translation process from scientific articles to public/political initiative and medical interventions. Making funding decisions solely based on the quantity of produced research papers as well as the quality (i.e. publications in high impact journals) may not be an effective measure of research benefit for the future and only trigger short-sighted research efforts. Funding decisions may benefit from a deeper knowledge of processes that help to transfer scientific knowledge to the public domain.

WP3
WP3 produced findings at two levels: (a.) the Strategic Level, which involves a non-exhaustive set of recommendations regarding gaps, overlaps and potential synergies for the future funding of NCD research; and (b.) the Tactical Level, which involves retrospective observations about the scale, scope and an impact of European investments in CRD research. These recommendations must be regarded as non-exhaustive, and subject to revision as the field of CRD research necessarily shifts and advances. This WP makes three principal strategic recommendations:

• Stratified Medicine: Future investments in CRD research should develop connections with the practice of stratified medicine, which involves subdividing patients into groups based on their responses to therapies or their risk profile for developing certain disease conditions. Under this model, future CRD research investments should be concerned with early diagnosis and predicting the onset of disease using biological markers that divide patients in to groups for the purpose treating CRDs much earlier.
• New Collaborations: Future investments in CRD research should involve large collaborations that consolidate data resources from across Europe and build a ‘team science’, or ‘big science’, approach to the conduct of CRD research. These collaborations should also involve industry. With increasing costs of developing new therapies, public sector research organizations should conduct the basic science of CRD research partnering with industry to deliver therapeutic advances. These collaborations should involve careful consideration and negotiations for the sharing of Intellectual Properties Rights.

• Broad Portfolio of Research: Future investments in CRD research should take account of donor perspectives and involve a broad portfolio of research that delivers both short term gains and long term goals. The CRD investment pipeline should involve five year cycles, which accommodate policy making frameworks. These cycles should be even distributed between: short term investments that translate into therapies within the next five years; and long terms investments which translate into therapies beyond the next five years.

At the Tactical Level, the WP produced 12 recommendations:

1. The study identified 131 RFOs for CRDs across the European area. Many of these RFOs are active funders of NCD research in other disease areas (n=119). The study identified 38 RFOs with an annual spend of above € 1.0 million. It also identified 13 RFOs with an annual spend below the € 0.1 million threshold, the majority of which were from smaller European countries.
2. Currently funded CRD research projects have an average life span of about 28 months. They generally focus on therapeutic and drug development in relation to the major diseases within CRD. Most currently funded research focuses on asthma (32%) with COPD (24%) and cystic fibrosis (14%) following behind.
3. Typically, RFOs fund research conducted at universities or university research centers. They do not usually fund private sector organizations, or even large consortiums of public sector research centers. In fact, the EC is perhaps the only RFO that sponsors large cross-border collaborations.
4. In Europe, private pharmaceutical companies have generally increased their commitment to R&D over the past four years. However, the sector demonstrates a limited commitment to investment in CRD research. Only five European pharmaceutical companies currently active in the development of New Molecular Entities (NMEs) for CRDs.
5. The Medical Devices Industry provides a mixed picture in relation to investment in CRD research. Broadly, industry’s commitment to R&D investment remained steady over the past four years but advancements were limited to individual companies. Most companies in the sector had not developed any CRD relevant medical devices.
6. The great majority of European RFOs are national public sector organizations that make funding investments at the national level (n=60). There are no private sector RFOs operating at the international level, and very few public sector RFOs making investments at the international level. The overwhelming picture generated by these results suggests that CRD research is being conducted at the national level on the basis of national level priorities.
7. Government was the major funding source for 72% of RFOs across the European Area. By comparison, other funding sources, such as the private sector and general fundraising activities played surprisingly minimal roles.
8. In terms of research output, CRDs are a very small subject area by comparison with other NCDs, and even in Europe only averages 0.8% of the papers in biomedicine overall. In terms of publications, the representation for major CRD areas is very unequal, with asthma representing over 40% of the total, and bronchiectasis barely 1%. COPD measures about 24%.
9. In terms of research output, some European countries are doing well and other less well. For example, the UK has the highest research output for CRD investment, more than twice as high as the second country, France, which is publishing almost twice as much as expected given their relative wealth. Similarly, Sweden and the Netherlands are also publishing about as twice as much as expected. On the other hand, Austria is publishing very little, and Germany, Norway and Switzerland are doing barely half of what might be expected from their wealth.
10. In terms of pharmaceuticals, there limited publications for CRDs across the private sector with the exception of Novartis, which seems to dominate CRD research, and is responsible for 79.4% of all scientific papers. In terms of medical devices, bibliometric analysis also determined that publication of scientific papers was limited. Again, Novartis was responsible for the overwhelming majority of papers (76%) with Covidien claiming a 5.7% share, Johnson and Johnson 3.0%, Koninklijke Philips NV 2.8% and Siemens 2.6%.
11. Basic strategies for the prevention and management of CRDs have found practical expression in a variety of policy initiatives. Many of these initiatives have been successful, but they are not usually part of a coordinated action plan. Ideally, the basic strategies for the treatment of CRDs should feed into a coordinated policy program involving all of the following four themes: emission standards, pharmaceutical therapies, disease management and regulatory controls for tobacco.
12. In addition to the thematic programs, policy makers need to continue to support research funding needs in the general area of CRDs. Broadly, key research needs for CRDs include: Modelling and Projections of Disease Burden, Triggers for Asthma, Interaction of Risk Factors for COPD and Regulatory Controls for Tobacco

WP4
In Europe, diabetes, in particular type 2 diabetes, is mainly associated with obesity, unhealthy lifestyle and socio-economic inequalities. Policy-makers have engaged in funding research for understanding the etiology and genetic basis of the disease, developing interventions and prevention strategies to mitigate the impact of diabetes as a public health challenge. Our map highlights a number of important facts that allows for the development of a non-exhaustive set of recommendations regarding the future funding and research focus for DIAB, which are necessarily subject to revision as the field shifts and advances.

Tackling overspecialization through new forms of collaborations
In this study, both the survey-based analysis and the bibliometric tool have shown that although the magnitude of diabetes research in Europe is increasing over the years, there is a heterogeneity across MSs in terms of funding amounts and scientific production. The latter aspect concerns both heterogeneity in volumes (with Scandinavian countries for instance exhibiting higher scientific production than expected) as well as topics of research, with Northern European countries devoting relatively more attention to Type I diabetes, and Southern countries on Type II. This suggests that policy agenda are different across countries perhaps due to different burden of diseases or specialization of the research investigators and apparatus. In order to respond to overspecialization of research practices in different countries exchange experiences between academic and research centers could be helpful in widening the research skills and portfolio for teams across the EU.

Partnership in research funding
Differences also lie in the number and type of organizations providing funding, with Northern and Southern countries exhibiting a prevalence of public sector and non-profit organizations, while activities in Eastern Europe are mostly funded by public RFOs. Finally, Central Europe is characterized by a more balanced distribution of RFO types. Scandinavian countries have more private-non-profit sources, especially endowed foundations. Rather few of their DIABE papers do not report a funding acknowledgement, explicit or implicit. Czech Republic, France and Spain are notable for the high percentage of their papers explicitly funded by the public sector. Co-financing and partnerships seem a good way to approach the funding of research in a disease area that is multidimensional. The European Union is the largest single source of support in terms of the numbers of papers funded, with the second largest being the Danish company Novo Nordisk A/S. The remaining are mainly government agencies, with the exception of the Juvenile Diabetes Research Foundation, the Wellcome Trust and Diabetes UK. Compared to the US, European based companies have a stronger pipeline of products to address diabetes. However, our continent has declined in relative terms to the rest of the world when considering outputs of diabetes research papers. This is not necessarily a problem if a wider concept of impact of research is taken where the public and patients can express their preferences and assess whether the output of research has actually achieved something in terms of overall societal benefit.

Etiology, prevention and infrastructure for genetic-based studies
In terms of future research priorities, probably etiology, prevention and treatment improvement will be the areas of major interest. At the same time, bio-banks and registry development would be useful to speed up genetic-based and epidemiological studies. Social and health related quality of life aspects of people living with the disease, will call for multidisciplinary research projects that are necessary to tackle a disease that is more complex than the genetics behind it.

The dissemination plan for this research project has included or will include the following activities:
• Public website for the project (see below)
• Public workshop, presentation of results by partners in the project (4th December 2015, Berlin, TUB): around 40 experts, representatives of RFOs and academia attended the workshop. An informative brochure reporting the key findings of the project was handed out to participants.
• Conference presentation: the presentation “A system-wide perspective on funding for biomedical research in Italy” was given during the Annual Conference of the Italian Association of Health Economists (Alghero, 16th October 2015)

Scientific publications currently under way:
1. "A system-wide perspective on funding for biomedical research in Italy" - target journal Research policy through a coordinated submission by partners
2. "European NCDs research on diabetes 2002-13: bibliometric study of outputs and funding" - joint UB and KINGS paper, target journal to be defined
3. "Diabetes research agenda in Europe: lessons from a mixed method study" - UB team, target journal to be defined
WP5
Our map of European Cancer research funding shows that RFOs consider published research as key evidence of research impact. For this reason, the volume of publications and citation levels serves as a particularly measure of research activity within different types of cancers. Overall, cancer generated the largest of the five results for NCDs in Europe and worldwide. By comparison with the other NCDs, published research for Cancer shows a significant European presence, among the top three with an average of 38% of paper published, behind RESPI (56%) and CARDI (42%) and followed by 40% for DIABE and 35% for MENTH.

Germany, Italy, France and the UK are the most active European MSs in cancer research by volume of research output, and Cyprus and Malta are the least. However, some of the smaller countries are expanding their output rapidly – notably Romania, whose fractional count output rose from only 7 papers in 2002 to over 250 in 2013. Cancer research seems to focus on colo-rectal and breast cancer. They type of research is mostly prognosis, genes and chemotherapy, with the least done on surgery and palliative care. Tracking the correlation of relative disease burden to individual European countries, we find that the relationship is poor. In most European countries, the level of disease burden does not reflect the level of research activity measure in terms of published scientific outputs. Indeed, one such example is that high relative burden from melanoma in Scandinavia has not led to an above-average concentration of research effort on this manifestation of cancer.
In terms of medical devices, very few companies had produced research outputs in terms of scientific papers. Novartis was responsible for the overwhelming majority of papers (65.3%) with Jansen Pharmaceutical claiming an 8.0% share, Johnson and Johnson 6.4% and Siemens 5.2%.

RFOs, like the EC, are investing considerable resources in cancer research. Under FP6 and FP7, the EC has invested close to €1 billion in about 100 projects related to cancer research, which developed large consortiums of researchers and focused on genomics, biotechnology, preventive prognostic, diagnostic and therapeutic interventions. At national level, RFOs tended to focus on therapeutic and drug development, and did not fund collaborative multicenter consortiums. Most of the recipients of funding were universities or research centers (around 90%).

Our map also highlights some import facts concerning the activities of the private sector. In general, European pharmaceutical companies show a high degree of commitment to cancer research, with seven of the top ten European firms having New Molecular Entities (NMEs) for Cancer under development. At present, companies like Astrazeneca and Novartis, are most active in relation to Cancer with n=27 and n=25 relevant products currently under development. However, only seven of the world’s top 16 top medical device manufacturers are currently developing products in relation to cancer. Between 2011 and 2015, a total of 346 clinical were conducted for cancer corresponding to 4.62% of the total (16 clinical trials). Covidien has undertaken the most clinical trials, largely in relation to breast cancer and bone cancer, both for diagnostic and therapeutic purposes.

Although Europe has nine companies in the world top 20 pharmaceutical companies by investment in R&D, there are only five European pharmaceutical companies currently active in the development of New Molecular Entities (NMEs) for cancers. There are also few cancer relevant medical devices currently under development. Of the world’s top 16 Medical Device companies by R&D Investment, only five are active in developing new products for cancers. Of these five, none are developing products relevant to COPD, the major disease category for cancers; and only two companies are developing products relevant to asthma (n=4).

Our map also emphasizes some important characteristics of Cancer itself. If detected early, between half and a third of cancers are treatable. Moreover, with the promotion of lifestyle behaviors that mitigate specific risk factors, most cancers are indeed preventable. Like other NCDs, these risk factors include: tobacco smoking, alcohol consumption, lack of physical activity, and unhealthy diet. However, cancer is also much more diverse than other NCDs, which requires a multifaceted and geographically targeted prevention policies, rather than a uniform approach that focus on a limited set of behavioral risks.

For the future, our map suggests that funding investments in cancer should aim to increase funds available for cancer research and also decrease the administrative burdens associated with applying for and securing funds. RFOs should promote networks and cooperative relationships between researchers, which might include establishing networks of comprehensive cancer centres throughout Europe, and encouraging cooperation between basic translational and clinical researchers. There is a lack of dedicated calls for cancer research. Data resources are a challenges to future cancer research in Europe especially if amendments proposed to the European General Data Protection Regulation become obligatory by law. The main concern is how to manipulate all these data, how to store, analyse, share and understanding huge quantities of data collected. There is a need for IT professionals and specifically in bio-nformatics. Finally, RFOs also should endeavor to set cancer as national priority for research, with special focus on independent research, personal cancer medicine, multinational research infrastructures and observational studies.

WP6
WP6 mapped European MHD research activity through multiple measures, developing a multi-focal view of the current research funding landscape and identifying unmet needs, research overlaps and potential synergies to enable development of effective research funding strategies. Results were presented at a workshop in December 2015 and will be further disseminated through scientific publications.
Findings
The majority of MHD RFOs were public and received government funding. The level of involvement of charitable organizations in MHD research is low compared to other NCDs, such as cancer.
Analysis of the top-funded MHD projects identified in the five largest European Union (EU) Member States (MSs) showed that Alzheimer’s disease and schizophrenia received the bulk of the funding, followed by depression and to a much lesser extent anxiety. Identified national-level projects were not collaborative, although the largest MSs were highly involved in collaborative projects at the EU level. Smaller MSs have much lower participation in EU projects.
The level and extent of research and development (R&D) in drugs targeting MHDs has diminished in recent years, both in Europe and the US, due to the long, expensive and uncertain development process and the lack of identified treatment targets. An exception is Alzheimer’s disease, where the largest pharmaceutical companies had ongoing development activities. Moreover, smaller pharmaceutical companies are active in MHDs, including for MHDs that received low or no investment by the top 20 companies.
R&D investment in MHDs by medical device companies was not extensive, although recent innovations include different forms of transcranial stimulation. In addition, imaging devices are being used to identify treatment targets and for diagnostic purposes, with the development of tracers often undertaken by pharmaceutical companies. Other device innovations are being developed outside of the traditional medical device industry, including mHealth applications, which may provide expanded access and minimize issues of stigma. Additional research is needed in this area to provide a more solid evidence base, and the regulatory status of such applications must be clarified, with a particular focus on ensuring patient confidentiality. Beyond developments in drug- and device-related therapies, it is important to consider the role of psychological therapies for MHDs, which may be used alone or as an adjunct to other therapeutic modalities. A solid evidence base must be developed for these therapies, requiring increased investment.
MHD research outputs in terms of scientific publications by researchers in the 31 European included in our study account for almost 6% of all biomedical research and 40% of MHD research worldwide. Depression, Alzheimer’s disease and schizophrenia papers comprised 57% of the output for the years 2002-2013 among 10 MHDs. Some relatively high prevalence MHDs, including anxiety and alcohol abuse disorders, receive only a small share of research attention. Nonetheless, prevalence should not be the only metric considered in setting research priorities. Other factors, including treatment gaps, cost of illness and potential return on investment, should be part of the decision process.
Recommendations
1. The need to encourage and facilitate research collaboration at the European Level: Multiple programs, bodies and platforms have been established at the European level in recent years and have contributed to increased cross-national research collaborations. However, participation by some smaller countries has been very low or non-existent. Moreover, existing structures do not facilitate participation by certain key stakeholders, including patients and caregivers, in setting research priorities. The ROAMER project provided a model for this type of broad participative process. The EC should explore existing structures to determine whether there are barriers to participation that may be addressed and consider whether a neutral platform would encourage broader participation by MSs and the full range of stakeholders.
2. Support expanded MHD research participation by less traditional participants: We found that charitable RFOs, smaller pharmaceutical companies and developers of eHeath and mobile devices played important roles in funding research and development of innovative MHDs therapies, particularly for diseases that receive less attention from larger institutional research entities. The EC should explore how best to support and expand such research activity.
3. Research priorities for larger RFOs: Identification of biomarkers for MHDs is essential to improving diagnosis and screening and developing targeted treatments. With the withdrawal of major pharmaceutical firms from MHD drug development, public RFOs must expand their investment in fundamental research. This also implicates cross-national collaboration and coordination to avoid redundancy and to maximize the potential uses of existing data. While the EC already constitutes the key facilitator of European research collaboration, it should consider how to strengthen and further support breakthroughs in this essential aspect of MHD research.

WP7
Bibliometric filters developed in WP7 were used to determine funding sources for and the impact of NCD research in the EU. In this way, WP7 identified and assessed the impact of health programs and research activities above the specified funding threshold in each disease area. In each area, WP7 addressed whether or not the new investments built upon or represented significant improvements relative to existing or previous investments. Following on from that, results and impacts were grouped into categories, including (a) the level of unmet need within the disease area that the investment is addressing, (b) the severity of the condition or risk factor the investment addresses, (c) the amount and quality of available evidence supporting the investment in the condition, aspect or risk factor, and (d) the extent to which expert opinion is available and has influenced the investment.
1 Results
1.1 Cardiovascular research, including stroke (CARDI)
• the disease burden in individual countries is very varied, being as high as 37% of the total in Bulgaria but only 13% in France
• overall, CARDI receives somewhat less research attention than the burden would indicate
• Greece and the Netherlands publish more than expected relative to their wealth, but Bulgaria and Romania very much less
• there is a lack of research on ischaemic heart disease including myocardial infarction although it is the biggest component of CVD burden
• in terms of citation impact, the top countries are the Netherlands, the UK and Denmark
• the EUR31 countries are estimated to have spent €4.6 billion p.a. on CARDI in 2009-13.
1.2 Diabetes research (DIABE)
• the disease burden is again very varied, highest in Cyprus (4.3%) and lowest in the UK (1.2%), but it is increasing in most countries particularly type II
• relative to the burden, DIABE was slightly over-researched but this seems a sensible precaution in view of the increase in incidence
• Denmark, Finland, Sweden and the UK published more than expected, but Norway and Romania much less in comparison with their wealth
• the UK's leading position seems strange in view of its low incidence as measured in two separate ways (disease burden in DALYs and incidence)
• type II diabetes attracted much more research (33% of all papers) than type I (14%)
• of the sequelae, cardiovascular complications were the most researched (14%), followed by nephropathy (7%) and retinopathy (4%); effects on feet much less (2.3%)
• in terms of citation impact, the top countries were Finland, Switzerland, Denmark and the UK
• the EUR31 countries are estimated to have spent €0.9 billion p.a. on DIABE in 2009-13
• the leading sources of funding were government (€228 m/yr, 26% of papers), private-non-profit RFOs (€165 m/yr, 19%), industry (€156 m/yr, 18%) and the EU (€26 m/yr, 3%). Institutional (non-acknowledged) funding provided 37% of support
• the leading industrial funders were Novo Nordisk A/S (€18 m/yr, 2%), Sanofi-Aventis s.a. (€8.5 m/yr, 1%) and Eli Lilly Inc. (€6.3 m/yr, 0.7%).
1.3 Mental disorders research (MENTH)
• the disease burden from mental disorders (including suicide and self-harm) averages 14.3% in the EUR31 countries. It tends to be relatively higher in the more affluent ones: Luxembourg 19.9%, Norway 19.4%, but Bulgaria is very low at 8.6%
• the amount of research overall seems appropriate, but too little is done in suicide and self-harm, alcoholism, unipolar depression and anxiety. Schizophrenia appears over-researched, as are the dementias, perhaps because of public concern over their rising social costs
• relative to their wealth, Croatia, the Netherlands, Finland, Sweden and the UK published about twice as much as expected, but Slovakia, Romania and Bulgaria did less than half as much
• in terms of citation impact, the leading countries were Ireland, Finland and Austria in the SCI, and Belgium, Austria, the Netherlands and France in the Social Sciences index
• the EUR31 countries are estimated to have spent €3.3 billion p.a. on MENTH in 2009-13
1.4 Oncology research (cancer, ONCOL)
• the disease burden from cancer in the EUR31 countries varies relatively little, from 19.1% in France and 18.5% in Italy down to 13.0% in Latvia and Cyprus
• cancer is well-researched compared to the other four NCDs in relation to its disease burden
• the leading type of cancer research is genetics (19%), followed by chemotherapy, prognosis and surgery (11%). Radiotherapy accounts for less than 5% of ONCOL papers, and there is very little work on palliative care and quality of life except in Scandinavia and the UK
• there is rather low correlation between the disease burden from cancer sites and the corresponding research output. Breast cancer appears over-researched, as does leukaemia, but lung and pancreatic cancers need more research
• relative to their wealth, the best-performing countries are Iceland, Croatia and Slovenia, but Luxembourg, Latvia, Cyprus, Romania and France publish fewer papers
• in terms of citations, the best-performing countries were Switzerland, the Netherlands, the UK and Iceland
• the references on cancer clinical guidelines (for lung, breast and colorectal cancers) included many on surgery and radiotherapy as well as chemotherapy, but few on genetics
• the EUR31 countries are estimated to have spent €6.2 billion p.a. on ONCOL in 2009-13
1.5 Respiratory disease research (RESPI)
• this NCD is seriously under-researched in relation to its disease burden. If it were to be funded relatively on the same scale as cancer its budget would need to be quadrupled. Europe nevertheless contributes 56% of world output, compared with 40% in other NCDs
• within RESPI, COPD is treated less well than asthma, perhaps because it usually occurs in old men who smoke, whereas asthma particularly affects children. However COPD papers are well cited compared with those in asthma and cystic fibrosis
• the UK published much the most papers, more than the next two countries (France and Italy) combined. Sweden and the Netherlands also published almost twice the numbers expected from their wealth. Austria and Germany published relatively little on RESPI
• Ireland was very strong in cystic fibrosis research
• the EUR31 countries are estimated to have spent €0.42 billion p.a. on RESPI in 2009-13
• the leading source of funding was from national and regional governments (€0.09 billion, 21%), followed by PNP sources (€0.08 billion, 19%) and industry (€0.065 billion, 15%). The EU provided €0.012 billion and was a close second to GlaxoSmithKline plc as a funder.
2 Impacts
2.1 National health advisory committes
• the research experience of members (data for 21 countries) parallels EUR31 outputs, but departs more from the pattern of disease burden, so that existing disparities are likely to be reinforced rather than ameliorated
• the members are well-connected with researchers in other EUR31 countries, so it is likely that European research will be influential on other countries' health policies
2.2 Newspaper stories and the papers they cite
• the stories are dominated by cancer (about 50% of the total) and mainly cover its causes (epidemiology and genetics) rather than its treatment
• mental disorders are also well covered (about 25%), with much attention paid to the dementias and depression, but little to alcoholism
• most newspapers over-cite own-country research, but there are many stories about UK research
3 Recommendations arising from the findings
3.1 Funding
• The biggest need is for more funding for respiratory diseases, especially COPD
• In all five NCDs, there are many hundreds or even thousands of small funders, both charities and foundations, and commercial firms. It would be desirable to encourage the formation of umbrella bodies, such as the UK's Association of Medical Research Charities, in other MS. They could co-ordinate policies, provide information and lobby for better fiscal treatment.
• Some research activities are relatively neglected. In ONCOL it is radiotherapy and palliative care, and work on lung and pancreatic cancer; in CARDI it is ischaemic disease and myocardial infarction; in MENTH it is depression and alcoholism; in RESPI it is COPD. Means need to be developed to make funders aware of the need to change their priorities to make them accord better with the burdens of disease.
• Some types of research may require quite different types of support in order to be translated into better clinical care. For example, short trans-national visits (from a few weeks to a few months) may prove very useful as a way to spread best practice, especially in surgery and radiotherapy for cancer, and for cognitive behavioural therapy in mental disorders.
• More work is needed on alternative impact indicators, such as citations on clinical guidelines and in media stories. These should be incorporated into research submissions so that funders are made aware of the practical impacts of previously-supported work by applicants. Because there is no public database of these citations (as there are for journal papers and academic citations, namely the WoS and SCOPUS), it would be desirable to develop them, starting with the data already established during this project.
List of Websites:
www.ncd-map.eu

Documents connexes

final1-mapping_ncd_final-report_final.pdf

Final Report Summary - MAPPING_NCD (Mapping Chronic Non-Communicable Diseases Research Activities and their Impact)

Documents connexes

Partager cette page

Télécharger