Periodic Reporting for period 2 - REITER 3 (SUPPORT TO THE DEVELOPMENT AND ANALYSIS OF A RESEARCH AND INNOVATION POLICY TAXONOMY AND QUESTIONNAIRE – PHASE 3)
Reporting period: 2022-02-01 to 2022-07-31
REITER 3 is the third phase of a project that originally started in 2016 and that has the overarching goal of contributing to the monitoring and assessment of research and innovation (R&I) policies in Europe and beyond. It does this by surveying countries on their R&I policies, linking the collected policy data with other information (including statistics and other databases), and providing open data services through a dedicated web portal, the STIP Compass (http://stip.oecd.org(opens in new window)). The objectives of REITER 3 are:
(1) Maintaining and updating the core infrastructure of STIP Compass
• Updating and improving the quality of STIP Compass R&I policy data.
• Linking STIP Compass policy data with other sources of data.
• Relaunching the STIP Compass website with enhanced services and graphical user interfaces.
(2) Expanding the uses and user base for STIP Compass through a focus on specific topics
• Establishing a new STIP Data Lab, which will undertake various types of analysis of the STIP Compass R&I policy data using R, Python, Stata and other statistical packages.
• Launching online portals, which will operate as interactive repositories that gather R&I policy data and analyses on selected specific high-priority topics.
(1) Maintaining and updating the core infrastructure of STIP Compass
• Updating and improving the quality of STIP Compass R&I policy data.
• Linking STIP Compass policy data with other sources of data.
• Relaunching the STIP Compass website with enhanced services and graphical user interfaces.
(2) Expanding the uses and user base for STIP Compass through a focus on specific topics
• Establishing a new STIP Data Lab, which will undertake various types of analysis of the STIP Compass R&I policy data using R, Python, Stata and other statistical packages.
• Launching online portals, which will operate as interactive repositories that gather R&I policy data and analyses on selected specific high-priority topics.
Work Package 1: Evaluation of STIP Compass
• Established an Evaluation Steering Group and agree on its terms of reference and the scope of the evaluation.
• Ran a short survey of existing and potential STIP Compass users and contributors.
• Conducted interviews with selected users and contributors to learn more about how different groups perceived STIP Compass, in terms of its current and potential costs and utility.
• Used web traffic statistics to monitor how the site is used and whether this has changed over time.
• Benchmarked STIP Compass web traffic statistics and dataset indicators against similar policy databases.
• Prepared a final evaluation report, drawing on the various strands of evidence collected and following the guidance given by the Evaluation Steering Group.
Work package 2: Updating and improving the quality of STIP Compass R&I policy data
• The project team ran a consultation process involving OECD and EC analysts and OECD delegates to ensure questions and survey taxonomies were adequate for 2020-22 project needs.
• Deployed the STIP data model corresponding to the 2021 edition of the survey in the LimeSurvey online survey administration software and Marklogic database.
• Launched and administered the 2021 EC-OECD STIP survey.
• Curated the policy data collected in the 2021 survey.
• Analysed the data collected in the 2021 survey in two reports (D2.3 and D4.2).
• Re-opened the monitoring tool to allow participating countries and territories to update their data for the remainder of the REITER 3 project’s duration.
Work Package 3: Linking STIP Compass policy data with other sources of data
• Updated and extended the indicators hosted within the STI.Scoreboard statistical infrastructure, and further embedded them across STIP Compass.
• Maintained and updated existing links to linked qualitative data.
• Implemented Open Alex as a replacement open catalogue of worldwide scholarly papers.
• Updated the innovation policy vocabulary to incorporate new OECD and EC concepts.
• Organised a hybrid workshop on linked data where representatives from relevant R&I data initiatives presented their initiatives, data quality issues, data cycles and data accessibility (e.g. APIs).
Work package 4: STIP Data Lab: Producing analytical content from STIP Compass data
• Conducted exploratory analyses, mapping the available policy evidence collected in STIP Compass.
• Ran experimental analyses to explore the possibilities offered by STIP Compass data models in view of developing new graphical user interfaces.
• Prepared ten data stories to surface the value of the STIP Compass database and its use cases.
• Analysed the state-of-the-art in knowledge management systems for R&I policy and its implications for future rounds of REITER.
Work Package 5: Relaunch STIP Compass with enhanced services and graphical user interfaces
• Completely redesigned the online portal, to consolidate the functions that have been gradually introduced since the launch of STIP Compass in 2018.
Work package 6: Develop specific topic portals to extend the uses of STIP Compass
• Developed a template for thematic portals following the site’s re-design.
• Relaunched the STIP Covid-19 Watch thematic portal under the new template.
• Launched a beta version of the “STI policies for net zero” thematic portal to coincide with the COP26 meeting in Glasgow.
• Launched thematic portals on “space policies” and “research security”.
• Established an Evaluation Steering Group and agree on its terms of reference and the scope of the evaluation.
• Ran a short survey of existing and potential STIP Compass users and contributors.
• Conducted interviews with selected users and contributors to learn more about how different groups perceived STIP Compass, in terms of its current and potential costs and utility.
• Used web traffic statistics to monitor how the site is used and whether this has changed over time.
• Benchmarked STIP Compass web traffic statistics and dataset indicators against similar policy databases.
• Prepared a final evaluation report, drawing on the various strands of evidence collected and following the guidance given by the Evaluation Steering Group.
Work package 2: Updating and improving the quality of STIP Compass R&I policy data
• The project team ran a consultation process involving OECD and EC analysts and OECD delegates to ensure questions and survey taxonomies were adequate for 2020-22 project needs.
• Deployed the STIP data model corresponding to the 2021 edition of the survey in the LimeSurvey online survey administration software and Marklogic database.
• Launched and administered the 2021 EC-OECD STIP survey.
• Curated the policy data collected in the 2021 survey.
• Analysed the data collected in the 2021 survey in two reports (D2.3 and D4.2).
• Re-opened the monitoring tool to allow participating countries and territories to update their data for the remainder of the REITER 3 project’s duration.
Work Package 3: Linking STIP Compass policy data with other sources of data
• Updated and extended the indicators hosted within the STI.Scoreboard statistical infrastructure, and further embedded them across STIP Compass.
• Maintained and updated existing links to linked qualitative data.
• Implemented Open Alex as a replacement open catalogue of worldwide scholarly papers.
• Updated the innovation policy vocabulary to incorporate new OECD and EC concepts.
• Organised a hybrid workshop on linked data where representatives from relevant R&I data initiatives presented their initiatives, data quality issues, data cycles and data accessibility (e.g. APIs).
Work package 4: STIP Data Lab: Producing analytical content from STIP Compass data
• Conducted exploratory analyses, mapping the available policy evidence collected in STIP Compass.
• Ran experimental analyses to explore the possibilities offered by STIP Compass data models in view of developing new graphical user interfaces.
• Prepared ten data stories to surface the value of the STIP Compass database and its use cases.
• Analysed the state-of-the-art in knowledge management systems for R&I policy and its implications for future rounds of REITER.
Work Package 5: Relaunch STIP Compass with enhanced services and graphical user interfaces
• Completely redesigned the online portal, to consolidate the functions that have been gradually introduced since the launch of STIP Compass in 2018.
Work package 6: Develop specific topic portals to extend the uses of STIP Compass
• Developed a template for thematic portals following the site’s re-design.
• Relaunched the STIP Covid-19 Watch thematic portal under the new template.
• Launched a beta version of the “STI policies for net zero” thematic portal to coincide with the COP26 meeting in Glasgow.
• Launched thematic portals on “space policies” and “research security”.
The project is expected to have several impacts. At the highest level, it is expected to enhance the evidence base for R&I policies, which should in turn contribute to better national and European R&I policy and programme design. More specifically, the project is expected to:
(1) Facilitate access for government officials, policy analysts and academics to up-to-date information on how countries design policies and deploy policy instruments for R&I in various contexts. REITER 3 will update and upgrade STIP survey data by providing tools to measure its quality and enforce standards. Through these measures, the project’s users will rely on STIP Compass as an open source of harmonised R&I policy data.
(2) Provide a first source of evidence for the EC’s European Semester process and for projects led by the OECD. The STIP Compass database covers a comprehensive range of policy themes that are well aligned with the priorities of both organisations. The secretariats will use it to obtain an initial understanding of the current policy landscape for projects, identify possible case studies, and scope requirements for additional data collection.
(3) Facilitate policy learning through re-designed and new interactive interfaces. Country dashboards, the policy explorer and the query builder provide various ways to explore STIP survey data and linked data. Users will be able to generate printable reports summarising the policy data around a given topic for selected countries. Topic portals will provide interactive repositories of available R&I data and evidence on selected R&I issues.
(4) Generate useful evidence and analyses for national and European R&I policy. The STIP Data Lab will exploit the data using new analytical capabilities that provide exemplars of the data’s analysis potential. It will do so engaging country officials and experts in the R&I field, thereby building a community of STIP Compass expert users.
(5) Inform academics and other policy analysts tackling research questions in the field of R&I policy. The various types of policy data found in STIP Compass is a unique resource that provides a wide range of opportunities to conduct evidence-based research projects.
(6) Openly publishes the latest data model and databases used in the project, that analysts and policy makers are able to able to easily call up using dedicated open APIs. National knowledge management systems can also use them to structure data in their own data infrastructures. This can save them considerable time and effort and supports the interoperability of their systems.
(7) Further exploited the data using new analytical capabilities to generate useful evidence for national and European R&I policy.
(1) Facilitate access for government officials, policy analysts and academics to up-to-date information on how countries design policies and deploy policy instruments for R&I in various contexts. REITER 3 will update and upgrade STIP survey data by providing tools to measure its quality and enforce standards. Through these measures, the project’s users will rely on STIP Compass as an open source of harmonised R&I policy data.
(2) Provide a first source of evidence for the EC’s European Semester process and for projects led by the OECD. The STIP Compass database covers a comprehensive range of policy themes that are well aligned with the priorities of both organisations. The secretariats will use it to obtain an initial understanding of the current policy landscape for projects, identify possible case studies, and scope requirements for additional data collection.
(3) Facilitate policy learning through re-designed and new interactive interfaces. Country dashboards, the policy explorer and the query builder provide various ways to explore STIP survey data and linked data. Users will be able to generate printable reports summarising the policy data around a given topic for selected countries. Topic portals will provide interactive repositories of available R&I data and evidence on selected R&I issues.
(4) Generate useful evidence and analyses for national and European R&I policy. The STIP Data Lab will exploit the data using new analytical capabilities that provide exemplars of the data’s analysis potential. It will do so engaging country officials and experts in the R&I field, thereby building a community of STIP Compass expert users.
(5) Inform academics and other policy analysts tackling research questions in the field of R&I policy. The various types of policy data found in STIP Compass is a unique resource that provides a wide range of opportunities to conduct evidence-based research projects.
(6) Openly publishes the latest data model and databases used in the project, that analysts and policy makers are able to able to easily call up using dedicated open APIs. National knowledge management systems can also use them to structure data in their own data infrastructures. This can save them considerable time and effort and supports the interoperability of their systems.
(7) Further exploited the data using new analytical capabilities to generate useful evidence for national and European R&I policy.