"The interventions and measures for which the EFFORTI toolbox aims to provide evaluation support are embedded in different contexts with regard to socio-economic and political systems and organizational settings and thus develop differently according to their context. For this purpose, seven country notes have been written (AT, DK, FR, DE, HU, ES, SWE) where the contextual framework conditions like the structure and performance of the innovation Systems and the governance and relevance of gender equality have been described. These reports also mapped existing evaluations of initiatives to promote gender equality and the dominant evaluation culture in each country.
In order to develop the conceptual evaluation framework, the study team carried out a comprehensive desk research as a basis for the collection of a preliminary list of relevant indicators. Particular emphasis was given to map existing evaluation concepts of GE measures and instruments, concepts for the measurement of research and innovation outputs, approaches for impact assessment and studies how to monitor RRI. Based on existing evidence, the project team undertook the following steps: (1) Identifying the most relevant indicators suggested by the literature review, (2) Clustering these indicators into different categories, dimensions and sub-dimensions. (3) Substantiating and conceptualising these indicators according to an evaluation logic model differentiating between input, (throughput,) output, outcome and impact aspects. (4) The indicators were illustrated at micro/individual or team level, meso/organisational level and macro/policy or country level. Contextual indicators were treated as cross-cutting indicators which are assumed to have relevant influence on indicators from input to outcome. Finally, the project carried out 19 case studies distributed over six countries. The case studies validated the evaluation framework in different empirical settings and tested its applicability for different types of equality measures in three main ways 1) theory based evaluation: theory of change, 2) validation of indicators, 3) impact stories.
The overall EFFORTI Toolkit v2.0 is deployed in the official website
https://www.efforti.eu(opens in new window) and provides (a) a detailed overview of the EFFORTI evaluation framework which is a critical aspect of the project, applied to all tools and a constant reference point for the end-user, and (b) access to the three core tools and services of the toolkit. These core tools are: The Impact Story Knowledge Base; The Programme Theory Generator and The Evaluation Framework. The toolbox captures in a systematic and visually appealing manner the work undertaken throughout the project and can be used by users to design their own evaluations based on the theoretical background and the methodology provided by the EFFORTI project. At the same time the visual illustration of the indicators allows to make connections that are not self-evident and thus might allow to further experimentation in specific cases.
Overall, the project team published eight project reports and eleven peer-reviewed scientific articles, among them ten in a special issue of the high-ranked journal ""Evaluation and Program Planning"". The team also presented EFFORTI at 17 national and international conferences. Furthermore, the EFFORTI team compiled three summary papers that contain the most important results of the context analysis, the coneptual evaluation framework and the case study work, wrote 17 so-called impact stories as ideal-type impact pathways for gender equality interventions and visualized these impact stories via log frame pictures. Finally, the EFFORTI team summarized its main lessons learned and key recommendations and gave an overview on the main performance effects of gender diversity."
