Final Report Summary - EERQI (European Educational Research Quality Indicators)
The EERQI project was motivated by the fact that the international notion of scientific quality as being the main determinant on which research is funded and supported may cause undesired side effects if the questions of how quality is interpreted and how it is measured are not adequately answered. Current instruments for ‘measuring’ quality via citation counting and similar methods do cause such side effects, as they are strongly biased and largely inadequate for research in the SSH disciplines.
The EERQI project developed an approach to detect the quality of research texts – with educational research serving as model case – by applying an intelligent combination of different approaches that complement each other. This is what we call the EERQI Prototype Framework. It consists of products and methods that can serve as alternatives in processes of assessment of quality in SSH research. The possibility of multilingual assistance of assessment processed by EERQI’s multilingual search engine and automatic semantic analysis are tailor-made for strengthening the European research space. The EERQI products and methods consist of:
- A content base with educational research texts in the four European languages that were exemplary included in the EERQI project: English, German, French and Swedish.
- A multilingual search engine that includes query expansion: an effective tool dedicated to educational research in general, capable of finding educational research texts in the Web in the four EERQI languages.
- An automatic semantic analysis for the detection of key sentences in a text; the method is applicable to educational research publications (in at least) the four EERQI languages.
- A combination of bibliometric/ webometric approaches for the detection of ‘extrinsic’ quality indicators (tool aMeasure).
- First tests of a citation analysis method that has the potential to be further developed for the application to educational research (and other SSH) texts.
- A set of text-immanent indicators for the detection of quality in educational research publications that has been presented to the research community and was positively evaluated.
- An accompanying peer review questionnaire that was tested for reliability and practicality.
- A set of use-case scenarios that give advice on how to use which resp. combination of the above-mentioned tools.
- First attempts to detect interrelations between ‘extrinsic’ and ‘intrinsic’ quality indicators.
The EERQI Prototype Framework accompanies the process of quality detection:
- The process begins with the detection of potential quality via identification of relevant texts from different sources. The EERQI content base (educational research texts provided by the EERQI publisher partners) and the multilingual search and query engine are the relevant tools for this step.
- The application of ‘aMeasure’ - a stack of tools and programs to measure extrinsic characteristics of research publications (such as citations and Web mentions) – allows for collecting information about extrinsic indicators of the quality of publications.
- The application of automated semantic analysis provides support for evaluating the internal quality of a text. The method developed in EERQI allows for the identification of key sentences that indicate parts of documents to which peer reviewers should pay particular attention.
- The application of a Peer Review Questionnaire that contains a tested operationalization of the intrinsic indicators of quality that were developed by the EERQI project supports the readers’ final judgments on the quality of texts.
List of Websites: www.eerqi.eu
The EERQI project developed an approach to detect the quality of research texts – with educational research serving as model case – by applying an intelligent combination of different approaches that complement each other. This is what we call the EERQI Prototype Framework. It consists of products and methods that can serve as alternatives in processes of assessment of quality in SSH research. The possibility of multilingual assistance of assessment processed by EERQI’s multilingual search engine and automatic semantic analysis are tailor-made for strengthening the European research space. The EERQI products and methods consist of:
- A content base with educational research texts in the four European languages that were exemplary included in the EERQI project: English, German, French and Swedish.
- A multilingual search engine that includes query expansion: an effective tool dedicated to educational research in general, capable of finding educational research texts in the Web in the four EERQI languages.
- An automatic semantic analysis for the detection of key sentences in a text; the method is applicable to educational research publications (in at least) the four EERQI languages.
- A combination of bibliometric/ webometric approaches for the detection of ‘extrinsic’ quality indicators (tool aMeasure).
- First tests of a citation analysis method that has the potential to be further developed for the application to educational research (and other SSH) texts.
- A set of text-immanent indicators for the detection of quality in educational research publications that has been presented to the research community and was positively evaluated.
- An accompanying peer review questionnaire that was tested for reliability and practicality.
- A set of use-case scenarios that give advice on how to use which resp. combination of the above-mentioned tools.
- First attempts to detect interrelations between ‘extrinsic’ and ‘intrinsic’ quality indicators.
The EERQI Prototype Framework accompanies the process of quality detection:
- The process begins with the detection of potential quality via identification of relevant texts from different sources. The EERQI content base (educational research texts provided by the EERQI publisher partners) and the multilingual search and query engine are the relevant tools for this step.
- The application of ‘aMeasure’ - a stack of tools and programs to measure extrinsic characteristics of research publications (such as citations and Web mentions) – allows for collecting information about extrinsic indicators of the quality of publications.
- The application of automated semantic analysis provides support for evaluating the internal quality of a text. The method developed in EERQI allows for the identification of key sentences that indicate parts of documents to which peer reviewers should pay particular attention.
- The application of a Peer Review Questionnaire that contains a tested operationalization of the intrinsic indicators of quality that were developed by the EERQI project supports the readers’ final judgments on the quality of texts.
List of Websites: www.eerqi.eu
