Periodic Reporting for period 1 - WebMAT (Web-based system for multi-arrangement task for rapid acquisition of subjectivejudgements.)
Reporting period: 2016-04-01 to 2017-09-30
1.1 Summary of the context and overall objectives of the project (For the final period, include the conclusions of the action). This section should include information on: What is the problem/issue being addressed?; Why is it important for society? ; What are the overall objectives?
We set out to develop a technique for collecting continuous similarity judgments from human participants. Collecting this kind of human judgment typically takes a long time, because a judgment of similarity must be communicated for each pair of the items to be judged. For hundreds of items, there are many thousands of pairs, so conventional methods are very time consuming.
1.2 Work Performed from the beginning of the project to the end of the period covered by the report and main results achieved so far (For the final period, please include an overview of the results and their exploitation and dissemination.
We developed a new and more efficient method for collecting similarity judgments, building on previous developments in cognitive science. The participant arranges a number of items (little photos or icons) on a computer screen by mouse drag-and-drop operations. This multi-arrangement method enables participants to communicate multiple continuous object-pair similarities by a single drag-and-drop placement and to adjust the judgements in the context of the entire set.
1.3 Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far).
The present Proof of Concept (PoC) project produced a web-based platform for the multi-arrangement method that is compatible with modern technologies (e.g. tablets, touchscreen laptops and desktop PCs). We used JavaScript, Python, and HTML to implement an easy-to-use web-based tool that makes the MA method available to a broad base of users. The web-based platform provides important savings, reducing programming costs, research assistant work, and volunteer travel expenses. This development could in the future enable researchers in science, government, and industry to quickly and easily acquire substantial amounts of subjective human judgments at lower costs.
We started the company Meadows Research, which has enabled other labs to use the new method. Launched on 15 February 2018, Meadows currently works with a number of labs in different countries. The following institutions have finished projects on Meadows: Cambridge University, University of Muenster, University of Berlin, University of Tel-Aviv, George Washington University (DC), MIT, Temple University, University of Birmingham, and Cardiff University. Some of the researchers involved are collaborators, not paying customers.
1.4 URL of the project’s public website
Public website of Meadows Research
http://meadows-research.com/(opens in new window)
We set out to develop a technique for collecting continuous similarity judgments from human participants. Collecting this kind of human judgment typically takes a long time, because a judgment of similarity must be communicated for each pair of the items to be judged. For hundreds of items, there are many thousands of pairs, so conventional methods are very time consuming.
1.2 Work Performed from the beginning of the project to the end of the period covered by the report and main results achieved so far (For the final period, please include an overview of the results and their exploitation and dissemination.
We developed a new and more efficient method for collecting similarity judgments, building on previous developments in cognitive science. The participant arranges a number of items (little photos or icons) on a computer screen by mouse drag-and-drop operations. This multi-arrangement method enables participants to communicate multiple continuous object-pair similarities by a single drag-and-drop placement and to adjust the judgements in the context of the entire set.
1.3 Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far).
The present Proof of Concept (PoC) project produced a web-based platform for the multi-arrangement method that is compatible with modern technologies (e.g. tablets, touchscreen laptops and desktop PCs). We used JavaScript, Python, and HTML to implement an easy-to-use web-based tool that makes the MA method available to a broad base of users. The web-based platform provides important savings, reducing programming costs, research assistant work, and volunteer travel expenses. This development could in the future enable researchers in science, government, and industry to quickly and easily acquire substantial amounts of subjective human judgments at lower costs.
We started the company Meadows Research, which has enabled other labs to use the new method. Launched on 15 February 2018, Meadows currently works with a number of labs in different countries. The following institutions have finished projects on Meadows: Cambridge University, University of Muenster, University of Berlin, University of Tel-Aviv, George Washington University (DC), MIT, Temple University, University of Birmingham, and Cardiff University. Some of the researchers involved are collaborators, not paying customers.
1.4 URL of the project’s public website
Public website of Meadows Research
http://meadows-research.com/(opens in new window)