Designing better surveys
Surveys help researchers understand numerous issues and aspects related to society, its beliefs and its values. With increasingly busy lifestyles of responders, however, survey responses have suffered, with the result that statistics collected may not always represent a large enough sample or true picture of the issue at hand. Efforts have been made to collect a larger amount of data and yield more representative results, but even these efforts have led to biased information. New quality indicators are therefore needed to measure if survey responders still properly represent the population. The EU-funded 'Representativity indicators for survey quality' (RISQ) project developed and tested quality indicators that enable evaluation and comparison of the representativity of response to surveys and registers. These can be used as tools in monitoring response during survey data collection and for improving representativity of response through tailored survey designs. The project began by examining the concept of quality indicators and how they may be estimated through international comparison and empirical validation. RISQ extended and elaborated existing theory, looking at international validation of indicators in five countries. It worked on developing new indicators that would be easy to interpret, compute and translate into practical information, particularly if the survey's goal is to compare and evaluate responses. To redesign survey data collection in order to improve representativity, other goals such as effectiveness of the survey need to be met. With these different goals in mind, the project conducted theoretical and empirical studies to further develop these indicators. The empirical part of the project consisted of two designed data collection experiments that helped RISQ develop representativity indicators (referred to as R-indicators and partial R-indicators). The indicators were documented, extensively tested and implemented as open source modules for representativity and sample survey design. The analyses, tests and experiments conducted within the project showed that the indicators are valuable tools and that the novel survey designs elaborated improved the representativity of response significantly. This information was disseminated at several workshops and conferences, which encouraged the use of the indicators as standards in quality reports. In addition to evaluating and designing survey data collection, the indicators also proved useful in monitoring the completion of registry data. The indicators have been made available for downloading from the project's website, along with a manual and test datasets. While RISQ ended officially in 2010, informal activities surrounding the project are still ongoing. A second release of the project indicators has already been published, and a third is planned for spring 2011. The RISQ website is being updated on a regular basis and will contain future releases. This initiative will help produce better surveys and give researchers much more accurate information regarding a variety of research issues, social aspects, and much more.
