Final Report Summary - MYWEB (Measuring Youth Well-Being)
Executive Summary:
Our work has shown the desirability, feasibility and direct policy benefits of an accelerated cohort survey of children and young people’s well-being. We have provided the proof of concept. The practical steps required to undertake such a survey more broadly than in one country requires further development. There are precedents for this with regard to Research Infrastructures in use across the EU such as the European Social Survey (for comparative cross-sectional) and the SHARE survey (a longitudinal survey of older people in 20 European countries. We recommend that in order to further the development of a Europe-wide (or even international) longitudinal survey of children and young people’s well-being, that the work of MYWeB should be continued and focused upon developing a suitable research design which could subsequently be taken forwards for piloting. Future work should involve political and policy decision makers who have a vested interest in evidence-based policy interventions.
The policy challenge is a gap in evaluation of child and young people’s well-being as policy decisions across Europe are currently made based on data that is not fit for purpose.
MYWeB took a balanced approach to assessing the feasibility of a European Longitudinal Study for Children and Young People (ELSCYPWB) through prioritising both scientific and policy imperatives. Striking the appropriate balance between science and policy was guaranteed through the use of an evaluation/appraisal methodology, which ensured that the outcomes were methodologically robust, technically feasible and represented value for money. This report concludes that whilst an ELSCYPWB would feature many challenges, it is desirable, feasible and would offer great (money saving) benefits for policy makers.
Our main recommendations are that:
1. Further work is required to add greater precision to the costs and benefits of an ELSCYPWB. This should include robust economic modelling and the development of case studies which highlight the specific benefits of longitudinal survey data over cross-sectional data and administrative (whether cross-sectional or longitudinal) data.
2. Key political and policy decision makers should be invited to contribute to discussions of costs and benefits to provide a practical realpolitik, which will inform the scope and limits of an ELSCYPWB.
3. The methodology of an accelerated cohort ELSCYPWB should be further developed, addressing the specific technical requirements of such a research design on a comparative EU level.
In addition, we recommend that:
4. CYP should be involved in the development, testing and implementation of the survey, notably through a CYP Advisory Board and their involvement in the ethics committee.
5. Future work should involve key international stakeholders including UNICEF and OECD.
6. There should be a strong focus on respondents’ engagement, potentially through sustained communication and social media.
7. There is a need for knowledge transfer across EU Member States in order to build capacity for longitudinal surveys.
8. The survey should consider the entire life span of children from birth to 25 and should include both objective and subjective measures of well-being. In addition, it should use a range of modes of data collection.
Project Context and Objectives:
MYWEB (Measuring Youth Well Being) was a major cross-European project classified as a Framework 7 Coordinating Support Action, which undertook a feasibility study into the development of a longitudinal survey of children and young people’s well-being.
The MYWEB project ran from 1st March 2014 until 2nd September 2016 and involved a total of 14 beneficiary organisations in 11 countries:
1. (Coordinator) Manchester Metropolitan University UK
2. University of Bremen Germany
3. Pompeu Fabra University Spain
4. Ivo Pilar Institute of Social Sciences, Zagreb Croatia
5. University of Debrecen Hungary
6. Daugavpils University Latvia
7. Panteion University of Social and Political Sciences Athens Greece
8. Tallinn University Estonia
9. University of SS Cyril and Methodius Slovakia
10. Centre for Research and Studies in Sociology, ISCTE,
Lisbon University Institute Portugal
11. Caucasus Research Resource Centers, Georgia Georgia
12. The Institute for Social and Economic Research at the University of Essex UK
13. The Chancellor, Masters and Scholars of the University of Cambridge UK
14. Catalan Youth Agency Spain
MYWeB took a balanced approach to assessing the feasibility of a European Longitudinal Study for Children and Young People (ELSCYP) through prioritising both scientific and policy imperatives. Striking the appropriate balance between science and policy was guaranteed through the use of an evaluation/appraisal methodology which ensured that the outcomes were methodologically robust, technically feasible and represented value for money. A cognitive pilot study in six countries meant that original empirical data on field experiences provided direct evidence of the feasibility of an ELSCYP. Engagement with a wide range of stakeholders including policy-makers at a European, Member State and regional level ensured that the project outcomes took into account the broadest range of policy makers. Questions about the “value added” that a longitudinal survey can offer over a cross-sectional survey were fully informed by policy agendas. Children and Young People were integrated into the project plan to contribute to the operationalisation of notions of well-being as well as in understanding the best modes of conducting an ELSCYP. The MYWeB consortium contained researchers from a variety of disciplinary backgrounds and provided expertise in the areas of children and young people’s well-being, childhood care, education, the environment in which a child grows up, childhood/youth work and leisure and participation. In addition, all teams were experienced in undertaking questionnaire survey research.
The empirical scientific objectives of MYWEB were as follows:
WP2: Mapping Existing Policies, Studies and Data
1. To collate a comprehensive database of child and youth policies and legislation across Europe.
2. To map out these existing child and youth-related policies across each Member State and consortium member relating to well-being, childhood care, access to education, family life, civic participation and leisure activities.
3. To design and implement a research instrument to interview key informants at a national level on the main issues which need to be addressed when measuring children and young people’s well-being.
4. To collate a comprehensive database of existing data sets across Europe that collect data on children and young people’s well-being, detailing both substantive content and the technical specifications of how the data was collected.
5. To map out existing data sets that capture data on children and young people across each Member State relating to youth well-being, childhood care, access to education, family life, civic participation and leisure activities.
Through the collation and comparative analysis of the various national policy frameworks for children and young people, a mapping exercise identified areas where there are similarities and contrast this with key points where differences exist. Policy information was largely drawn from local, national and supranational bodies. Each delivery partner identified key informants (national policy makers and/or academics) who influence national policy. In addition the WP leaders identified Europe wide policy makers. Each key informant was interviewed using a semi-structured interview.
WP3
1. Produce a review of literature on measuring key concepts such as childhood care, access to education, family life, civic participation and leisure activities.
2. Produce a review of literature on the methodological challenges associated with different types of cross-sectional and longitudinal surveys with young people.
3. Produce first long-list of options for studying children and young people’s well-being and first long-list of political, technical and cost criteria against which options will be assessed.
A systematic review methodology allowed the project team to assess how methodologically robust current research on CYP’s well-being is and hence how much confidence policy-makers should place in the current knowledge base. It helped to identify the key gaps in knowledge and to better define the nature of those gaps.
WP4
1. To select a diverse range of children and young people from each country to inform the progress of empirical WP’s.
2. To use interviews and focus groups with these young people to explore the meanings that they attach to notions of well-being.
3. To use interviews and focus groups with these young people, to assess the effectiveness of different modes of data collection and of particular challenges of longitudinal data collection.
4. To set up a Children and Young Persons Advisory Group in each Delivery Partner country to contribute to the development of the research instruments and the fieldwork processes to include both young people and experts in education and youth work.
The project took into consideration the views and opinions of young people and children on their understanding of well-being and its different dimensions, following a holistic perspective. We know that it is not easy to get the involvement of young people (and teenagers and children) in a consultative and participatory process. In order to select and have the involvement of young people and children, two strategies were used.
1) Selecting “organised” young people and children via regional and national organisations.
2) Selecting “non-organised” young people and children via schools.
WP5
1. To undertake a robust and transparent ‘options appraisal exercise’.
2. To identify a preferred option that meets the key selection criteria, namely that the preferred option is policy relevant, technically feasible and represents value for money
3. To engage a broad range of stakeholders in the options appraisal exercise, so that the options appraisal process has wide support and therefore the preferred option, whilst not necessarily the preferred option of all stakeholders is, nevertheless, accepted by all stakeholders.
We used a Delphi survey to include:
- policy-makers at a regional, national and European level with an interest in childhood well-being and youth transitions.
- experts in and advocates for aspects of childhood well-being and youth transitions.
- academics and researchers involved in relevant research studies and surveys.
There were three rounds of surveys followed by an options appraisal exercise.
WP6
1. To develop a research design to undertake the cognitive testing of new measures of well-being for children around the age of 7.
2. To undertake all appropriate developmental work in relation to the translation and testing of the measures developed in objective 1.
3. To undertake phase 1 of the ‘Process Pilot Survey'.
3.1. Using the outputs from WP2 and WP3, supplemented by further research, to create a database of all birth cohort and accelerated cohort studies, including a well-being theme and which have had a data collection round within the past ten years.
3.2. To use the database created in objective 3.1 to collate technical information on research designs (instrument development and testing, sampling), pilot tests, and field experiences.
3.3. To undertake a systematic evaluation of the information collated in objective 3.2 in order to develop a set of recommendations for best practice in relation to: birth cohort (prenatal data collection, post-natal data collection) age 7 cohort data collection, age 15 cohort data collection.
This WP comprised three distinct but linked elements. Firstly, the development of longitudinal questionnaire instruments that include age appropriate measures and can be deployed in an international context. Secondly, a technical field test strategy for cognitive testing which to be used to assess the relative effectiveness of the different elements of this instrument. The third task was phase 1 of a ‘process pilot study’ to develop the database of surveys initiated in WP2 with a focus on collecting in-depth information in regard to design features, field experiences, operational experiences, policy impact and costs.
WP7
1. To undertake cognitive testing of a limited number of new wellbeing questions for seven year olds.
2. To evaluate potential promotional methods for advertising campaigns.
3. To undertake phase 2 of the process pilot survey.
3.1 To consult the Children and Young People Advisory Group in relation to the findings of phase 1 of the process pilot.
3.2 To evaluate the process pilot survey, in terms of both its technical features, from the perspective of both the respondents and the field research team.
3.3 To undertake a cost benefit analysis of an ELSCYP.
This Work package involved two distinct elements: firstly conducting cognitive pilot surveys in delivery partner countries to field test new child well-being items. The results of the cognitive pilot provided new child-centered measure of well-being, which would be important components of an ELSCYP. The second element of this WP is phase 2 of the process pilot, which involves a systematic analysis of the information collected in phase 1, including a cost-benefit analysis.
Project Results:
Table 1: Overview of the research activities undertaken
Work Package Name Research Activities
WP1 Stocktaking and design N/A.
WP2 Mapping existing policies, studies, and data Review of 256 policies across Europe, 457 sources for administrative data, 370 sources for research data, and interviews with 83 key informants.
WP3 State of the art in the literature and practice across the EU and beyond 411 papers identified on the concept of survey methods, of which 75 were included in the review
8 717 papers identified on the concept of well-being, of which 121 were included in the review.
WP4 Direct engagement with young people Interviews with 440 children and young people in 11 European countries, from 9 to 24 years old.
WP5 Options appraisal Delphi survey engaging 334 experts across Europe.
WP6 Survey design Review of 65 surveys (European and world wide longitudinal surveys and key European cross-sectional surveys).
WP7 Pilot surveys 195 cognitive interviews with children aged 7 and 8 in 6 European countries.
WP8 Policy and impact Workshops involving 51 experts and 51 young people (aged 12 to 24) in 6 European countries.
WP9 Dissemination Dissemination activities across Europe and beyond (e.g. USA, South Africa).
WP10 Project management N/A.
1 Theoretical perspectives on children and youth well-being.
Holistic approach. Well-being is a complex concept which can be understood from different perspectives. There is a consensus that an ELSCYPWB (European Longitudinal Study of Children and Young People’s Well-Being) should have a holistic approach comprising the following premises:
• Hedonic and eudaimonic perspectives. Evidence from a number of investigations indicates that well-being is best conceived as a multidimensional phenomenon that includes aspects of both the hedonic (subjective) and eudaimonic (psychological) conceptions of well-being. The questionnaire will be based on a holistic approach focusing on both aspects. In other words, the psychological, affective (both positive and negative), and cognitive aspects (life satisfaction and its domain specific satisfaction) of well-being will be explored. However, it is argued that the affective aspect of well-being is less stable over time because of its reliance on people's experience of positive and negative emotions, which typically varies over time. Therefore, in the proposed questionnaire the measurement of affect has a limited space and life satisfaction is prioritised.
• Children’s perspective. Childhood is not merely a preparation for adult life, but a life stage in its own right. Children have to be considered as social actors and therefore their conceptions of well-being have to be taken into account together with those issues that research has identified to be important for children’s well-being. Therefore, the proposed questionnaire also responds to the children’s perspective and conceptions of well-being.
• Objective and subjective measures. Both objective and subjective measures are crucial for the understanding of children’s well-being. With regard to subjective measures, both hedonic and eudaimonic measures should be included. With regard to objective measures, the questionnaire seeks an equilibrium between those that focus on the quality of their lives in the present (Children’s rights perspective) and those preventative indicators that can help researchers to predict their future well-being (Developmental perspective). Some preventative variables have been included. However, the first wave draft questionnaire is focused on children’s Subjective Well-Being (SWB) and therefore these issues have a bigger role. Preventative indicators should have a bigger role in the parent’s questionnaire.
Material conditions. Material conditions of existence are fundamental to understand children’s well-being and its determinants. However, these objective measures can be gathered not only directly from the children but also from other sources, such as their parents. Considering the early age of the primary respondents, these external sources are probably more reliable to gather this kind of information, thus enabling children to focus on their perception and evaluation of their living material conditions.
Independent, explanatory variables. An overview of pre-existing questionnaires on children’s well-being gives rise to an important question about the depth of analysis that general well-being surveys allow for. If the questionnaires are centred on subjective well-being, and even if there is a complementary questionnaire for parents focused on material conditions, there may be little capacity of analysis of the elements that are linked, or explain, the well-being results. That is, if the questionnaire asks for well-being in the relationship with peers but does not contain information about practices (number of friends, meeting frequency, type of relationship, etc.) the analysis of the information will be mostly descriptive. The general surveys on children’s well-being that have been analysed (ISCWeb and Millenium Cohort) somehow respond to this patron; on the other hand, thematic questionnaires such as PISA, HBSC, etc. combine both subjective questions about well-being and objective questions about practices – allowing, apparently, for a deeper analysis. The proposed questionnaire combines both subjective and practice descriptive measures.
Dynamic approach. In order to grasp causality, there is also a consensus that the study of well-being needs a dynamic approach. This stems from the fact that well-being changes over time and that understanding the processes that determine well-being is even more relevant than measuring it. It has been concluded that a longitudinal perspective is necessary and that a panel survey is the option that best responds to this need. The proposed questionnaire should be the first wave questionnaire of an ELSCYPWB designed to grasp change and its effects on children’s well-being.
2 Involving children and young people in research and instrument design.
MYWeB is underpinned by a belief that involving CYP in research is both a democratic requirement and an effective tool to better design and carry out research, as if their points of views are not taken into account, basic mistakes can be made in the approach to the subject, the instruments that are used and the results’ analysis. CYP can offer unique insights of a view that can be different from adults’ views. In agreement with the UN Convention of the Rights of the Child, MYWeB holds that children should participate in their own concerns.
See Figure 1: Children and young people's engagement in MYWeB in PDF version of this report
The Concept of well-being as understood by CYP
Well-being in relation to CYP is generally assumed to involve relationships: family (parents, siblings, extended family), friends and school based relationships (teachers and school mates). Studies of well-being cover:
• Material conditions. Wide approach: family incomes, neighbourhood, public transport, access to healthcare, family’s legal status, etc. This is more prevalent if they come from working class backgrounds.
• Studies and education. The school/institute frames their lives (homework, exams, stressful situations, bullying).
• Other issues (health, disabilities, maturity, self-acceptance, partnership).
The MYWeB fieldwork also found that the meaning and importance of these concepts are not homogeneous between countries:
• The meaning of well-being is not clear. This is evident in the context of international research, with cultural and translation-related issues. It has been found that the direct use of the abstract concept of well-being is problematic and that CYP are more comfortable with experience-centred questions.
• In addition, the fieldwork has highlighted that the theoretical divisions of subjective well-being are not clear for CYP. The borders between satisfaction, happiness and psychological well-being are not always distinguished and they should not to be included as direct items in the questionnaire.
Finally, the fieldwork also highlighted that:
• Involvement in social and political groups is important for their understanding of well-being: the more active CYP are, the more their discourse and values tend to include a social and community dimension.
• Many CYP pointed out the material conditions as their main focus of subjective well-being. An ELSCYPWB should therefore provide information about their context and material living conditions, in order to better understand the determinants of subjective well-being.
The perception of well-being is greatly transformed as children grow up into young people. Growing means broadening one’s environment, relationships and interests:
• Children tend to base their well-being on family and school;
• Teenagers are in a process of identity construction and peer relationships and self-perception are basic for their well-being; and
• Young people tend to have a wider range of interests and sources of well-being and worries (studies, work, social and political involvement, etc.)
Therefore, an ELSCYPWB should have a developmental approach. However, it is difficult to draw a clear age-based line as their worries and interests do not only depend on their material situation (household, main activity, etc.) but also on subjective, psychological aspects.
The fieldwork identified specific well-being situations of particular groups and therefore the need for inclusive research:
• CYP from ethnic minorities or a low social origin and with disabilities or a particular legal status have special needs.
• It is important to make efforts to stimulate “apathetic” CYP to engage in a longitudinal survey.
Our research also suggests that there is a need for some level of qualitative data collection to better include the needs of marginalised groups, but also to grasp the meaning of some situations difficult to measure using a quantitative tool. It may, therefore, be important to complement a survey with qualitative tools for a comprehensive understanding of well-being.
Survey engagement:
Most of the CYP we studied considered that it was important to study their well-being, as this type of research allows them to:
a) express themselves and voice their views; which should lead to
b) a better understanding of their needs, which in turn should lead to
c) an improvement of their situation in society.
They further stated that highlighting the potential of research to improve their situation is a key element to encouraging participation. Therefore, giving timely feedback to participants is important to stimulate long-term participation in an ELSCYPWB. This should serve to counterbalance the scepticism that some young people expressed about the impact that research can have in their situation. In addition, some young people considered that a material incentive would encourage participation among their counterparts who might be less engaged in society.
Involving CYP in research is a key element to ensure the success of an ELSCYPWB. CYP have enlightened MYWeB with basic information that directly affect the project.
The well-being and needs of CYP often are associated with factors that cannot be pre-defined by adult researchers. For example:
• Questionnaire design: the cognitive testing has shown that CYP must be involved in designing the questionnaire, otherwise fundamental mistakes are made.
• CYP involvement in the survey: several strategies suggested by CYP should be taken into account in order to increase participation rates.
• The impact of the research will be greater with the involvement of CYP, through giving feedback about the results to the participants.
These issues would be a basic requirement in national research. In the framework of an international research, they must necessarily be taken into account to guarantee the success of the project.
The Delphi Survey and the Role of Children
Respondents largely agreed that children’s (85% of respondents) and young people’s (94% of respondents) views are an important input into designing the research instrument. This was particularly the case for young people (68% strongly agree) compared to children (45% strongly agree). Respondents working for NGOs had a tendency to consider children’s views as more important than those working in policy or research. Qualitative comments indicated that CYP views are important because they provide a unique insight on children and young people’s well-being, which can be quite different from those expressed by adults.
A number of respondents also mentioned the UN Convention on the Rights of the Child, which stipulates that children have the right to participate in decisions that concern them.
The majority (56.8%) of respondents consider that evidence-measuring children’s and young people’s well-being should always include views from parents and primary carers. However, a third of the respondents (36.8%) considered that information from parents and primary carers is necessary only for some age groups. Some respondents suggested that evidence should also include a wider range of views (teachers, health professionals etc.).
Research undertaken with CYP on their understanding of well-being for this project indicated that that some young people are interested in participating in such a study, because they perceive interviews as a chance to express themselves in front of someone who is willing to listen to them. Children and young people also indicated that incentives would enhance their willingness to participate. Delphi participants considered this information and there was a very strong consensus among participants (93.1%) that complementary qualitative interviews should be introduced in a longitudinal survey in order to enhance participation. Respondents also largely agree (66.7%) that incentives should be introduced to enhance participation.
Further analysis of the qualitative work undertaken with children and young people revealed that their will to participate in research often draws on the fact that it is an opportunity to express their views and that it could help other children and young people. They are interested in the impact of the research. This can be an important factor in empowering children and young people and keeping them engaged throughout the course of a longitudinal survey.
Experts suggested the following methods to disseminate findings: Social and multimedia such as videos, podcasts, and apps (cited 74 times), events such as seminars and workshops or participation camps (cited 22 times), presentations in schools (cited 14 times), and making sure the content is targeted to different age groups (cited 12 times).
3 Well-being data in Europe
Existing data on child and youth well-being
Objective aspects of well-being are captured in both administratively collected data and survey data. Subjective aspects of well-being, however, are collected in surveys only.
Much of the survey data that is collected on the well-being of CYP in Europe is cross-sectional with some notable exceptions where there are birth cohort and dedicated children cohort studies. Topics which are well covered tend to be measures of material well-being such as: socio-demographic and economic inequalities and education and health. Surveys supplement this with measures of abstract concepts, such as happiness, as well as of relationships.
Data gaps
There are significant gaps in data availability. Firstly in terms of countries where there is a distinct lack of data on CYP well-being across Europe, but in particular in post-Soviet countries. This is true in terms of both administrative and survey data. In addition, children and young people’s views on well-being are not routinely into account. Existing systems therefore miss aspects that are important from the subjective child/youth perspective. There are only a few longitudinal surveys, which cover well-being topics for CYP. Furthermore, psychological aspects of well-being are not routinely measured. Only a few surveys comprise children below the age of 11 years hence there is a tendency to miss important school aged experiences. Vulnerable (minority) groups tend not covered by surveys (e.g. Roma, children/young people in homeless hostels, disabled children/young people). This is partly a technical artefact of sampling whereby they are both less prevalent and ‘hard to reach’ but there appears to be little effort to broach the issue of representing these groups adequately.
In some cases there are topics which are hardly covered if at all for example, culture, gender, environment.
The Delphi Survey and the existence and desirability of well-being data
The review of research studies on children’s and young people’s well-being demonstrated that there is a large range of studies that cover the concept of well-being. A number of quantitative surveys, in particular, have been dedicated to this topic during recent years. Many have been conducted in classical survey modes such as Paper and Pencil (sometimes carried out in school classes) and face-to-face (often combined with CAPI support). In most cases, cross-sectional studies were conducted.
Coming to qualitative surveys it is important to note that in-depth interviews and focus groups are covered well in children`s/young people`s well-being studies.
Administrative data contributing to illustrations of children’s/young people’s well-being can be considered an important source, supplementing quantitative or qualitative studies. Administrative data is, in most cases, annually updated and almost always reflects the whole national population. Additionally, most administrative data sets gathered allow for an age distribution in order to consider children or young people separately.
Turning to the content of the gathered data sets, socio-demographics, economical aspects, education and health are included in many of both, survey and administrative data. Additionally, relationships to peers and parents are covered well in quantitative and qualitative research studies. The advice for combining objective and subjective measurements of well-being as it was, for instance, expressed by Stiglitz, Sen and Fitoussi (2009), is followed by most of the current studies. Also, abstract concepts like happiness that are hard to measure and thus, mean a barrier in survey planning processes, are included in studies quite often.
Administrative data particularly contributes statistics on objective well-being aspects that are related to the state (e.g. figures on taxes, social benefits, unemployment, school achievements and enrolment rates and crime statistics).
Which gaps are obvious?
There are recent studies on children’s/young people’s well-being in all EU countries (except in Cypress and Luxembourg). However, East European (Bulgaria, Czech Republic, Hungary, Romania, Lithuania, Poland) and smaller (Denmark, Ireland, Malta) countries are not represented often compared to other European countries. The Netherlands and Sweden stand out in this regard. Fewer administrative data sets were reported for the East European countries mentioned above as well as Slovakia. Thus, it is of particular importance to include East European countries in future surveys on children’s/young people’s well-being, to receive an overall European picture.
Recent well-being surveys also consider the children’s/young people’s views of well-being (Mason & Danby, 2011). To ensure the inclusion of this aspect, it is necessary to have qualitative studies in well-being research that can provide the basis for developing well-being concepts used in quantitative surveys. This has not been taken account of adequately in Europe so far. Instead, quantitative surveys relying on adults’ conceptualisation of well-being have been more common.
Innovative survey modes that appeal particularly to young people (e.g. implementing video sequences, avatars or other web 2.0 innovations) have not been used so far in wellbeing surveys. The main reason for this is the preference of classical survey modes (Paper and Pencil, Face-to-Face). New and innovative survey modes hold several unique possibilities for surveying young people that should be at least piloted in studies on child and youth well-being.
In both research studies and administrative data sets, topics like the environment, risky behaviour, safety, culture and participation tend not to be covered in depth. Although, these topics are covered to some extent in some data sets, detailed insights into them cannot be delivered by existing data. Administrative data sources especially lack information on indirect measurements, such as the provision of free school meals or other basic needs, Information and Communication Technology devices in school and at home (economical), immunisation, doing sports, weight information, nutrition, breastfeeding (health), (il)literacy, learning difficulties, bullying (education), overcrowding, noise pollution and park frequencies (environment). For culture and participation, even broader categories are not covered well by the existing data.
The surveys have not always explored psychological aspects of well-being as has been proposed by Ryan and Deci (2001) (see also Ryff & Keyes, 1995). A number of single and multi-item scales and indices measuring well-being across a variety of domains have been developed and tested in recent years, both in national and transcultural surveys (see for instance the Brief Multidimensional Students’ Life Satisfaction Scale by Seligson et al. 2003, the Satisfaction with Life Scale by Diener et al. 1985 and the Personal Well-Being Index for school children by Cummins and Lau 2005). Already existing scales and indices should be taken into account within the MYWeB project, particularly ones that were especially developed for measuring children’s/young people’s well-being and tested in international surveys. MYWeB will also consider the ground-breaking work done by researchers involved in the International Survey of Children’s Well-being (http://www.iscweb.org).
Data coverage of particular population groups varies strongly. For instance, young carers are a group for which data was considered not to be covered at all by 57.6% of the respondents. Over a third of the respondents also indicated that data is not covered at all for absent parenting (43.2%) and children and young people who have suffered significant harm (38%).
Data coverage is better for children and young people from single households, children and young people from low-income families, children and young people from a minority background, and children and young people with physical disabilities.
Furthermore, difference across countries can be very important (e.g. lack of data about the number of young people living with a physical disability in Georgia or lack of data collected about ethnic minorities in France). However, there is a strong consensus amongst experts than a cross-European longitudinal survey should gather data from all children and young people, rather than focus on specific groups.
4 Policy context and need
What is already covered well?
In general, most of the EU countries embed child and youth policy into all three levels of policy making: the municipal, district and national level; while the legislative frame is put on national level (with the exception of Spain, where the responsibility for youth is devolved exclusively to the regional or local level). Generally, local government has the role of implementing the policies on local level according to specific needs and circumstances, and according to their respective budget. Usually in the form of a ministry, department or office, nearly all EU countries have a national governmental authority responsible for youth. They range widely in financial resources, cross-sectorial influence, integration, and responsibility. While some countries have a dedicated ministry for youth, most of the others cover youth issues as part of a wider portfolio.
Generally, laws cover the main components of objective wellbeing – be it independent child/youth laws or general laws that also cover the matters and interests of children and young people. There are rights that are specific to young people, such as protection of young workers, juvenile justice and laws that ensure National Youth Councils. For children, there are basic rights ensuring child protection, their best interests and non-discrimination. In nearly all countries, the legislative framework is supplemented by existing action plans and programs for child and youth wellbeing, mostly promoted by the EU.
New programmes and draft laws in EU countries show that the topic of child/youth wellbeing is constantly being revised and adjusted to the present needs of those countries. There are promising developments around the institutional structure of youth policies (e.g. integrating municipalities into youth services, institutional changes that heighten transparency about existing measures, the creation of new National Councils, and the strengthening of regional youth policy) and the promotion of early child education and childcare systems.
Which gaps are obvious?
A significant conceptual problem is defining the age groups covered by children and young people in relation to policies on wellbeing. Children’s age groups are less often specified than young people’s age groups.
A significant gap identified through the interview programme and policy capture was between the scope of child and youth policies.
• Youth policy on wellbeing: There are still five countries that have no elaborated independent youth law: Denmark, Spain, France, Italy, and Sweden. In these cases, youth rights are part of non-age specific, general laws and legislations. Laws concerning child and youth matters differ considerably with regard to their quality and coverage (e.g. juvenile justice or the actual participation possibilities in National Youth Councils). These shortcomings are the result of conceptual deficits, where concrete terms and measures are missing and therefore hinder implementation.
• Child policy on wellbeing: Compared to existing youth policies in Europe, child policy is less developed and less coherent. The policies we reviewed display a paternalistic view of children that aims at securing their basic human rights – mostly in terms of objective wellbeing indicators – while neglecting to foster their participation in decision-making processes. Indeed, the policy trends show that children are not perceived as autonomous human beings whose views and opinions are important. Perhaps because of this, children’s own subjective wellbeing is widely neglected. However, child poverty is still present in all EU countries. There are still countries that lack the basic health facilities, child protection, and basic education. Migrants, Roma children and children from large families count to the most often excluded groups – being both excluded from policies and from research focus. As long as these obvious deficits remain, it is perhaps understandable that a focus on subjective aspects of child wellbeing is of less priority.
The Delphi Survey and policy
Experts involved in the Delphi agreed that the role of evidence is important in social policy, particularly in order to have a better understanding of policy impact (65.6%), review the design of social policies (63.2%), monitor progress (60%) and measure the distribution of policy outcomes across different target groups (56.4%). There was a strong consensus amongst panel members that the evaluation of children and youth policies supports policy makers in improving policies (84.7%).
The majority of respondents (63.0%) considered that member state level policy is more important than regional or local level policies, in the context of a longitudinal cross-European survey.
Results from the Delphi survey clearly pointed out that well-being domains had high policy relevance. On a scale of one to five where five is high policy relevance, the following domains had the highest scores: education and skills (4.59) health (4.62) family and home (4.30) and personal well-being (4.07). Respondents also indicated that those domains have relatively good data coverage, except for personal well-being. The dimensions pertaining to the eudaimonic approach to well-being (i.e. competence, autonomy, purpose in life and amount of choice) display an important gap between their data coverage and their policy relevance. Qualitative comments pointed out that there are differences between European and national coverage of those domains.
Experts were asked to identify three major well-being policy challenges for children and young people. Respondents identified education (cited 89 times), poverty (cited 58 times), and health (cited 40 times) for children and education (cited 99 times), employment (cited 97 times), and participation (cited 30 times) for young people.
All the responses provided were then combined into broad categories of policy challenges organised under three themes: community/economic factors, family factors, child/young person specific factors. Experts were asked to indicate the extent to which the data collected through a longitudinal survey could address existing gaps in data coverage and therefore inform decision-making. Respondents considered that a longitudinal survey could address existing gaps in data coverage and therefore inform decision making for all of the policy challenges suggested:
“From my experience, the data collected through a longitudinal survey can play a key role in addressing existing gaps in data coverage and therefore can be valuable resource at the policy and decision making level.” (Policy expert, Delphi 2)
With regard to community/economic factors, a longitudinal survey could be particularly relevant to address data coverage gaps in relation to equality, multi-culturalism, youth migration, and joined up policies/better coordination of services.
Overall, family factors had higher scores than community factors, indicating that longitudinal surveys could provide information that is more useful at this level. Results indicate that a longitudinal survey could provide information-filling gaps in relation to education (including early years), family support (including fathers), and poverty. A few policy challenges appear to have been scored lower by respondents working in research than those working in policy or NGOs: food and nutrition, flexible working for parents, and child protection.
At a child/young person specific level, a longitudinal survey could improve the data coverage of the transition from school to work, mental health, and participation /empowerment. A number of policy challenges were scored higher by respondents working for NGOs: disabilities, young people Not in Education, Employment, or Training, addictions, resilience, and media/new technologies. Results also indicate that data collected through longitudinal surveys is less relevant to leisure and sports than the other challenges identified.
5 Advantages/Benefits of a pan-European longitudinal study
Longitudinal studies are powerful tools that facilitate the collection of data which measures individual transitions and patterns of change. They can produce unbiased data, which helps us to understand social and individual stability and change over time. They are also important in being able to understand causation processes.
Transitions: Longitudinal surveys gather “much longer continuous histories of events and transitions (...) than could be collected retrospectively in a single interview” (Lynn, 2009: 6). The data collected can also be more accurate than data gathered during a single interview, which relies on memories subject to recall errors. Furthermore, longitudinal surveys have the unique advantage of being able to compile unbiased information about expectations that can be analysed against measures of outcomes collected at a later stage.
Patterns of change over time: Longitudinal surveys allow for the measure of stability or instability and the identification of causality. Individual-level change can only be understood in the context of changes taking place over a considerable amount of time. This type of analysis enables researchers to identify patterns of change (e.g. steady growth, fluctuation around a low level, sudden decline followed by stability) (Lynn, 2009). For example, if the proportion of children and young people satisfied with their life is relatively stable over time, there might be many of them starting to feel satisfied with their lives while others are not satisfied anymore. A small proportion of children and young people might be satisfied with their lives on a continuous basis, while the majority show strong variations in time. This insight provides greater information about the dynamics and the factors associated with children and young people being satisfied with their lives. Longitudinal surveys also capture characteristics such as frequency, timing or duration. As noted by Richardson (2012), longitudinal studies make time trend analysis possible, which in the case of children and youth well-being would be useful to identify priorities in child well-being measurements and to validate child well-being statistics.
Longitudinal surveys and their impact
Surveys collect data about the economic, social, political, and cultural shape of a country. Policy makers can use this information to make informed decisions about future policies and evaluate the effectiveness of past or current policies.
Surveys can be divided in two main types: cross-sectional and longitudinal. The key difference is that cross-sectional surveys occur once whereas longitudinal surveys take place on multiple occasions over time (Lynn, 2009: 1). Cross-sectional surveys tell us how people live and what people are thinking or doing at one point in time. Repeating the same measures among the same group of respondents over a long period of time offers insights into how respondents socio-economic situation, health but also their views and attitudes and behaviour developed (Buck 2008). Longitudinal Studies thus help researchers to understand change and stability at the level of the individual, rather than for the population as a whole (Buck 2008).
Longitudinal surveys can impact on policy. More specifically a European Longitudinal Study for Children and Young People can be used to:
- monitor wellbeing;
- study the evolution of wellbeing; and
- evaluate national policy.
Monitoring wellbeing
Currently wellbeing is often monitored both nationally and internationally using high-level social indicators. Longitudinal surveys provide an alternative method of monitoring wellbeing.
Social Indicators and their limitations
Social indicators initially focused on measurement and trends in child well-being primarily using ‘survival indicators’ (Ben-Arieh, 2008) such as rates of mortality, disease, and social problems affecting children (e.g. illiteracy, school failure). Major work informed by this approach is OECD research on the comparison of child well-being across its 30 member countries (Chapple and Richardson 2009) and UNICEF publications (2007).
Most help to understand children and young people's well-being at the objective macro level. While such indicators are important to begin to redress issues of inequality and social exclusion that negatively affect children’s health and wellbeing, they provide only limited insight into social change over time and the causes of change. They also tend to ignore the potential, attributes and strengths of children and it can be argued they treat children as 'passive agents not capable of evaluating their own lives'.
Self-report surveys
An alternative to the use of social indicators is self-report surveys. A number of surveys collect self-reported data on children and young peoples’ wellbeing:
• The international Health Behaviour in School-aged Children survey covers a number of key areas of young people’s health and well-being, with 43 countries involved in the latest wave of the survey in 2009/10.
• Some large longitudinal social surveys have begun to incorporate self-report instruments for young people. For example, two household panel surveys [the European Social Survey (ESS) and the European Quality of Life Survey (EQLS)] included some questions on well-being and its various domains for young people in various age groups. For a full review of these surveys, see Richardson (2012).
• Some national, longitudinal surveys have included questions on well-being. For example, the UK Household Longitudinal Study, previously known as The British Household Panel Survey, has a youth questionnaire for young people aged 11 to 15 about their happiness, feeling troubled and self-esteem.
There are several advantages to surveys of self-reported well-being:
• international surveys among children and young people provide precious comparable data on child well-being covering countries in the EU and beyond.
• household panel surveys provide new opportunities to explore the effect of changes in young people's lives on their overall well-being.
However, the concepts and domains of well-being used in these surveys were developed primarily from concepts which originated from the study of adult well-being. Moreover, as Bradshaw (2009) argues, most of these studies include only a limited number of well-being domains and therefore do not provide the full picture on the state of well-being for children and young people.
Recently, some child and young people-centric well-being studies have been undertaken including the 2005 survey in the UK undertaken by The Children’s Society. This approach has been taken further by an international group of researchers linked to the International Society for Child Indicators. They have developed Children’s World, an international survey in 12 countries on children’s subjective well-being. However, these two surveys are cross-sectional, not longitudinal and therefore are not suitable for monitoring and evaluation.
Evaluating policy
A comparative longitudinal survey of child wellbeing offers policy-makers at a European and Member State level a number of new possibilities for policy formulation. A practical example of data from a longitudinal survey changing our understanding of a social issue and impacting policy responses is the role of the British Household Panel Survey in understanding the dynamics of poverty. Jenkins (2008) recounts how, in the 1990s inequality and poverty rates flattened off and it appeared that there was little or no change in the income distribution from one year to the next. However, the British Household Panel Survey revealed that apparent cross-sectional stability hid longitudinal flux – households’ incomes fluctuate between one year and the next, and there was substantial turnover in the membership of the low income population. Jenkins goes on to demonstrate how these findings influenced policy including much of the emphasis in the Labour government’s welfare reforms from the late 1990s that reflected a dynamic perspective with a focus on moving people into work and making work pay.
Longitudinal studies can be viewed as a form of quasi experimental evaluation design (ISER, 2002). Although they do not involve the allocation of individuals to treatment or control the temporal sequencing of longitudinal data offers a means of controlling fort the effects of independent variables (ibid.). In this sense longitudinal studies provide an evaluation design akin to an interrupted time series. As ISER (2002) note, we can at least say with certainty from the statistical analysis of biographies what events preceded others, even though we still have problems in deciding which event in relation to another event was the underlying cause.
An example cited by ISER (2002) that illustrates the use of birth cohort survey data in the evaluation of policy comes from the UK where returns to qualifications and returns to education more generally have been understood using longitudinal data. Outcomes in adulthood such as occupation and earnings are set against qualifications, taking account of ability as tested in childhood and numerous other circumstances and experiences earlier in life which might be confounded with them. ISER (2002) concludes that “Statistical modeling of this kind, is not a perfect substitute for the controls offered in randomized experiments, but goes some of the way to producing the most plausible accounts of micro economic processes.”.
While insights from longitudinal surveys have much to offer to policy it is worth noting that different designs will be associated with answering different research and policy questions.
Birth cohort studies allow researchers to chart the development of the human life course. As ISER (2002) notes, the data collected for any single birth cohort confounds age, period and cohort effects at any particular point in time. In addition, comparison of more than one cohort enables the researcher to hold constant one of these three ‘extrinsic’ factors, for example, comparing cohorts at a given age to establish a cohort effect or cohorts at the same age at different times to establish a period effect. The data can then be used prospectively to make predictions about the outcomes of particular circumstances and experiences in life occurring at particular points in time or retrospectively to identify the circumstances and experiences in earlier life that underpin a given outcome later (ISER 2002).
Household panel surveys sample the whole population rather than single years of age with the aim of understanding the dynamics of change of the whole population, and its evolution over the lifetime of the study (ISER 2002). These surveys usually follow all the people living in the sample household, not just a reference individual. The origin of household panel studies was in the need to explore the dynamics of poverty and income and an understanding that these could not be explored through separate snapshots, but rather required an approach to collecting a continuous record about income in particular (ibid.). However, the household panel survey is also particularly suited to the analysis of the dynamics of household formation and dissolution, and associated events and outcomes (ibid.).
A recent analysis of evidence-based policy has highlighted the importance of longitudinal survey data in contributing to the development of child well-being policies (Breckon and Dodson, 2016).
The Delphi Survey and the need for longitudinal data
Over three quarters of the respondents acknowledged a need to improve the lack of longitudinal perspectives (81.6%), improve coverage of particular population groups (78.8%), and improve subjective measures of well-being (75.2%).
What can a European longitudinal survey add?
In short, an ELSCYPWB would facilitate better monitoring of well-being both nationally and comparatively across Europe. This would have an impact of public policy on children and youth well-being and the likelihood that CYP well-being would be significantly enhanced.
6 Technical challenges
Implementing a pan European longitudinal survey measuring children and youth well-being comprises different sets of challenges linked to the characteristics of the survey:
• European: International surveys require conceptual comparability and translation.
• Longitudinal: Such surveys involve different sampling strategies and challenges linked to sample attrition (panel mortality).
• Children and youth: Working with children and young people requires adapting research instruments and securing parental consent.
International surveys also have challenges specific to their design. They mostly concern the validity of concepts and words across different languages and cultures. Consequently, developing an international survey requires a focus on:
Data harmonisation: Pre-testing and piloting phase are used to harmonise data. They can be either:
• Output-harmonised, where national teams develop their own questions, under the condition that they cover specific domains.
• Standardised questions that are meaningful, relevant to all countries, expressed in an equivalent manner in all relevant languages.
It is possible to have core harmonised questions complemented by local questions.
Translation: Translation occurs in both international and national surveys where versions are made available in minority languages. There is a wide range of practice across Europe, ranging from informal translation made by bi-lingual researchers to formal professional agencies. Errors appear to occur in both instances. A commonly used method is the double forward translation, with back translation and validation. MYWeB has successfully used a TRAPD (Translation, Review, Adjudication, Pretesting and Documentation) approach as described in section 9.6.
Developing child-friendly questions is central to undertaking research with children and young people. This is also explored in section 9.6.
Longitudinal studies have a number of challenges in terms of data collection and analysis (Lynn, 2009):
Sample attrition: This refers to the continued loss of respondents from the sample due to nonresponses through deaths, moving, and subjects withdrawing from the research. Some surveys successfully re-engage with sample members that have been non-respondents at one or more wave. However, the general response rate of sample units that would have responded to each questionnaire may be lower than in cross-sectional surveys.
Panel conditioning: Responses from participants may be conditioned by their previous experience of taking the survey. Panel conditioning can affect the way respondents report events, or even change respondents’ behaviour.
Coverage error: This error occurs when there is a gap between the sampling frame and total population. Longitudinal surveys are more likely to suffer from coverage error if the sample does not include, or under-represents, additions to the population since the sample was selected. This coverage error is likely to increase with time.
Time and cost: It generally takes longer to prepare a longitudinal survey than a cross-sectional one since questionnaire instruments cannot be designed independently from those used at a later stage. Overall, longitudinal surveys are more costly than cross-sectional surveys.
The Delphi Survey and the obstacles to developing and implementing an ELSCYPWB
Qualitative answers from the Delphi Survey point out to a number of challenges to surveys with children and young people. To summarise:
• Resources are one of the key obstacles to implement a cross-European longitudinal research instrument.
• Respondents also indicated that the EU and national governments might not be interested and that differences between countries would represent an important challenge (culture, policies, languages, data protection).
• In terms of methodology, the majority of challenges pertain to reaching an agreement about measures, concepts, target group, and general methodology.
• In terms of implementation, some respondents noted the lack of capacity to implement such a survey (i.e. trained fieldworkers or lack of institutional capacity), as well as the challenges surrounding getting parents’ participation and consent.
• Challenges to surveys with young people showed similar results, with an emphasis on their participation and migration / mobility.
7 Proposed methodology
7.1. Political desirability
The Delphi Survey and the political desirability of an ELSCYPWB
A large majority (73.2%) considered that a cross-European longitudinal survey was desirable, with 55.6% considering it to be very desirable. When considering the political feasibility of a longitudinal child and youth well-being survey, respondents largely agreed with the following statements:
• It is important that Member State governments are supportive of the survey if it is to be implemented (99% agree, of which 59.4% strongly agree).
• Demonstrating that the survey will have continuing policy relevance for many years to come will be important if the survey is to be implemented (99% agree, of which 56.8% strongly agree).
• It is important that significant research groups in each Member State are supportive if the survey is to be implemented (96.3% agree, of which 45.5% strongly agree).
• It is important that significant NGOs in each Member State are supportive of the survey if it is to be implemented (89.9 agree, of which 31.9 strongly agree).
Funders and stakeholders
As longitudinal surveys are significantly more expensive than cross-sectional surveys, the funders and stakeholders are almost entirely government-linked. This can be directly through one or more governmental departments, or indirectly through a national, government funded, research organisation. The purpose and content of such surveys tends to reflect the priorities of stakeholders. Medical, economic (including employment and education) and social government departments are most likely to be involved in some way in the commissioning, development and analysis.
7.2. Technical feasibility
The Delphi Survey and the technical feasibility of an ELSCYPWB
About half of the respondents (59%) consider the cross-European longitudinal survey to be technically feasible. There was a general agreement that it is feasible to build a longitudinal survey from a pre-existing Europe-wide cross-sectional survey. A higher percentage of respondents working in research/academia (38.7%) indicated that this was not feasible than respondents working in policy (28.1%) or NGOs (31.6%). There was no strong agreement about the feasibility of linking a longitudinal survey to pre-existing national administrative data.
The majority of respondents (59%) agreed that the survey should focus on the entire life-course of a young person from birth to the age of 25. Some respondents indicated that the age limit of a young person should be extended to 30 years old to reflect societal changes (e.g. young people still living at home because of studies or unemployment).
Respondents generally agreed that prenatal information could improve child well-being social policy. A greater percentage of respondents working in policy (93.9%) agreed with this statement than those working in research (82.7%). Qualitative comments indicated that a better understanding of the prenatal period could improve preventative policy.
The results from the first questionnaire showed that the majority of respondents (77.2%) agreed that standardised European measures should be complemented by country-specific modules or questions. This was further explored in the second questionnaire during which experts were asked to indicate whether well-being domains can be captured through standardised European measures or if they can only be captured using nationally specific measures. The majority of respondents indicated that standardised European measures are applicable to the different domains of well-being: competence (84.7%), health (84.5%), education and skills (81.9%), time use (70.2%), and personal well-being (70.0%), amount of choice (67%), personal appearance (63.7%), money and possession (62.9%), relationship with peers (62.3%), purpose in life (58.9%) and family and home (55.4%). However, the majority of respondents (58.9%) considered community and neighbourhood to be a domain that could only be captured using nationally specific measures. This issue was clarified in the third questionnaire, where 81.0% of the respondents answered that it is feasible to develop a classification of neighbourhoods on the basis of an index of relative deprivation that is comparable across Europe.
Yet, concerns were expressed in relation to the complexity of the task, its cost, different meanings attached in different countries to categories such as homeownership, and how one’s subjective feeling about their neighbourhood impacts their well-being more than objective indicators.
Experts suggested that a cross-European well-being survey could be made more sensitive to local variations concerning community and neighbourhood by using additional measurement for each country and/or qualitative measures. It was also suggested that a local expert panel should be appointed in each country to analyse and clean the data.
Whilst respondents working in policy or NGOs considered this feasible (63.9% and 61% respectively), those working in research indicated (53.6%) that it is not feasible.
As indicated in the previous section, obstacles to developing and implementing cross-European longitudinal research instruments identified in the first questionnaire include resources (cited 95 times), European Union and national governments’ support or interest (cited 50 times), challenges pertaining to methodology (cited 132 times), differences between countries (cited 88 times), implementation (cited 57 times), data analysis (41 times), access to particular groups (cited 9 times), and utilisation (cited 6 times).
In the second questionnaire, respondents indicated that the following criteria were feasible, on a scale of one to five where five is very feasible: building child friendly questions that allow adequate comprehension and suitable response modes (4.28) translating the survey into European languages, that will result in robustly comparable international data (4.20) to achieve a representative sample survey in each country (4.12) managing a large and complex data set (4.09) and to obtain children and young people’s informed consent (3.92). Some items scored slightly lower such as: implementing consistent fieldwork practice (3.79) to keep the sample members in future data collection phases (3.66) and getting parental consent (3.62).
7.3. Preparation and Piloting
It is recognised that the preparation of a longitudinal survey is a time consuming process that takes several years. For instance, the ALSPAC (Avon Longitudinal Study of Parents and Children, UK) study required five years of preparation. This involves a pre-test period, which usually takes 12 months. Common pitfalls that slow down the pre-test period include delay in deciding the content, difficulties with developing the questionnaire, and availability of field resources.
Piloting is an essential phase of survey design. It can involve different stages used to progressively test the questionnaire and delivery procedures. For example, an early pilot phase would be used to start testing the flow of the questionnaire and to identify problematic questions. It can be followed by a pre-test or cognitive interviews, used to evaluate questionnaire length, identify further issues with specific questions and their sequencing and consider training requirements. Finally, a dress rehearsal tests the final version of the questionnaire and the procedures to be used in wave 1.
Governance structures are important to ensure the successful implementation of a survey. Their early establishment contributes to rigorous piloting. Structures, such as Research Design and/or Ethic Subcommittees, are often established to support this process. Quality control can also be inserted when external institutions oversee piloting. Furthermore, consultation with the target population (i.e. children) guarantees that their voices are captured in the research design. This engagement can be formalised in structures, such as a Children Advisory Forum, which acts as a feedback mechanism throughout the development of the project. Consultations with other experts, such as academics and government departments, also strengthen the research design.
Piloting usually takes place one to two years before the fieldwork starts. In some cases, piloting took as many as five years (ALSPAC). It is important to allow for a realistic timeframe, as practical reasons (e.g. delay in deciding content, difficulties with developing the questionnaire, availability of field resources) can create delays.
The piloting phase should test the following:
• Instrument design (e.g. theory and hypothesis underpinning, methodology)
• Sampling procedure (e.g. feasibility, logistics, cost efficacy, sampling biases, weighting)
• Data collection tools (e.g. sequence of questions, understanding of questions, question wording, age appropriateness, length of questionnaire, scales, standardisation and validity)
• Field procedures (e.g. incentive strategies, recruitment strategies, data collection modes, data collection environments such as home or school)
Whilst an exhaustive piloting is required before the first wave of data collection, lessons need to be incorporated continuously throughout the life of the survey. Best practice would involve a piloting phase before each wave of data collection. This phase can be limited to testing new questions added to the questionnaire and can involve a small number of participants (i.e. about 20 participants for POLPAN or DJI Kinderpanel) or be more systematic, such as for the Millennium Cohort Study, which had a consultation, a pre-test and a dress rehearsal before each wave.
Finally, piloting the survey is also an opportunity to familiarise oneself with the research tools and procedures. Training is an essential part of quality control. Lessons from the international survey SPARCLE conclude research associates need to speak sufficient English to take part in workshops that focus on administering questionnaires, engaging children, disability issues and the rationale for the study. In this case, research associates conducted five pilot visits in their own countries and met again for a second training workshop to discuss challenges and identify potential solutions. A successful pilot is also a means to securing grants for the survey.
7.3.1 Recruitment strategy and promotional campaigns
Recruitment to a survey can be promoted with publicity (e.g. via YouTube, press releases, media appearances, posters) and through the offering of incentives such as vouchers. Promotional advertising appears more intense when longitudinal surveys are regional rather than national. For instance, ALSPAC had considerable local and national coverage in the press, radio and television. Alternatively, it can be achieved through running recruitment events, and/or consultation exercises with communities and institutions that host potential respondents. Recruitment also works better when contact is made before the actual stage of recruitment (e.g. by letter between a month and two weeks in advance). Where potential participants are slow to sign up to a sampling programme, reminders can be sent to them after initial approach (e.g. after two weeks) by post or through other communication mediums. Where potential recruits are reluctant to sign up to a programme, a research team member can productively be dedicated to the task of enrolling such individuals. As some refusals are down to potentially temporary reasons, it is good practice to try to enrol un-enrolled individuals/institutions approximately 2 months after first refusal.
Contact details for recruits can be sourced from institutions, such as tax offices and education departments. Recruitment is more successful if it is part of a universally mandatory process in a country (e.g. a medical exam for all children starting schooling).
Recruitment approaches are most likely well-received when they reduce burden and appear organised. Such approaches attract goodwill that can lead to less attrition at a later date. Researchers can appear organised and smooth the recruitment process, by sending enrolment packs to individuals/institutions in advance of a study. These might include research handbooks, consent forms and other preparatory materials. Personal touches, such as the signing of recruitment letters, are also advantageous in generating goodwill. The process of recruitment can also be eased and simplified by recruiting gatekeepers (i.e. reputable institutions/organisations) that can facilitate access to large numbers of recruits. For instance, convincing a Ministry of Education of the need for a study can deliver subsequent access to the many schools that take a lead from the Ministry. In turn, those schools can deliver parents/children who take their lead from that school.
As part of recruitment, it is desirable to inform participants of the purpose of research, both for ethical reasons and because more informed samples are less likely to suffer attrition as a study progresses. This can be done through approach letters, websites and primary approach brochures. Where parents are to be enrolled, letters can be sent to mothers and fathers before birth.
Participants in a study should also be given the contact details of qualified people outside of the study who will be able to advise them (e.g. a support line) without them fearing reprisal if they have concerns about their participation in the study. The recruitment process should be one sensitive to individual needs. At risk participants should be recruited and engaged in formats they are comfortable with (e.g. young mothers should be recruited by fellow females).
7.4. Preferred option
The Delphi Survey and the preferred option for an ELSCYPWB.
The first questionnaire did not bring consensus amongst experts about the survey design. Overall 56.6% of the respondents indicated that the most suitable design would be a cohort design whereas 43.4% indicated it would be a household panel design. When asked about further options in the second round, respondents indicated a preference for an accelerated cohort design where data collection would start simultaneously with different age cohorts covering a specific life span. This contrasts with both a narrow age based cohort design which traces a single age cohort as they grow up and a wide age sample which does not differentiate specific age cohorts. The accelerated cohort design was particularly popular for respondents working in policy as illustrated in Figure 2.
see Figure 2: Survey design preference by respondents' background om PDF version of this report
A third of the experts (31%) indicated that if possible, a cross-European longitudinal survey should have an unlimited length. Another third (30%) indicated it should last between 10 and 15 years. Responses indicate that an interval of either two (20.7%) or three years (39.0%) between survey waves was preferable.
An exhaustive selection exercise was undertaken by the project team in conjunction with the Delphi participants to consider all options to use/collect data on children and young people’s well-being from doing nothing, through using administrative data to completely new Europe -wide surveys as shown in Figure 3.
see Figure 3: Options considered to use/collect data on children and young people’s well-being in PDF version of this report
The option that commanded the broadest support was a new, pan-European ‘accelerated’ birth cohort survey with a series of age specific cohorts from birth to 25, based on nationally representative samples in each Member State. Currently, in the UK the Growing up in Scotland (GUS) survey is the closest equivalent to such a survey (see).
See Table2: UK survey options in PDF version of this report
7.5. Accelerated cohort study
A Longitudinal cohort development analysis facilitates analysis of age, period and cohort. Parallel cohorts allow a comparison of different cohorts from the outset. This is not truly longitudinal, but contributes to early analysis of age effects. Using a range of cohorts has the advantage of not fixing the survey to a single point in history, which gives partial control over acute period effects. In addition, it creates policy relevant data for several cohorts starting from the first wave. As with all longitudinal designs, it enables the identification of transition periods and turning points relevant to policy and helps to detect important periods for policy intervention. Moreover, it helps to identify important issues in high impact policy areas (for instance education, family unit etc.). The accelerated cohort design enables policy impact evaluation as well as macro and micro level programme process evaluation. The on-going nature of the survey means that policy adjustments can be made and impact evaluations undertaken for subsequent cohorts. Importantly, through the data analysis and policy interventions, an accelerated design enables medium term cost savings to be made at a national level. Lastly, assuming that an accelerated survey was rolled out across Europe, this survey design facilitates both national level policy evaluation and EU level comparisons.
A Europe-wide national sample accelerated cohort survey
This, in essence, is a series of national birth cohorts rolled out in parallel on different age cohorts. In this sense, they are independent surveys as there are different sampling designs and different survey instruments. What unites them is a common purpose – to study a single set of phenomena for a range of age cohorts as each cohort gets older.
Advantages of an accelerated cohort design
Longitudinal cohort development analysis facilitates analysis of age, period and cohort. The parallel cohorts allow a comparison of different cohorts from the outset. This is not truly longitudinal but contributes to early analysis of age effects. Using a range of cohorts does not fix the survey to a single point in history, which gives partial control over acute period effects. Feedback to children and young people could include a temporal dimension (i.e. how are the young people two years older than me doing?). It would also engage with the same children and young people who do not currently feel that their voice is heard in society. Children and young people are interested in hearing about how the research influences policy. Policy relevant information is generated across several cohorts of young people and enables policy makers to compare life cycle changes and policy impact across different cohorts within waves and as time progresses, same age cohorts across time. The survey helps to identify transition and intervention points and may make policy design more efficient and interventions more effective. The wealth and complexity of data allows researchers to make multidimensional comparisons and enables macro-level programme process evaluation and policy adjustment for subsequent waves. Overall, policy effort and effort in specific policy areas (education, childcare provision, family welfare etc.) can be compared across different regions in EU member states.
Disadvantages of an accelerated cohort design
An accelerated birth cohort survey introduces operational complexity, much of which is frontloaded. The success of this research design is underpinned by rigorous scoping and planning for which a highly structured and sequential forward planning methodology is required, which assess the risks at each stage.
Longitudinal surveys are costly, none more so than large sample cohort surveys. This is addressed below with a consideration of costs weighed against benefits.
see Figure 4: Accelerated cohort survey: phased design in PDF version of this report
7.6. Testing well-being questions
7.6.1 Comprehension of different question formulations and wordings
Findings related to children’s comprehension of different question formulations and wordings confirmed the importance of using simple and short questions, with common everyday expressions and phrases that are directly related to children’s experiences, concrete activities and behaviours. All things considered, children without major difficulties most frequently comprehend questions that enquire about frequencies of specific activities, behaviours or affect. From this study, these are questions about child-parent relationships, bullying, and children’s positive and negative affects. Conversely, findings across countries show that more complex questions, consisting of several parts or conditional formulations (such as those aiming to capture children’s autonomy and the level of respecting the child's voice) are frequently hard to understand. In such cases, children often overlook some parts and only respond to the simpler, specific parts of the question.
Additional issues related to such questions are intended concepts (e.g. autonomy) relevant to children at this age and if they are relevant, how they manifest in terms of children’s everyday activities and behaviours which could be used for more appropriate question formulations. Findings of this study suggest that some of the tested questions (e.g. How frequently you choose what you eat for breakfast?) could be a good starting point.
Regarding the comprehension of specific words or phrases, problems were detected primarily for expressions that were more ambiguous (adult, normally, enough), not age appropriate (e.g. decide, complaint), used in figurative senses (e.g. one thing) or not embedded in a specific culture (e.g. hobby, being happy with).
Additionally, findings of the CI study demonstrate difficulties in finding the right balance between the level of generality and specificity of question formulation in surveys for young children. As some of the described examples demonstrate, by adding a particular context or some explanation, a question became less prone to different interpretations and easier to comprehend.
7.6.2 Response scale types
Findings that relate to children’s recalling and judgement with regard to several types of response scales suggest that simple frequency scales with general verbal responses such as: ‘Always’, ‘Sometimes’, ‘Never’ or ‘Always’, ‘Often’, ‘Rarely’ (‘Not so often’), ‘Never’ are relatively the most applicable scales for survey questions for young children (eight and seven year olds).
Response scales with smileys should be avoided as a general response scale. Children interpreted the smiley-scale out of the question context, and their responses took into account their general level of happiness.
Regarding the Likert type scale, although children correctly understood the meaning of verbal responses, recalling and judgement with such scales was more complex for children than in the case of the frequency scale that is more compatible with the question formulation applied.
With reference to the frequency scales tested in this study, the simplest - three point scale (‘Always’, ‘Sometimes’, ‘Never’) can be recommended as the most applicable. This scale was mostly understood and suitable for seven and eight years old children. At the same time, a simple general four-point frequency scale (‘Always’, ‘Often’, ‘Rarely’, ‘Never’) was appropriate for eight years old children only. Namely, the use of this scale among seven years old children pinpointed some difficulties in understanding the differences between ‘Rarely’ and ‘Often’. The general four-point frequency scale referring to days (‘Every day’, ‘Most days’, ‘Sometimes’, ‘Never’) was appropriate for eight year old children and can also be used with seven year old children without major difficulties. However, the four-point frequency scales referring to number of days (‘Every day’, ‘Most days’, ‘Once or twice’, ‘Never’) or to number of times (‘Four or more times’, ‘Two or three times’, ‘Once’, ‘Never’) were not so applicable since problems in general understanding and differentiating between the scale points were noted.
Open-ended questions, in general, could be used among seven and eight year old children, especially if they required short and simple answers. However, they cannot be recommended as a preferred option for the questionnaire in the English language. Since it could be expected that younger children (seven year olds) will still have some problems with spelling in open-ended questions at that age, it is suggested that this should be substituted with more simple multiple-choice questions.
Generally speaking, findings suggest that response scales and the offered verbal points should, as closely as possible, match the question formulation. More specifically, if a question starts with the general formulation such as ‘Do you...’, ‘Have you...’, ‘Can you...’ children spontaneously expect and respond with dichotomous answers, (e.g. ‘yes / no’ or ‘I can / I cannot’). Thus, if a frequency scale is used, the recommended introductory formulation is ‘How often you...?’
7.6.3 Time frames and context
Findings related to specific time frames and specific context suggest that seven and eight year old children were not entirely capable of recalling their experiences in an exact period of time and space. Specifically, the results of this study showed that time frames specified as: ‘last week’, ‘during this school year’ and ‘since you joined the class’ were not appropriate. Children when answering do not always only take into account the time period mentioned in the questions and/or do not understand concept of specific period uniformly.
However, not putting constraints on any given time frames was related to different recalling periods among children when answering the same questions. Thus, based on the findings of this study it is hard to give a definite recommendation about (not) using the time frame in survey questions for seven and eight year old children. General advice would be to use general formulations or either very short time frames, such as yesterday, today, while in the case of slightly longer time periods (e.g. last week) to ensure that the time-frame reference is explained in detail and unified throughout the questionnaire.
Additionally, regarding the specification of the context, findings from this study showed that, although children understood the relevant words such as ‘at school’, sometimes in answering they also refer to their experiences in other places, not only to the place/context specified in the question. Thus, the general recommendation would be to avoid specifying exact contexts if this is not necessary.
7.6.4 Other recommendations
It is clearly demonstrated that the survey questionnaire planned for self-completion by seven and eight year old children should be as short as possible. Focus of the question should refer to only one experience or type of action, while the use of conditional question formulations (‘if – then’) should be avoided. The wording of questions should be simple. Only one type of instruction for selection and marking the answer (e.g. circling) should be used throughout the whole questionnaire. Additional introductory sentences and/or instructions that are not crucial should be avoided. A simple layout in the form of one question per page is recommended. Moreover, any unnecessary content or information should be avoided in a questionnaire for eight and seven year old children, since this can distract children from answering the questions. It is also recommended that questions should be written in capital letters using a simple font of a larger size without graphical solutions (e.g. bold type) for emphasis.
Potential Impact:
Cost and benefits of a ELSCYPWB
The principal policy impact of MYWEB is in providing a platform from which policy makers understand the needs for robust evidence in order to make sound policy decisions which benefit children and young people and which are financially sustainable. This is delivered through the cost benefit analysis that places the (high) costs of an accelerated cohort survey of children and young people’s well-being across Europe alongside the money spent in social policies, which have a direct impact on children and young people. Deliverable 7.2 has a more detailed explanation of the methodology and results of the cost benefit analysis.
Financial sustainability
The Delphi Survey and the costs and benefits of an ELSCYPWB
Over a third of respondents (41%) considered the survey to be financially sustainable. Yet, the majority of respondents indicated that improving data coverage is somewhat more important than constraining the cost. This is especially the case for respondents working in research (61.1%) compared to those working in policy (52.4%) and NGOs (50.0%). About a third of the respondents consider that improving coverage is top priority. Furthermore, experts strongly agreed with the following points:
• the economic benefits of improved well-being far exceed the cost of a longitudinal survey (93.8%, of which 50.6% strongly agree)
• a longitudinal survey will support policy makers to improve policy design and impact and make policies more efficient (97.9%, of which 58.6% strongly agree)
• evidence gathered by such a survey will contribute to improved well-being (95.6%, of which 45.3% strongly agree)
About half of the respondents agreed with the following sentence: “The economic value of improved wellbeing far exceeds the cost of implementing the survey. However, these economic benefits are not easy to demonstrate (i.e. they are not 'cashable') and therefore cost would be a barrier to implementation.” This was especially the case for policy makers (57.5%) compared to researchers (48.9%) and those working in NGOs (47.6%).
The various strategies identified by the experts to demonstrate policy relevance, draw key decision makers in the process and demonstrate the economic benefits of a well-being longitudinal survey to potential funders are quite similar. They include the following recommendations: clearly identifying specific policy questions the survey will address, demonstrating the need for an evidence-based decision making process, involving the media and increasing public awareness, undertaking cost analysis (SROI, cost of reinsertion vs early intervention, cost of no survey), and providing a sustainability plan.
Costs and benefits
What level of change would be necessary for the investment to be worthwhile?
It is estimated that a European longitudinal survey of children and young people’s wellbeing will cost approximately €722 million over twenty-five years to commission (see Appendix 1 of Deliverable 7.2 for the assumptions and computations that arrive at this figure). Assuming data from this survey is used to affect changes in member state government expenditure in this area, what level of change would be necessary for this investment to be considered worthwhile? To address this question, it is important to first consider member state expenditure on children and young people’s wellbeing and then to compare the cost of the survey to this expenditure. Such a comparison will give an indication of the scale of change necessary for the investment to be considered valuable.
Drawing on published data and analysis from the OECD and Eurostat, data on current spending across EU member states on children and young people’s wellbeing is presented in two ways, namely per capita and total expenditure. Error! Reference source not found. below gives an estimate of government expenditure on children by age group in twenty EU member states in 2011. This is the most recent year for which data is available. Error! Reference source not found. draws on analysis published by the OECD and therefore only covers those EU member states that are also members of the OECD. It provides aggregate, national government expenditure on education, childcare, cash benefits and tax breaks and other benefits in kind. To enable comparison, the OECD analysis is given in per capita spend in US dollars at purchasing power parity (PPP ).
Analysing data on total spend on children and young people’s well-being is less straightforward. To do this, the OECD per capita spend by age group data for 2011 has been combined with Eurostat population data (aggregated to the relevant age group, for 2011). This provides a broad estimate of the scale of national spending by several EU member states on public children and young people’s services. The spending included in this analysis relates to welfare spending (cash benefits, tax breaks and other benefits in kind) and education spending (childcare and compulsory schooling). It should be stressed that the underlying spending data is aggregated by OECD from the individual national budgets and there are a number of significant limitations around this data (the limitations of this approach are presented in the OECD publication, Doing Better for Children (Chapple and Richardson, 2009)). Table 2 presents the underlying estimate spend (nearest billion , USD PPP) and relevant population figures.
see Table 2: Estimated gross spend by member state on child wellbeing, 2011 (USD PPP) in PDF version of this report
The estimates presented in
Table 2 above suggest a combined expenditure of around $1 trillion in 2011 in US dollars (PPP) for the twenty EU member states that are also members of the OECD. We have estimated the likely cost of a European longitudinal survey on children and young people’s wellbeing at between €722 million over a twenty-five year period (see Appendix One). The cost of the survey is therefore an extremely small fraction of overall expenditure in this field (in the order of 0.003%). This suggests that only very small improvements in the effectiveness of member state expenditure on child wellbeing would need to be affected by the availability of longitudinal survey data for the benefits of such a survey to outweigh the costs. Figure 5 below provides data on the anticipated cost of the survey by member state, in year 1, as a percentage of spend on child welfare services . This further suggests that the costs of an ELSCYPWB would be a small fraction of overall spending in this area of social policy.
see Figure 5: Estimate of member state survey spend in year 1 as share of total spend on child welfare services in PDF version of this report
Although the cost of the survey, at member state level, is a very small share of the overall spend on child wellbeing services in year 1 (assumed 2017) and across the whole life of the survey (25 years), this in itself does not give a precise picture of the level of efficiencies that would be necessary for the benefits of the survey to outweigh the costs. The potential for such efficiencies is not evenly spread between member states; some (notably the UK, Germany and Ireland) already commission several longitudinal and other surveys, which provide data on child and youth wellbeing, while others do not currently have access to such data. An ELSCYPWB would complement existing data sources in those countries with a history of investment in this area whilst providing an opportunity for those member states that have yet to make such an investment. In addition, the potential benefits of direct member state comparisons through the use of a common research instrument and fieldwork time frame suggests that there are further benefits that cannot yet be anticipated or quantified.
The analysis set out here identifies evidence that longitudinal survey data are useful to and used by, policy makers in the UK. The UK is used as an example here because it is a member state with a long history of commissioning longitudinal surveys. We have also drawn on findings from the MYWeB research, which strongly suggests that policy makers, academics, practitioners, and others involved in child wellbeing across the EU believe that such a survey would be useful, would be used by policy makers, and would improve the quality and efficiency of public expenditure on wellbeing. We suggest that the costs of the ELSCYPWB would be a small fraction of the overall expenditure in child wellbeing services, suggesting that very small increases in the cost effectiveness of such programmes and services would be necessary for the investment in the survey to be worthwhile.
The project dissemination pursued a three-stage approach: during the 1st phase the main concern was to achieve visibility, during the 2nd phase – to spread knowledge about the project’s aims and its initial steps. The 3rd phase of the project included evaluation and reviewing of initial activities and feeding the results into more mature offerings.
The PDF version of this report contains figures and graphics
A summary description of objectives of the WP9
The key objectives of the dissemination and communication activities in the MYWeB project were: to construct a project web-site dedicated to facilitating dissemination activities in the public sphere and providing a secure 'intranet' facility to project partners and collaborators; provide ongoing dissemination of scientific findings of the research project to policy, practitioner and community audiences at regional, national and European level; to bring the objectives and findings of the project to the attention of public audiences through EU information portals (CORDIS); to provide ongoing dissemination of research findings to the academic community through: the website and individual publications in high quality academic journals; to disseminate key messages on 'quick wins' in the measurement of childhood well-being across the EU; to produce an accessible summary of the findings of the project aimed at children and young people.
A description of the main results and foregrounds
To disseminate findings and recommendations of the MYWeB project regularly throughout the duration of the project (for a detailed description see Deliverable 9.2: Dissemination Strategy). The dissemination strategy was prepared under management of WP9 leader Daugavpils University in collaboration with the coordinating institution Manchester Metropolitan University.
Logos
Logos were used on documents, promotional materials and outputs where appropriate. The MYWeB logo established a ‘brand’ identity to the project, making it easily identifiable. A number of suggestions for the project logo were developed by WP9 leader Daugavpils University and consortium members selected the logo by vote.
The EU logos were important components for outputs. The grant number FP7-613368 was prominently placed on outputs. Institutional logos were used where appropriate. As the lead institution, Manchester Metropolitan University placed its logo on all appropriate documents. In addition, partner institutions placed their own logo alongside MYWeB, EU and MMU logos. Any dissemination activities in the project acknowledged the European Community’s Seventh Framework Programme funding.
Information sheets
The information sheets outlining the MYWeB project’s objectives and main activities were used as a first handout to inform the relevant target groups about the project. Because the WP4 and WP6 included direct involvement of the children and young people, information sheets for teenagers and children were developed by the WP4 and WP6 leads. The cooperation with teachers started with the distribution of the leaflets on the aims and objectives of the MYWeB project.
Templates
Where appropriate, templates, logos and palette information were developed and uploaded to the resources page of the website intranet for partners to use in documents and presentations. Consistency of style of presentation helped partners to produce dissemination outputs.
Project website
The central element in the MYWeB dissemination plan for the 1st period was the project website (for a detailed description see Deliverable 9.1: Project Website). The website http://fp7-myweb.eu/ went live on the 31st March 2014 and it is evolving over the course of the project. During the 1st period, it was one of the main communication channels of the MYWEB project, i.e. the website was the main resource for partners and stakeholders involved in MYWEB, as well as for the general audience, to gain information on the project progress, on the main outcomes (e.g. deliverables), and on key events organized in the scope of the project. It consisted of two parts:
(1) Open Information Resources which provided an informative, easy to read and navigate, comprehensive overview of the project and disseminate information, documentation on the project and findings to a broad audience including scholars, students and the general public.
(2) Intranet database designed for the project partners who entered a password protected intranet space from where they were able to access key project working documents and timetables, event details, reporting requirements. Included within the project area was the description of work, files and presentations from meetings, Deliverables and Outputs areas.
During the 2nd period, on June, 2016 the new version of the project website was launched to provide ongoing dissemination of research findings to four target groups: (1) teachers and researchers, (2) children and young people, (3) practitioners, (4) policy makers.
The section ‘Teachers and Researchers’ includes four sub-sections: (1) Delphi survey; (2) Testing surveys; (3) Our project proposal; (4) Learning from young people. Under section ‘Policy Makers’ the results of Delphi survey, Policy briefs and posters are available. As per the “Practitioners” section, it includes information on Delphi survey and Testing surveys. The fourth section ‘Children and Young People’ was designed to bring the findings of the MYWeB project to attention of children and young people.
Statistics of accesses to the MYWeB website:
Conference Papers
Presentation at academic conferences and dissemination of the project objectives and results have started early in the project and will continue to be an important part of the dissemination activities. Conferences were useful forum to consult with the academic audience in a face-to-face capacity and to address issues relevant to the project. Presenting at conferences provided the opportunity to publicise the MYWeB project and disseminate the first outcomes of the project to a wider audience (see Template A2: List of Dissemination Activities).
Publications
One academic paper for each participating country has prepared for publication. Some of them have been already published, the others are submitted to print/in print (see Template A1: List of Scientific (peer reviewed) publications).
Workshops
The MYWeB project members have contributed to the diary of dissemination activities by recording information about individual contributions to workshops where the objectives of MYWeB have been publicised and presented. While targeted at smaller groups of participants, workshops provided opportunity to involve a much higher and more active level of engagement. Workshops helped to familiarise members’ institutions consortium with the MYWeB project (see Template A2: List of Dissemination Activities).
Social Media
To ensure a presence of MYWeB project on social media platforms the page has been created on Twitter: https://twitter.com/projectMYWeB.
Promotional Materials
The promotional materials of different forms (bookmark, UBS stick, pen/pencil etc.) containing MYWeB project contact details and summarizing the project rationale served to raise awareness of the project and encourage those interested to contact researchers and use the website.
Posters
Posters have been prepared to make the project more visible for the audience of the Policy Forum and of those conferences where MYWEB presentations were made or just poster session planned. A collection of posters from the MYWeb project is available from MYWEB website, see: http://fp7-myweb.eu/policymakers.html
EU information portal
The MYWeB project brought the objectives and findings of the project to the attention of public audiences through EU information portal CORDIS.
The potential impact and the main dissemination activities and exploitation of results
The MYWeB consortium targeted a specific audience from the children and young people themselves, with a dedicated part of the project website, up to EU policy makers. The dissemination included two policy-briefs produced by the team in each participating country, editing of one collected volume and the final project conference.
Policy-briefs
Each partner prepared two policy-briefs tailored appropriately for regional and national agencies in the relevant consortium state. Addressed to policy makers focusing on children and young people, employees of the public administration at all levels, non-governmental organisations and specialists working with children and young people, the policy-briefs highlight national relevant findings and policy implications. They cover a range of themes in relation to: indicators of children’s and youth’s well-being (Croatia), understanding of Estonian children and youth about well-being (Estonia), the importance of a longitudinal study on well-being of children and young people in Georgia, multilevel governance in German child and youth policy, young people’s well-being in Greece, the concept of well-being in European child and youth research, family, the most important domain important to children and young people well-being (Latvia), economic well-being - a transversal priority for Portuguese society, the concept of well-being and its lack of understanding and provision in Slovakia, differences and inequalities in the understanding of well-being among children and young people (Spain), children’s understandings of well-being: the value of consulting young experts (UK), children and youth's participation and well-being (Croatia), results of project MYWeB: experts' opinions about the all-European study of child and youth well-being (Estonia), what young people in Georgia need for well-being and happiness, gaps in current German child and youth well-being policy, children's well-being in Greece, how do young people understand well-being (Hungary), the main obstacles to implement child and youth policy in Latvia, aiming to tackle young people’s unemployment problems in Portugal, the domains and major assumptions of children and young people’s well-being in Slovakia, involving children and young people in research (Spain), the benefits of Europe wide longitudinal data to the United Kingdom.
Edited volume
The main objective of task 9.3 was to communicate MYWeB outcomes in the academia. Overall, academic dissemination in MYWeB involved publishing of an edited volume with the input of all partners. The edited collection with a provisional title ‘Children’s and Young People’s Well-Being: How a pan-European longitudinal survey could improve policy’ will be published.
Conference
The final conference “Children and young people’s well-being and research in Europe” organized in Barcelona on the 21st July, 2016 were intended: (1) to discuss the state of the art about children and young people’s well-being and related policies and identify research gaps from a European perspective; (2) to explain the interest of achieving a better knowledge on policy impacts with regard to children and young people’s well-being and explain the MYWEB project; (3) to present the main results of the EC’s funded project: “MYWEB. Measuring Youth Well-Being”; (4) to discuss the project with the relevant actors in the field: children and young people, academics and policy makers; (5) to disseminate the project through the media for it to reach EU citizens and well-being related professionals. Participants included researchers in children and youth well-being, policy makers related to children and youth policies, MYWEB Advisory Groups and Advisory Board members, children and young people’s representatives and MYWEB members.
Use and dissemination of foreground
The MYWeB consortium intend to carry forward its dissemination activities to provide direct evidence of the feasibility of a European Longitudinal Study for Children and Young People. The key objectives of the further dissemination is to stimulate feedback from stakeholders on the project’s deliverables, to encourage discussion among stakeholders of the project’s findings and recommendations; to facilitate and enable collaboration between different categories of stakeholders; and to strengthen the research and knowledge base of both researchers in children and youth well-being and policy makers related to children and youth policies. It is expected that the continued research on the methodological options will impact developing of longitudinal survey.
List of Websites:
http://fp7-myweb.eu
Our work has shown the desirability, feasibility and direct policy benefits of an accelerated cohort survey of children and young people’s well-being. We have provided the proof of concept. The practical steps required to undertake such a survey more broadly than in one country requires further development. There are precedents for this with regard to Research Infrastructures in use across the EU such as the European Social Survey (for comparative cross-sectional) and the SHARE survey (a longitudinal survey of older people in 20 European countries. We recommend that in order to further the development of a Europe-wide (or even international) longitudinal survey of children and young people’s well-being, that the work of MYWeB should be continued and focused upon developing a suitable research design which could subsequently be taken forwards for piloting. Future work should involve political and policy decision makers who have a vested interest in evidence-based policy interventions.
The policy challenge is a gap in evaluation of child and young people’s well-being as policy decisions across Europe are currently made based on data that is not fit for purpose.
MYWeB took a balanced approach to assessing the feasibility of a European Longitudinal Study for Children and Young People (ELSCYPWB) through prioritising both scientific and policy imperatives. Striking the appropriate balance between science and policy was guaranteed through the use of an evaluation/appraisal methodology, which ensured that the outcomes were methodologically robust, technically feasible and represented value for money. This report concludes that whilst an ELSCYPWB would feature many challenges, it is desirable, feasible and would offer great (money saving) benefits for policy makers.
Our main recommendations are that:
1. Further work is required to add greater precision to the costs and benefits of an ELSCYPWB. This should include robust economic modelling and the development of case studies which highlight the specific benefits of longitudinal survey data over cross-sectional data and administrative (whether cross-sectional or longitudinal) data.
2. Key political and policy decision makers should be invited to contribute to discussions of costs and benefits to provide a practical realpolitik, which will inform the scope and limits of an ELSCYPWB.
3. The methodology of an accelerated cohort ELSCYPWB should be further developed, addressing the specific technical requirements of such a research design on a comparative EU level.
In addition, we recommend that:
4. CYP should be involved in the development, testing and implementation of the survey, notably through a CYP Advisory Board and their involvement in the ethics committee.
5. Future work should involve key international stakeholders including UNICEF and OECD.
6. There should be a strong focus on respondents’ engagement, potentially through sustained communication and social media.
7. There is a need for knowledge transfer across EU Member States in order to build capacity for longitudinal surveys.
8. The survey should consider the entire life span of children from birth to 25 and should include both objective and subjective measures of well-being. In addition, it should use a range of modes of data collection.
Project Context and Objectives:
MYWEB (Measuring Youth Well Being) was a major cross-European project classified as a Framework 7 Coordinating Support Action, which undertook a feasibility study into the development of a longitudinal survey of children and young people’s well-being.
The MYWEB project ran from 1st March 2014 until 2nd September 2016 and involved a total of 14 beneficiary organisations in 11 countries:
1. (Coordinator) Manchester Metropolitan University UK
2. University of Bremen Germany
3. Pompeu Fabra University Spain
4. Ivo Pilar Institute of Social Sciences, Zagreb Croatia
5. University of Debrecen Hungary
6. Daugavpils University Latvia
7. Panteion University of Social and Political Sciences Athens Greece
8. Tallinn University Estonia
9. University of SS Cyril and Methodius Slovakia
10. Centre for Research and Studies in Sociology, ISCTE,
Lisbon University Institute Portugal
11. Caucasus Research Resource Centers, Georgia Georgia
12. The Institute for Social and Economic Research at the University of Essex UK
13. The Chancellor, Masters and Scholars of the University of Cambridge UK
14. Catalan Youth Agency Spain
MYWeB took a balanced approach to assessing the feasibility of a European Longitudinal Study for Children and Young People (ELSCYP) through prioritising both scientific and policy imperatives. Striking the appropriate balance between science and policy was guaranteed through the use of an evaluation/appraisal methodology which ensured that the outcomes were methodologically robust, technically feasible and represented value for money. A cognitive pilot study in six countries meant that original empirical data on field experiences provided direct evidence of the feasibility of an ELSCYP. Engagement with a wide range of stakeholders including policy-makers at a European, Member State and regional level ensured that the project outcomes took into account the broadest range of policy makers. Questions about the “value added” that a longitudinal survey can offer over a cross-sectional survey were fully informed by policy agendas. Children and Young People were integrated into the project plan to contribute to the operationalisation of notions of well-being as well as in understanding the best modes of conducting an ELSCYP. The MYWeB consortium contained researchers from a variety of disciplinary backgrounds and provided expertise in the areas of children and young people’s well-being, childhood care, education, the environment in which a child grows up, childhood/youth work and leisure and participation. In addition, all teams were experienced in undertaking questionnaire survey research.
The empirical scientific objectives of MYWEB were as follows:
WP2: Mapping Existing Policies, Studies and Data
1. To collate a comprehensive database of child and youth policies and legislation across Europe.
2. To map out these existing child and youth-related policies across each Member State and consortium member relating to well-being, childhood care, access to education, family life, civic participation and leisure activities.
3. To design and implement a research instrument to interview key informants at a national level on the main issues which need to be addressed when measuring children and young people’s well-being.
4. To collate a comprehensive database of existing data sets across Europe that collect data on children and young people’s well-being, detailing both substantive content and the technical specifications of how the data was collected.
5. To map out existing data sets that capture data on children and young people across each Member State relating to youth well-being, childhood care, access to education, family life, civic participation and leisure activities.
Through the collation and comparative analysis of the various national policy frameworks for children and young people, a mapping exercise identified areas where there are similarities and contrast this with key points where differences exist. Policy information was largely drawn from local, national and supranational bodies. Each delivery partner identified key informants (national policy makers and/or academics) who influence national policy. In addition the WP leaders identified Europe wide policy makers. Each key informant was interviewed using a semi-structured interview.
WP3
1. Produce a review of literature on measuring key concepts such as childhood care, access to education, family life, civic participation and leisure activities.
2. Produce a review of literature on the methodological challenges associated with different types of cross-sectional and longitudinal surveys with young people.
3. Produce first long-list of options for studying children and young people’s well-being and first long-list of political, technical and cost criteria against which options will be assessed.
A systematic review methodology allowed the project team to assess how methodologically robust current research on CYP’s well-being is and hence how much confidence policy-makers should place in the current knowledge base. It helped to identify the key gaps in knowledge and to better define the nature of those gaps.
WP4
1. To select a diverse range of children and young people from each country to inform the progress of empirical WP’s.
2. To use interviews and focus groups with these young people to explore the meanings that they attach to notions of well-being.
3. To use interviews and focus groups with these young people, to assess the effectiveness of different modes of data collection and of particular challenges of longitudinal data collection.
4. To set up a Children and Young Persons Advisory Group in each Delivery Partner country to contribute to the development of the research instruments and the fieldwork processes to include both young people and experts in education and youth work.
The project took into consideration the views and opinions of young people and children on their understanding of well-being and its different dimensions, following a holistic perspective. We know that it is not easy to get the involvement of young people (and teenagers and children) in a consultative and participatory process. In order to select and have the involvement of young people and children, two strategies were used.
1) Selecting “organised” young people and children via regional and national organisations.
2) Selecting “non-organised” young people and children via schools.
WP5
1. To undertake a robust and transparent ‘options appraisal exercise’.
2. To identify a preferred option that meets the key selection criteria, namely that the preferred option is policy relevant, technically feasible and represents value for money
3. To engage a broad range of stakeholders in the options appraisal exercise, so that the options appraisal process has wide support and therefore the preferred option, whilst not necessarily the preferred option of all stakeholders is, nevertheless, accepted by all stakeholders.
We used a Delphi survey to include:
- policy-makers at a regional, national and European level with an interest in childhood well-being and youth transitions.
- experts in and advocates for aspects of childhood well-being and youth transitions.
- academics and researchers involved in relevant research studies and surveys.
There were three rounds of surveys followed by an options appraisal exercise.
WP6
1. To develop a research design to undertake the cognitive testing of new measures of well-being for children around the age of 7.
2. To undertake all appropriate developmental work in relation to the translation and testing of the measures developed in objective 1.
3. To undertake phase 1 of the ‘Process Pilot Survey'.
3.1. Using the outputs from WP2 and WP3, supplemented by further research, to create a database of all birth cohort and accelerated cohort studies, including a well-being theme and which have had a data collection round within the past ten years.
3.2. To use the database created in objective 3.1 to collate technical information on research designs (instrument development and testing, sampling), pilot tests, and field experiences.
3.3. To undertake a systematic evaluation of the information collated in objective 3.2 in order to develop a set of recommendations for best practice in relation to: birth cohort (prenatal data collection, post-natal data collection) age 7 cohort data collection, age 15 cohort data collection.
This WP comprised three distinct but linked elements. Firstly, the development of longitudinal questionnaire instruments that include age appropriate measures and can be deployed in an international context. Secondly, a technical field test strategy for cognitive testing which to be used to assess the relative effectiveness of the different elements of this instrument. The third task was phase 1 of a ‘process pilot study’ to develop the database of surveys initiated in WP2 with a focus on collecting in-depth information in regard to design features, field experiences, operational experiences, policy impact and costs.
WP7
1. To undertake cognitive testing of a limited number of new wellbeing questions for seven year olds.
2. To evaluate potential promotional methods for advertising campaigns.
3. To undertake phase 2 of the process pilot survey.
3.1 To consult the Children and Young People Advisory Group in relation to the findings of phase 1 of the process pilot.
3.2 To evaluate the process pilot survey, in terms of both its technical features, from the perspective of both the respondents and the field research team.
3.3 To undertake a cost benefit analysis of an ELSCYP.
This Work package involved two distinct elements: firstly conducting cognitive pilot surveys in delivery partner countries to field test new child well-being items. The results of the cognitive pilot provided new child-centered measure of well-being, which would be important components of an ELSCYP. The second element of this WP is phase 2 of the process pilot, which involves a systematic analysis of the information collected in phase 1, including a cost-benefit analysis.
Project Results:
Table 1: Overview of the research activities undertaken
Work Package Name Research Activities
WP1 Stocktaking and design N/A.
WP2 Mapping existing policies, studies, and data Review of 256 policies across Europe, 457 sources for administrative data, 370 sources for research data, and interviews with 83 key informants.
WP3 State of the art in the literature and practice across the EU and beyond 411 papers identified on the concept of survey methods, of which 75 were included in the review
8 717 papers identified on the concept of well-being, of which 121 were included in the review.
WP4 Direct engagement with young people Interviews with 440 children and young people in 11 European countries, from 9 to 24 years old.
WP5 Options appraisal Delphi survey engaging 334 experts across Europe.
WP6 Survey design Review of 65 surveys (European and world wide longitudinal surveys and key European cross-sectional surveys).
WP7 Pilot surveys 195 cognitive interviews with children aged 7 and 8 in 6 European countries.
WP8 Policy and impact Workshops involving 51 experts and 51 young people (aged 12 to 24) in 6 European countries.
WP9 Dissemination Dissemination activities across Europe and beyond (e.g. USA, South Africa).
WP10 Project management N/A.
1 Theoretical perspectives on children and youth well-being.
Holistic approach. Well-being is a complex concept which can be understood from different perspectives. There is a consensus that an ELSCYPWB (European Longitudinal Study of Children and Young People’s Well-Being) should have a holistic approach comprising the following premises:
• Hedonic and eudaimonic perspectives. Evidence from a number of investigations indicates that well-being is best conceived as a multidimensional phenomenon that includes aspects of both the hedonic (subjective) and eudaimonic (psychological) conceptions of well-being. The questionnaire will be based on a holistic approach focusing on both aspects. In other words, the psychological, affective (both positive and negative), and cognitive aspects (life satisfaction and its domain specific satisfaction) of well-being will be explored. However, it is argued that the affective aspect of well-being is less stable over time because of its reliance on people's experience of positive and negative emotions, which typically varies over time. Therefore, in the proposed questionnaire the measurement of affect has a limited space and life satisfaction is prioritised.
• Children’s perspective. Childhood is not merely a preparation for adult life, but a life stage in its own right. Children have to be considered as social actors and therefore their conceptions of well-being have to be taken into account together with those issues that research has identified to be important for children’s well-being. Therefore, the proposed questionnaire also responds to the children’s perspective and conceptions of well-being.
• Objective and subjective measures. Both objective and subjective measures are crucial for the understanding of children’s well-being. With regard to subjective measures, both hedonic and eudaimonic measures should be included. With regard to objective measures, the questionnaire seeks an equilibrium between those that focus on the quality of their lives in the present (Children’s rights perspective) and those preventative indicators that can help researchers to predict their future well-being (Developmental perspective). Some preventative variables have been included. However, the first wave draft questionnaire is focused on children’s Subjective Well-Being (SWB) and therefore these issues have a bigger role. Preventative indicators should have a bigger role in the parent’s questionnaire.
Material conditions. Material conditions of existence are fundamental to understand children’s well-being and its determinants. However, these objective measures can be gathered not only directly from the children but also from other sources, such as their parents. Considering the early age of the primary respondents, these external sources are probably more reliable to gather this kind of information, thus enabling children to focus on their perception and evaluation of their living material conditions.
Independent, explanatory variables. An overview of pre-existing questionnaires on children’s well-being gives rise to an important question about the depth of analysis that general well-being surveys allow for. If the questionnaires are centred on subjective well-being, and even if there is a complementary questionnaire for parents focused on material conditions, there may be little capacity of analysis of the elements that are linked, or explain, the well-being results. That is, if the questionnaire asks for well-being in the relationship with peers but does not contain information about practices (number of friends, meeting frequency, type of relationship, etc.) the analysis of the information will be mostly descriptive. The general surveys on children’s well-being that have been analysed (ISCWeb and Millenium Cohort) somehow respond to this patron; on the other hand, thematic questionnaires such as PISA, HBSC, etc. combine both subjective questions about well-being and objective questions about practices – allowing, apparently, for a deeper analysis. The proposed questionnaire combines both subjective and practice descriptive measures.
Dynamic approach. In order to grasp causality, there is also a consensus that the study of well-being needs a dynamic approach. This stems from the fact that well-being changes over time and that understanding the processes that determine well-being is even more relevant than measuring it. It has been concluded that a longitudinal perspective is necessary and that a panel survey is the option that best responds to this need. The proposed questionnaire should be the first wave questionnaire of an ELSCYPWB designed to grasp change and its effects on children’s well-being.
2 Involving children and young people in research and instrument design.
MYWeB is underpinned by a belief that involving CYP in research is both a democratic requirement and an effective tool to better design and carry out research, as if their points of views are not taken into account, basic mistakes can be made in the approach to the subject, the instruments that are used and the results’ analysis. CYP can offer unique insights of a view that can be different from adults’ views. In agreement with the UN Convention of the Rights of the Child, MYWeB holds that children should participate in their own concerns.
See Figure 1: Children and young people's engagement in MYWeB in PDF version of this report
The Concept of well-being as understood by CYP
Well-being in relation to CYP is generally assumed to involve relationships: family (parents, siblings, extended family), friends and school based relationships (teachers and school mates). Studies of well-being cover:
• Material conditions. Wide approach: family incomes, neighbourhood, public transport, access to healthcare, family’s legal status, etc. This is more prevalent if they come from working class backgrounds.
• Studies and education. The school/institute frames their lives (homework, exams, stressful situations, bullying).
• Other issues (health, disabilities, maturity, self-acceptance, partnership).
The MYWeB fieldwork also found that the meaning and importance of these concepts are not homogeneous between countries:
• The meaning of well-being is not clear. This is evident in the context of international research, with cultural and translation-related issues. It has been found that the direct use of the abstract concept of well-being is problematic and that CYP are more comfortable with experience-centred questions.
• In addition, the fieldwork has highlighted that the theoretical divisions of subjective well-being are not clear for CYP. The borders between satisfaction, happiness and psychological well-being are not always distinguished and they should not to be included as direct items in the questionnaire.
Finally, the fieldwork also highlighted that:
• Involvement in social and political groups is important for their understanding of well-being: the more active CYP are, the more their discourse and values tend to include a social and community dimension.
• Many CYP pointed out the material conditions as their main focus of subjective well-being. An ELSCYPWB should therefore provide information about their context and material living conditions, in order to better understand the determinants of subjective well-being.
The perception of well-being is greatly transformed as children grow up into young people. Growing means broadening one’s environment, relationships and interests:
• Children tend to base their well-being on family and school;
• Teenagers are in a process of identity construction and peer relationships and self-perception are basic for their well-being; and
• Young people tend to have a wider range of interests and sources of well-being and worries (studies, work, social and political involvement, etc.)
Therefore, an ELSCYPWB should have a developmental approach. However, it is difficult to draw a clear age-based line as their worries and interests do not only depend on their material situation (household, main activity, etc.) but also on subjective, psychological aspects.
The fieldwork identified specific well-being situations of particular groups and therefore the need for inclusive research:
• CYP from ethnic minorities or a low social origin and with disabilities or a particular legal status have special needs.
• It is important to make efforts to stimulate “apathetic” CYP to engage in a longitudinal survey.
Our research also suggests that there is a need for some level of qualitative data collection to better include the needs of marginalised groups, but also to grasp the meaning of some situations difficult to measure using a quantitative tool. It may, therefore, be important to complement a survey with qualitative tools for a comprehensive understanding of well-being.
Survey engagement:
Most of the CYP we studied considered that it was important to study their well-being, as this type of research allows them to:
a) express themselves and voice their views; which should lead to
b) a better understanding of their needs, which in turn should lead to
c) an improvement of their situation in society.
They further stated that highlighting the potential of research to improve their situation is a key element to encouraging participation. Therefore, giving timely feedback to participants is important to stimulate long-term participation in an ELSCYPWB. This should serve to counterbalance the scepticism that some young people expressed about the impact that research can have in their situation. In addition, some young people considered that a material incentive would encourage participation among their counterparts who might be less engaged in society.
Involving CYP in research is a key element to ensure the success of an ELSCYPWB. CYP have enlightened MYWeB with basic information that directly affect the project.
The well-being and needs of CYP often are associated with factors that cannot be pre-defined by adult researchers. For example:
• Questionnaire design: the cognitive testing has shown that CYP must be involved in designing the questionnaire, otherwise fundamental mistakes are made.
• CYP involvement in the survey: several strategies suggested by CYP should be taken into account in order to increase participation rates.
• The impact of the research will be greater with the involvement of CYP, through giving feedback about the results to the participants.
These issues would be a basic requirement in national research. In the framework of an international research, they must necessarily be taken into account to guarantee the success of the project.
The Delphi Survey and the Role of Children
Respondents largely agreed that children’s (85% of respondents) and young people’s (94% of respondents) views are an important input into designing the research instrument. This was particularly the case for young people (68% strongly agree) compared to children (45% strongly agree). Respondents working for NGOs had a tendency to consider children’s views as more important than those working in policy or research. Qualitative comments indicated that CYP views are important because they provide a unique insight on children and young people’s well-being, which can be quite different from those expressed by adults.
A number of respondents also mentioned the UN Convention on the Rights of the Child, which stipulates that children have the right to participate in decisions that concern them.
The majority (56.8%) of respondents consider that evidence-measuring children’s and young people’s well-being should always include views from parents and primary carers. However, a third of the respondents (36.8%) considered that information from parents and primary carers is necessary only for some age groups. Some respondents suggested that evidence should also include a wider range of views (teachers, health professionals etc.).
Research undertaken with CYP on their understanding of well-being for this project indicated that that some young people are interested in participating in such a study, because they perceive interviews as a chance to express themselves in front of someone who is willing to listen to them. Children and young people also indicated that incentives would enhance their willingness to participate. Delphi participants considered this information and there was a very strong consensus among participants (93.1%) that complementary qualitative interviews should be introduced in a longitudinal survey in order to enhance participation. Respondents also largely agree (66.7%) that incentives should be introduced to enhance participation.
Further analysis of the qualitative work undertaken with children and young people revealed that their will to participate in research often draws on the fact that it is an opportunity to express their views and that it could help other children and young people. They are interested in the impact of the research. This can be an important factor in empowering children and young people and keeping them engaged throughout the course of a longitudinal survey.
Experts suggested the following methods to disseminate findings: Social and multimedia such as videos, podcasts, and apps (cited 74 times), events such as seminars and workshops or participation camps (cited 22 times), presentations in schools (cited 14 times), and making sure the content is targeted to different age groups (cited 12 times).
3 Well-being data in Europe
Existing data on child and youth well-being
Objective aspects of well-being are captured in both administratively collected data and survey data. Subjective aspects of well-being, however, are collected in surveys only.
Much of the survey data that is collected on the well-being of CYP in Europe is cross-sectional with some notable exceptions where there are birth cohort and dedicated children cohort studies. Topics which are well covered tend to be measures of material well-being such as: socio-demographic and economic inequalities and education and health. Surveys supplement this with measures of abstract concepts, such as happiness, as well as of relationships.
Data gaps
There are significant gaps in data availability. Firstly in terms of countries where there is a distinct lack of data on CYP well-being across Europe, but in particular in post-Soviet countries. This is true in terms of both administrative and survey data. In addition, children and young people’s views on well-being are not routinely into account. Existing systems therefore miss aspects that are important from the subjective child/youth perspective. There are only a few longitudinal surveys, which cover well-being topics for CYP. Furthermore, psychological aspects of well-being are not routinely measured. Only a few surveys comprise children below the age of 11 years hence there is a tendency to miss important school aged experiences. Vulnerable (minority) groups tend not covered by surveys (e.g. Roma, children/young people in homeless hostels, disabled children/young people). This is partly a technical artefact of sampling whereby they are both less prevalent and ‘hard to reach’ but there appears to be little effort to broach the issue of representing these groups adequately.
In some cases there are topics which are hardly covered if at all for example, culture, gender, environment.
The Delphi Survey and the existence and desirability of well-being data
The review of research studies on children’s and young people’s well-being demonstrated that there is a large range of studies that cover the concept of well-being. A number of quantitative surveys, in particular, have been dedicated to this topic during recent years. Many have been conducted in classical survey modes such as Paper and Pencil (sometimes carried out in school classes) and face-to-face (often combined with CAPI support). In most cases, cross-sectional studies were conducted.
Coming to qualitative surveys it is important to note that in-depth interviews and focus groups are covered well in children`s/young people`s well-being studies.
Administrative data contributing to illustrations of children’s/young people’s well-being can be considered an important source, supplementing quantitative or qualitative studies. Administrative data is, in most cases, annually updated and almost always reflects the whole national population. Additionally, most administrative data sets gathered allow for an age distribution in order to consider children or young people separately.
Turning to the content of the gathered data sets, socio-demographics, economical aspects, education and health are included in many of both, survey and administrative data. Additionally, relationships to peers and parents are covered well in quantitative and qualitative research studies. The advice for combining objective and subjective measurements of well-being as it was, for instance, expressed by Stiglitz, Sen and Fitoussi (2009), is followed by most of the current studies. Also, abstract concepts like happiness that are hard to measure and thus, mean a barrier in survey planning processes, are included in studies quite often.
Administrative data particularly contributes statistics on objective well-being aspects that are related to the state (e.g. figures on taxes, social benefits, unemployment, school achievements and enrolment rates and crime statistics).
Which gaps are obvious?
There are recent studies on children’s/young people’s well-being in all EU countries (except in Cypress and Luxembourg). However, East European (Bulgaria, Czech Republic, Hungary, Romania, Lithuania, Poland) and smaller (Denmark, Ireland, Malta) countries are not represented often compared to other European countries. The Netherlands and Sweden stand out in this regard. Fewer administrative data sets were reported for the East European countries mentioned above as well as Slovakia. Thus, it is of particular importance to include East European countries in future surveys on children’s/young people’s well-being, to receive an overall European picture.
Recent well-being surveys also consider the children’s/young people’s views of well-being (Mason & Danby, 2011). To ensure the inclusion of this aspect, it is necessary to have qualitative studies in well-being research that can provide the basis for developing well-being concepts used in quantitative surveys. This has not been taken account of adequately in Europe so far. Instead, quantitative surveys relying on adults’ conceptualisation of well-being have been more common.
Innovative survey modes that appeal particularly to young people (e.g. implementing video sequences, avatars or other web 2.0 innovations) have not been used so far in wellbeing surveys. The main reason for this is the preference of classical survey modes (Paper and Pencil, Face-to-Face). New and innovative survey modes hold several unique possibilities for surveying young people that should be at least piloted in studies on child and youth well-being.
In both research studies and administrative data sets, topics like the environment, risky behaviour, safety, culture and participation tend not to be covered in depth. Although, these topics are covered to some extent in some data sets, detailed insights into them cannot be delivered by existing data. Administrative data sources especially lack information on indirect measurements, such as the provision of free school meals or other basic needs, Information and Communication Technology devices in school and at home (economical), immunisation, doing sports, weight information, nutrition, breastfeeding (health), (il)literacy, learning difficulties, bullying (education), overcrowding, noise pollution and park frequencies (environment). For culture and participation, even broader categories are not covered well by the existing data.
The surveys have not always explored psychological aspects of well-being as has been proposed by Ryan and Deci (2001) (see also Ryff & Keyes, 1995). A number of single and multi-item scales and indices measuring well-being across a variety of domains have been developed and tested in recent years, both in national and transcultural surveys (see for instance the Brief Multidimensional Students’ Life Satisfaction Scale by Seligson et al. 2003, the Satisfaction with Life Scale by Diener et al. 1985 and the Personal Well-Being Index for school children by Cummins and Lau 2005). Already existing scales and indices should be taken into account within the MYWeB project, particularly ones that were especially developed for measuring children’s/young people’s well-being and tested in international surveys. MYWeB will also consider the ground-breaking work done by researchers involved in the International Survey of Children’s Well-being (http://www.iscweb.org).
Data coverage of particular population groups varies strongly. For instance, young carers are a group for which data was considered not to be covered at all by 57.6% of the respondents. Over a third of the respondents also indicated that data is not covered at all for absent parenting (43.2%) and children and young people who have suffered significant harm (38%).
Data coverage is better for children and young people from single households, children and young people from low-income families, children and young people from a minority background, and children and young people with physical disabilities.
Furthermore, difference across countries can be very important (e.g. lack of data about the number of young people living with a physical disability in Georgia or lack of data collected about ethnic minorities in France). However, there is a strong consensus amongst experts than a cross-European longitudinal survey should gather data from all children and young people, rather than focus on specific groups.
4 Policy context and need
What is already covered well?
In general, most of the EU countries embed child and youth policy into all three levels of policy making: the municipal, district and national level; while the legislative frame is put on national level (with the exception of Spain, where the responsibility for youth is devolved exclusively to the regional or local level). Generally, local government has the role of implementing the policies on local level according to specific needs and circumstances, and according to their respective budget. Usually in the form of a ministry, department or office, nearly all EU countries have a national governmental authority responsible for youth. They range widely in financial resources, cross-sectorial influence, integration, and responsibility. While some countries have a dedicated ministry for youth, most of the others cover youth issues as part of a wider portfolio.
Generally, laws cover the main components of objective wellbeing – be it independent child/youth laws or general laws that also cover the matters and interests of children and young people. There are rights that are specific to young people, such as protection of young workers, juvenile justice and laws that ensure National Youth Councils. For children, there are basic rights ensuring child protection, their best interests and non-discrimination. In nearly all countries, the legislative framework is supplemented by existing action plans and programs for child and youth wellbeing, mostly promoted by the EU.
New programmes and draft laws in EU countries show that the topic of child/youth wellbeing is constantly being revised and adjusted to the present needs of those countries. There are promising developments around the institutional structure of youth policies (e.g. integrating municipalities into youth services, institutional changes that heighten transparency about existing measures, the creation of new National Councils, and the strengthening of regional youth policy) and the promotion of early child education and childcare systems.
Which gaps are obvious?
A significant conceptual problem is defining the age groups covered by children and young people in relation to policies on wellbeing. Children’s age groups are less often specified than young people’s age groups.
A significant gap identified through the interview programme and policy capture was between the scope of child and youth policies.
• Youth policy on wellbeing: There are still five countries that have no elaborated independent youth law: Denmark, Spain, France, Italy, and Sweden. In these cases, youth rights are part of non-age specific, general laws and legislations. Laws concerning child and youth matters differ considerably with regard to their quality and coverage (e.g. juvenile justice or the actual participation possibilities in National Youth Councils). These shortcomings are the result of conceptual deficits, where concrete terms and measures are missing and therefore hinder implementation.
• Child policy on wellbeing: Compared to existing youth policies in Europe, child policy is less developed and less coherent. The policies we reviewed display a paternalistic view of children that aims at securing their basic human rights – mostly in terms of objective wellbeing indicators – while neglecting to foster their participation in decision-making processes. Indeed, the policy trends show that children are not perceived as autonomous human beings whose views and opinions are important. Perhaps because of this, children’s own subjective wellbeing is widely neglected. However, child poverty is still present in all EU countries. There are still countries that lack the basic health facilities, child protection, and basic education. Migrants, Roma children and children from large families count to the most often excluded groups – being both excluded from policies and from research focus. As long as these obvious deficits remain, it is perhaps understandable that a focus on subjective aspects of child wellbeing is of less priority.
The Delphi Survey and policy
Experts involved in the Delphi agreed that the role of evidence is important in social policy, particularly in order to have a better understanding of policy impact (65.6%), review the design of social policies (63.2%), monitor progress (60%) and measure the distribution of policy outcomes across different target groups (56.4%). There was a strong consensus amongst panel members that the evaluation of children and youth policies supports policy makers in improving policies (84.7%).
The majority of respondents (63.0%) considered that member state level policy is more important than regional or local level policies, in the context of a longitudinal cross-European survey.
Results from the Delphi survey clearly pointed out that well-being domains had high policy relevance. On a scale of one to five where five is high policy relevance, the following domains had the highest scores: education and skills (4.59) health (4.62) family and home (4.30) and personal well-being (4.07). Respondents also indicated that those domains have relatively good data coverage, except for personal well-being. The dimensions pertaining to the eudaimonic approach to well-being (i.e. competence, autonomy, purpose in life and amount of choice) display an important gap between their data coverage and their policy relevance. Qualitative comments pointed out that there are differences between European and national coverage of those domains.
Experts were asked to identify three major well-being policy challenges for children and young people. Respondents identified education (cited 89 times), poverty (cited 58 times), and health (cited 40 times) for children and education (cited 99 times), employment (cited 97 times), and participation (cited 30 times) for young people.
All the responses provided were then combined into broad categories of policy challenges organised under three themes: community/economic factors, family factors, child/young person specific factors. Experts were asked to indicate the extent to which the data collected through a longitudinal survey could address existing gaps in data coverage and therefore inform decision-making. Respondents considered that a longitudinal survey could address existing gaps in data coverage and therefore inform decision making for all of the policy challenges suggested:
“From my experience, the data collected through a longitudinal survey can play a key role in addressing existing gaps in data coverage and therefore can be valuable resource at the policy and decision making level.” (Policy expert, Delphi 2)
With regard to community/economic factors, a longitudinal survey could be particularly relevant to address data coverage gaps in relation to equality, multi-culturalism, youth migration, and joined up policies/better coordination of services.
Overall, family factors had higher scores than community factors, indicating that longitudinal surveys could provide information that is more useful at this level. Results indicate that a longitudinal survey could provide information-filling gaps in relation to education (including early years), family support (including fathers), and poverty. A few policy challenges appear to have been scored lower by respondents working in research than those working in policy or NGOs: food and nutrition, flexible working for parents, and child protection.
At a child/young person specific level, a longitudinal survey could improve the data coverage of the transition from school to work, mental health, and participation /empowerment. A number of policy challenges were scored higher by respondents working for NGOs: disabilities, young people Not in Education, Employment, or Training, addictions, resilience, and media/new technologies. Results also indicate that data collected through longitudinal surveys is less relevant to leisure and sports than the other challenges identified.
5 Advantages/Benefits of a pan-European longitudinal study
Longitudinal studies are powerful tools that facilitate the collection of data which measures individual transitions and patterns of change. They can produce unbiased data, which helps us to understand social and individual stability and change over time. They are also important in being able to understand causation processes.
Transitions: Longitudinal surveys gather “much longer continuous histories of events and transitions (...) than could be collected retrospectively in a single interview” (Lynn, 2009: 6). The data collected can also be more accurate than data gathered during a single interview, which relies on memories subject to recall errors. Furthermore, longitudinal surveys have the unique advantage of being able to compile unbiased information about expectations that can be analysed against measures of outcomes collected at a later stage.
Patterns of change over time: Longitudinal surveys allow for the measure of stability or instability and the identification of causality. Individual-level change can only be understood in the context of changes taking place over a considerable amount of time. This type of analysis enables researchers to identify patterns of change (e.g. steady growth, fluctuation around a low level, sudden decline followed by stability) (Lynn, 2009). For example, if the proportion of children and young people satisfied with their life is relatively stable over time, there might be many of them starting to feel satisfied with their lives while others are not satisfied anymore. A small proportion of children and young people might be satisfied with their lives on a continuous basis, while the majority show strong variations in time. This insight provides greater information about the dynamics and the factors associated with children and young people being satisfied with their lives. Longitudinal surveys also capture characteristics such as frequency, timing or duration. As noted by Richardson (2012), longitudinal studies make time trend analysis possible, which in the case of children and youth well-being would be useful to identify priorities in child well-being measurements and to validate child well-being statistics.
Longitudinal surveys and their impact
Surveys collect data about the economic, social, political, and cultural shape of a country. Policy makers can use this information to make informed decisions about future policies and evaluate the effectiveness of past or current policies.
Surveys can be divided in two main types: cross-sectional and longitudinal. The key difference is that cross-sectional surveys occur once whereas longitudinal surveys take place on multiple occasions over time (Lynn, 2009: 1). Cross-sectional surveys tell us how people live and what people are thinking or doing at one point in time. Repeating the same measures among the same group of respondents over a long period of time offers insights into how respondents socio-economic situation, health but also their views and attitudes and behaviour developed (Buck 2008). Longitudinal Studies thus help researchers to understand change and stability at the level of the individual, rather than for the population as a whole (Buck 2008).
Longitudinal surveys can impact on policy. More specifically a European Longitudinal Study for Children and Young People can be used to:
- monitor wellbeing;
- study the evolution of wellbeing; and
- evaluate national policy.
Monitoring wellbeing
Currently wellbeing is often monitored both nationally and internationally using high-level social indicators. Longitudinal surveys provide an alternative method of monitoring wellbeing.
Social Indicators and their limitations
Social indicators initially focused on measurement and trends in child well-being primarily using ‘survival indicators’ (Ben-Arieh, 2008) such as rates of mortality, disease, and social problems affecting children (e.g. illiteracy, school failure). Major work informed by this approach is OECD research on the comparison of child well-being across its 30 member countries (Chapple and Richardson 2009) and UNICEF publications (2007).
Most help to understand children and young people's well-being at the objective macro level. While such indicators are important to begin to redress issues of inequality and social exclusion that negatively affect children’s health and wellbeing, they provide only limited insight into social change over time and the causes of change. They also tend to ignore the potential, attributes and strengths of children and it can be argued they treat children as 'passive agents not capable of evaluating their own lives'.
Self-report surveys
An alternative to the use of social indicators is self-report surveys. A number of surveys collect self-reported data on children and young peoples’ wellbeing:
• The international Health Behaviour in School-aged Children survey covers a number of key areas of young people’s health and well-being, with 43 countries involved in the latest wave of the survey in 2009/10.
• Some large longitudinal social surveys have begun to incorporate self-report instruments for young people. For example, two household panel surveys [the European Social Survey (ESS) and the European Quality of Life Survey (EQLS)] included some questions on well-being and its various domains for young people in various age groups. For a full review of these surveys, see Richardson (2012).
• Some national, longitudinal surveys have included questions on well-being. For example, the UK Household Longitudinal Study, previously known as The British Household Panel Survey, has a youth questionnaire for young people aged 11 to 15 about their happiness, feeling troubled and self-esteem.
There are several advantages to surveys of self-reported well-being:
• international surveys among children and young people provide precious comparable data on child well-being covering countries in the EU and beyond.
• household panel surveys provide new opportunities to explore the effect of changes in young people's lives on their overall well-being.
However, the concepts and domains of well-being used in these surveys were developed primarily from concepts which originated from the study of adult well-being. Moreover, as Bradshaw (2009) argues, most of these studies include only a limited number of well-being domains and therefore do not provide the full picture on the state of well-being for children and young people.
Recently, some child and young people-centric well-being studies have been undertaken including the 2005 survey in the UK undertaken by The Children’s Society. This approach has been taken further by an international group of researchers linked to the International Society for Child Indicators. They have developed Children’s World, an international survey in 12 countries on children’s subjective well-being. However, these two surveys are cross-sectional, not longitudinal and therefore are not suitable for monitoring and evaluation.
Evaluating policy
A comparative longitudinal survey of child wellbeing offers policy-makers at a European and Member State level a number of new possibilities for policy formulation. A practical example of data from a longitudinal survey changing our understanding of a social issue and impacting policy responses is the role of the British Household Panel Survey in understanding the dynamics of poverty. Jenkins (2008) recounts how, in the 1990s inequality and poverty rates flattened off and it appeared that there was little or no change in the income distribution from one year to the next. However, the British Household Panel Survey revealed that apparent cross-sectional stability hid longitudinal flux – households’ incomes fluctuate between one year and the next, and there was substantial turnover in the membership of the low income population. Jenkins goes on to demonstrate how these findings influenced policy including much of the emphasis in the Labour government’s welfare reforms from the late 1990s that reflected a dynamic perspective with a focus on moving people into work and making work pay.
Longitudinal studies can be viewed as a form of quasi experimental evaluation design (ISER, 2002). Although they do not involve the allocation of individuals to treatment or control the temporal sequencing of longitudinal data offers a means of controlling fort the effects of independent variables (ibid.). In this sense longitudinal studies provide an evaluation design akin to an interrupted time series. As ISER (2002) note, we can at least say with certainty from the statistical analysis of biographies what events preceded others, even though we still have problems in deciding which event in relation to another event was the underlying cause.
An example cited by ISER (2002) that illustrates the use of birth cohort survey data in the evaluation of policy comes from the UK where returns to qualifications and returns to education more generally have been understood using longitudinal data. Outcomes in adulthood such as occupation and earnings are set against qualifications, taking account of ability as tested in childhood and numerous other circumstances and experiences earlier in life which might be confounded with them. ISER (2002) concludes that “Statistical modeling of this kind, is not a perfect substitute for the controls offered in randomized experiments, but goes some of the way to producing the most plausible accounts of micro economic processes.”.
While insights from longitudinal surveys have much to offer to policy it is worth noting that different designs will be associated with answering different research and policy questions.
Birth cohort studies allow researchers to chart the development of the human life course. As ISER (2002) notes, the data collected for any single birth cohort confounds age, period and cohort effects at any particular point in time. In addition, comparison of more than one cohort enables the researcher to hold constant one of these three ‘extrinsic’ factors, for example, comparing cohorts at a given age to establish a cohort effect or cohorts at the same age at different times to establish a period effect. The data can then be used prospectively to make predictions about the outcomes of particular circumstances and experiences in life occurring at particular points in time or retrospectively to identify the circumstances and experiences in earlier life that underpin a given outcome later (ISER 2002).
Household panel surveys sample the whole population rather than single years of age with the aim of understanding the dynamics of change of the whole population, and its evolution over the lifetime of the study (ISER 2002). These surveys usually follow all the people living in the sample household, not just a reference individual. The origin of household panel studies was in the need to explore the dynamics of poverty and income and an understanding that these could not be explored through separate snapshots, but rather required an approach to collecting a continuous record about income in particular (ibid.). However, the household panel survey is also particularly suited to the analysis of the dynamics of household formation and dissolution, and associated events and outcomes (ibid.).
A recent analysis of evidence-based policy has highlighted the importance of longitudinal survey data in contributing to the development of child well-being policies (Breckon and Dodson, 2016).
The Delphi Survey and the need for longitudinal data
Over three quarters of the respondents acknowledged a need to improve the lack of longitudinal perspectives (81.6%), improve coverage of particular population groups (78.8%), and improve subjective measures of well-being (75.2%).
What can a European longitudinal survey add?
In short, an ELSCYPWB would facilitate better monitoring of well-being both nationally and comparatively across Europe. This would have an impact of public policy on children and youth well-being and the likelihood that CYP well-being would be significantly enhanced.
6 Technical challenges
Implementing a pan European longitudinal survey measuring children and youth well-being comprises different sets of challenges linked to the characteristics of the survey:
• European: International surveys require conceptual comparability and translation.
• Longitudinal: Such surveys involve different sampling strategies and challenges linked to sample attrition (panel mortality).
• Children and youth: Working with children and young people requires adapting research instruments and securing parental consent.
International surveys also have challenges specific to their design. They mostly concern the validity of concepts and words across different languages and cultures. Consequently, developing an international survey requires a focus on:
Data harmonisation: Pre-testing and piloting phase are used to harmonise data. They can be either:
• Output-harmonised, where national teams develop their own questions, under the condition that they cover specific domains.
• Standardised questions that are meaningful, relevant to all countries, expressed in an equivalent manner in all relevant languages.
It is possible to have core harmonised questions complemented by local questions.
Translation: Translation occurs in both international and national surveys where versions are made available in minority languages. There is a wide range of practice across Europe, ranging from informal translation made by bi-lingual researchers to formal professional agencies. Errors appear to occur in both instances. A commonly used method is the double forward translation, with back translation and validation. MYWeB has successfully used a TRAPD (Translation, Review, Adjudication, Pretesting and Documentation) approach as described in section 9.6.
Developing child-friendly questions is central to undertaking research with children and young people. This is also explored in section 9.6.
Longitudinal studies have a number of challenges in terms of data collection and analysis (Lynn, 2009):
Sample attrition: This refers to the continued loss of respondents from the sample due to nonresponses through deaths, moving, and subjects withdrawing from the research. Some surveys successfully re-engage with sample members that have been non-respondents at one or more wave. However, the general response rate of sample units that would have responded to each questionnaire may be lower than in cross-sectional surveys.
Panel conditioning: Responses from participants may be conditioned by their previous experience of taking the survey. Panel conditioning can affect the way respondents report events, or even change respondents’ behaviour.
Coverage error: This error occurs when there is a gap between the sampling frame and total population. Longitudinal surveys are more likely to suffer from coverage error if the sample does not include, or under-represents, additions to the population since the sample was selected. This coverage error is likely to increase with time.
Time and cost: It generally takes longer to prepare a longitudinal survey than a cross-sectional one since questionnaire instruments cannot be designed independently from those used at a later stage. Overall, longitudinal surveys are more costly than cross-sectional surveys.
The Delphi Survey and the obstacles to developing and implementing an ELSCYPWB
Qualitative answers from the Delphi Survey point out to a number of challenges to surveys with children and young people. To summarise:
• Resources are one of the key obstacles to implement a cross-European longitudinal research instrument.
• Respondents also indicated that the EU and national governments might not be interested and that differences between countries would represent an important challenge (culture, policies, languages, data protection).
• In terms of methodology, the majority of challenges pertain to reaching an agreement about measures, concepts, target group, and general methodology.
• In terms of implementation, some respondents noted the lack of capacity to implement such a survey (i.e. trained fieldworkers or lack of institutional capacity), as well as the challenges surrounding getting parents’ participation and consent.
• Challenges to surveys with young people showed similar results, with an emphasis on their participation and migration / mobility.
7 Proposed methodology
7.1. Political desirability
The Delphi Survey and the political desirability of an ELSCYPWB
A large majority (73.2%) considered that a cross-European longitudinal survey was desirable, with 55.6% considering it to be very desirable. When considering the political feasibility of a longitudinal child and youth well-being survey, respondents largely agreed with the following statements:
• It is important that Member State governments are supportive of the survey if it is to be implemented (99% agree, of which 59.4% strongly agree).
• Demonstrating that the survey will have continuing policy relevance for many years to come will be important if the survey is to be implemented (99% agree, of which 56.8% strongly agree).
• It is important that significant research groups in each Member State are supportive if the survey is to be implemented (96.3% agree, of which 45.5% strongly agree).
• It is important that significant NGOs in each Member State are supportive of the survey if it is to be implemented (89.9 agree, of which 31.9 strongly agree).
Funders and stakeholders
As longitudinal surveys are significantly more expensive than cross-sectional surveys, the funders and stakeholders are almost entirely government-linked. This can be directly through one or more governmental departments, or indirectly through a national, government funded, research organisation. The purpose and content of such surveys tends to reflect the priorities of stakeholders. Medical, economic (including employment and education) and social government departments are most likely to be involved in some way in the commissioning, development and analysis.
7.2. Technical feasibility
The Delphi Survey and the technical feasibility of an ELSCYPWB
About half of the respondents (59%) consider the cross-European longitudinal survey to be technically feasible. There was a general agreement that it is feasible to build a longitudinal survey from a pre-existing Europe-wide cross-sectional survey. A higher percentage of respondents working in research/academia (38.7%) indicated that this was not feasible than respondents working in policy (28.1%) or NGOs (31.6%). There was no strong agreement about the feasibility of linking a longitudinal survey to pre-existing national administrative data.
The majority of respondents (59%) agreed that the survey should focus on the entire life-course of a young person from birth to the age of 25. Some respondents indicated that the age limit of a young person should be extended to 30 years old to reflect societal changes (e.g. young people still living at home because of studies or unemployment).
Respondents generally agreed that prenatal information could improve child well-being social policy. A greater percentage of respondents working in policy (93.9%) agreed with this statement than those working in research (82.7%). Qualitative comments indicated that a better understanding of the prenatal period could improve preventative policy.
The results from the first questionnaire showed that the majority of respondents (77.2%) agreed that standardised European measures should be complemented by country-specific modules or questions. This was further explored in the second questionnaire during which experts were asked to indicate whether well-being domains can be captured through standardised European measures or if they can only be captured using nationally specific measures. The majority of respondents indicated that standardised European measures are applicable to the different domains of well-being: competence (84.7%), health (84.5%), education and skills (81.9%), time use (70.2%), and personal well-being (70.0%), amount of choice (67%), personal appearance (63.7%), money and possession (62.9%), relationship with peers (62.3%), purpose in life (58.9%) and family and home (55.4%). However, the majority of respondents (58.9%) considered community and neighbourhood to be a domain that could only be captured using nationally specific measures. This issue was clarified in the third questionnaire, where 81.0% of the respondents answered that it is feasible to develop a classification of neighbourhoods on the basis of an index of relative deprivation that is comparable across Europe.
Yet, concerns were expressed in relation to the complexity of the task, its cost, different meanings attached in different countries to categories such as homeownership, and how one’s subjective feeling about their neighbourhood impacts their well-being more than objective indicators.
Experts suggested that a cross-European well-being survey could be made more sensitive to local variations concerning community and neighbourhood by using additional measurement for each country and/or qualitative measures. It was also suggested that a local expert panel should be appointed in each country to analyse and clean the data.
Whilst respondents working in policy or NGOs considered this feasible (63.9% and 61% respectively), those working in research indicated (53.6%) that it is not feasible.
As indicated in the previous section, obstacles to developing and implementing cross-European longitudinal research instruments identified in the first questionnaire include resources (cited 95 times), European Union and national governments’ support or interest (cited 50 times), challenges pertaining to methodology (cited 132 times), differences between countries (cited 88 times), implementation (cited 57 times), data analysis (41 times), access to particular groups (cited 9 times), and utilisation (cited 6 times).
In the second questionnaire, respondents indicated that the following criteria were feasible, on a scale of one to five where five is very feasible: building child friendly questions that allow adequate comprehension and suitable response modes (4.28) translating the survey into European languages, that will result in robustly comparable international data (4.20) to achieve a representative sample survey in each country (4.12) managing a large and complex data set (4.09) and to obtain children and young people’s informed consent (3.92). Some items scored slightly lower such as: implementing consistent fieldwork practice (3.79) to keep the sample members in future data collection phases (3.66) and getting parental consent (3.62).
7.3. Preparation and Piloting
It is recognised that the preparation of a longitudinal survey is a time consuming process that takes several years. For instance, the ALSPAC (Avon Longitudinal Study of Parents and Children, UK) study required five years of preparation. This involves a pre-test period, which usually takes 12 months. Common pitfalls that slow down the pre-test period include delay in deciding the content, difficulties with developing the questionnaire, and availability of field resources.
Piloting is an essential phase of survey design. It can involve different stages used to progressively test the questionnaire and delivery procedures. For example, an early pilot phase would be used to start testing the flow of the questionnaire and to identify problematic questions. It can be followed by a pre-test or cognitive interviews, used to evaluate questionnaire length, identify further issues with specific questions and their sequencing and consider training requirements. Finally, a dress rehearsal tests the final version of the questionnaire and the procedures to be used in wave 1.
Governance structures are important to ensure the successful implementation of a survey. Their early establishment contributes to rigorous piloting. Structures, such as Research Design and/or Ethic Subcommittees, are often established to support this process. Quality control can also be inserted when external institutions oversee piloting. Furthermore, consultation with the target population (i.e. children) guarantees that their voices are captured in the research design. This engagement can be formalised in structures, such as a Children Advisory Forum, which acts as a feedback mechanism throughout the development of the project. Consultations with other experts, such as academics and government departments, also strengthen the research design.
Piloting usually takes place one to two years before the fieldwork starts. In some cases, piloting took as many as five years (ALSPAC). It is important to allow for a realistic timeframe, as practical reasons (e.g. delay in deciding content, difficulties with developing the questionnaire, availability of field resources) can create delays.
The piloting phase should test the following:
• Instrument design (e.g. theory and hypothesis underpinning, methodology)
• Sampling procedure (e.g. feasibility, logistics, cost efficacy, sampling biases, weighting)
• Data collection tools (e.g. sequence of questions, understanding of questions, question wording, age appropriateness, length of questionnaire, scales, standardisation and validity)
• Field procedures (e.g. incentive strategies, recruitment strategies, data collection modes, data collection environments such as home or school)
Whilst an exhaustive piloting is required before the first wave of data collection, lessons need to be incorporated continuously throughout the life of the survey. Best practice would involve a piloting phase before each wave of data collection. This phase can be limited to testing new questions added to the questionnaire and can involve a small number of participants (i.e. about 20 participants for POLPAN or DJI Kinderpanel) or be more systematic, such as for the Millennium Cohort Study, which had a consultation, a pre-test and a dress rehearsal before each wave.
Finally, piloting the survey is also an opportunity to familiarise oneself with the research tools and procedures. Training is an essential part of quality control. Lessons from the international survey SPARCLE conclude research associates need to speak sufficient English to take part in workshops that focus on administering questionnaires, engaging children, disability issues and the rationale for the study. In this case, research associates conducted five pilot visits in their own countries and met again for a second training workshop to discuss challenges and identify potential solutions. A successful pilot is also a means to securing grants for the survey.
7.3.1 Recruitment strategy and promotional campaigns
Recruitment to a survey can be promoted with publicity (e.g. via YouTube, press releases, media appearances, posters) and through the offering of incentives such as vouchers. Promotional advertising appears more intense when longitudinal surveys are regional rather than national. For instance, ALSPAC had considerable local and national coverage in the press, radio and television. Alternatively, it can be achieved through running recruitment events, and/or consultation exercises with communities and institutions that host potential respondents. Recruitment also works better when contact is made before the actual stage of recruitment (e.g. by letter between a month and two weeks in advance). Where potential participants are slow to sign up to a sampling programme, reminders can be sent to them after initial approach (e.g. after two weeks) by post or through other communication mediums. Where potential recruits are reluctant to sign up to a programme, a research team member can productively be dedicated to the task of enrolling such individuals. As some refusals are down to potentially temporary reasons, it is good practice to try to enrol un-enrolled individuals/institutions approximately 2 months after first refusal.
Contact details for recruits can be sourced from institutions, such as tax offices and education departments. Recruitment is more successful if it is part of a universally mandatory process in a country (e.g. a medical exam for all children starting schooling).
Recruitment approaches are most likely well-received when they reduce burden and appear organised. Such approaches attract goodwill that can lead to less attrition at a later date. Researchers can appear organised and smooth the recruitment process, by sending enrolment packs to individuals/institutions in advance of a study. These might include research handbooks, consent forms and other preparatory materials. Personal touches, such as the signing of recruitment letters, are also advantageous in generating goodwill. The process of recruitment can also be eased and simplified by recruiting gatekeepers (i.e. reputable institutions/organisations) that can facilitate access to large numbers of recruits. For instance, convincing a Ministry of Education of the need for a study can deliver subsequent access to the many schools that take a lead from the Ministry. In turn, those schools can deliver parents/children who take their lead from that school.
As part of recruitment, it is desirable to inform participants of the purpose of research, both for ethical reasons and because more informed samples are less likely to suffer attrition as a study progresses. This can be done through approach letters, websites and primary approach brochures. Where parents are to be enrolled, letters can be sent to mothers and fathers before birth.
Participants in a study should also be given the contact details of qualified people outside of the study who will be able to advise them (e.g. a support line) without them fearing reprisal if they have concerns about their participation in the study. The recruitment process should be one sensitive to individual needs. At risk participants should be recruited and engaged in formats they are comfortable with (e.g. young mothers should be recruited by fellow females).
7.4. Preferred option
The Delphi Survey and the preferred option for an ELSCYPWB.
The first questionnaire did not bring consensus amongst experts about the survey design. Overall 56.6% of the respondents indicated that the most suitable design would be a cohort design whereas 43.4% indicated it would be a household panel design. When asked about further options in the second round, respondents indicated a preference for an accelerated cohort design where data collection would start simultaneously with different age cohorts covering a specific life span. This contrasts with both a narrow age based cohort design which traces a single age cohort as they grow up and a wide age sample which does not differentiate specific age cohorts. The accelerated cohort design was particularly popular for respondents working in policy as illustrated in Figure 2.
see Figure 2: Survey design preference by respondents' background om PDF version of this report
A third of the experts (31%) indicated that if possible, a cross-European longitudinal survey should have an unlimited length. Another third (30%) indicated it should last between 10 and 15 years. Responses indicate that an interval of either two (20.7%) or three years (39.0%) between survey waves was preferable.
An exhaustive selection exercise was undertaken by the project team in conjunction with the Delphi participants to consider all options to use/collect data on children and young people’s well-being from doing nothing, through using administrative data to completely new Europe -wide surveys as shown in Figure 3.
see Figure 3: Options considered to use/collect data on children and young people’s well-being in PDF version of this report
The option that commanded the broadest support was a new, pan-European ‘accelerated’ birth cohort survey with a series of age specific cohorts from birth to 25, based on nationally representative samples in each Member State. Currently, in the UK the Growing up in Scotland (GUS) survey is the closest equivalent to such a survey (see).
See Table2: UK survey options in PDF version of this report
7.5. Accelerated cohort study
A Longitudinal cohort development analysis facilitates analysis of age, period and cohort. Parallel cohorts allow a comparison of different cohorts from the outset. This is not truly longitudinal, but contributes to early analysis of age effects. Using a range of cohorts has the advantage of not fixing the survey to a single point in history, which gives partial control over acute period effects. In addition, it creates policy relevant data for several cohorts starting from the first wave. As with all longitudinal designs, it enables the identification of transition periods and turning points relevant to policy and helps to detect important periods for policy intervention. Moreover, it helps to identify important issues in high impact policy areas (for instance education, family unit etc.). The accelerated cohort design enables policy impact evaluation as well as macro and micro level programme process evaluation. The on-going nature of the survey means that policy adjustments can be made and impact evaluations undertaken for subsequent cohorts. Importantly, through the data analysis and policy interventions, an accelerated design enables medium term cost savings to be made at a national level. Lastly, assuming that an accelerated survey was rolled out across Europe, this survey design facilitates both national level policy evaluation and EU level comparisons.
A Europe-wide national sample accelerated cohort survey
This, in essence, is a series of national birth cohorts rolled out in parallel on different age cohorts. In this sense, they are independent surveys as there are different sampling designs and different survey instruments. What unites them is a common purpose – to study a single set of phenomena for a range of age cohorts as each cohort gets older.
Advantages of an accelerated cohort design
Longitudinal cohort development analysis facilitates analysis of age, period and cohort. The parallel cohorts allow a comparison of different cohorts from the outset. This is not truly longitudinal but contributes to early analysis of age effects. Using a range of cohorts does not fix the survey to a single point in history, which gives partial control over acute period effects. Feedback to children and young people could include a temporal dimension (i.e. how are the young people two years older than me doing?). It would also engage with the same children and young people who do not currently feel that their voice is heard in society. Children and young people are interested in hearing about how the research influences policy. Policy relevant information is generated across several cohorts of young people and enables policy makers to compare life cycle changes and policy impact across different cohorts within waves and as time progresses, same age cohorts across time. The survey helps to identify transition and intervention points and may make policy design more efficient and interventions more effective. The wealth and complexity of data allows researchers to make multidimensional comparisons and enables macro-level programme process evaluation and policy adjustment for subsequent waves. Overall, policy effort and effort in specific policy areas (education, childcare provision, family welfare etc.) can be compared across different regions in EU member states.
Disadvantages of an accelerated cohort design
An accelerated birth cohort survey introduces operational complexity, much of which is frontloaded. The success of this research design is underpinned by rigorous scoping and planning for which a highly structured and sequential forward planning methodology is required, which assess the risks at each stage.
Longitudinal surveys are costly, none more so than large sample cohort surveys. This is addressed below with a consideration of costs weighed against benefits.
see Figure 4: Accelerated cohort survey: phased design in PDF version of this report
7.6. Testing well-being questions
7.6.1 Comprehension of different question formulations and wordings
Findings related to children’s comprehension of different question formulations and wordings confirmed the importance of using simple and short questions, with common everyday expressions and phrases that are directly related to children’s experiences, concrete activities and behaviours. All things considered, children without major difficulties most frequently comprehend questions that enquire about frequencies of specific activities, behaviours or affect. From this study, these are questions about child-parent relationships, bullying, and children’s positive and negative affects. Conversely, findings across countries show that more complex questions, consisting of several parts or conditional formulations (such as those aiming to capture children’s autonomy and the level of respecting the child's voice) are frequently hard to understand. In such cases, children often overlook some parts and only respond to the simpler, specific parts of the question.
Additional issues related to such questions are intended concepts (e.g. autonomy) relevant to children at this age and if they are relevant, how they manifest in terms of children’s everyday activities and behaviours which could be used for more appropriate question formulations. Findings of this study suggest that some of the tested questions (e.g. How frequently you choose what you eat for breakfast?) could be a good starting point.
Regarding the comprehension of specific words or phrases, problems were detected primarily for expressions that were more ambiguous (adult, normally, enough), not age appropriate (e.g. decide, complaint), used in figurative senses (e.g. one thing) or not embedded in a specific culture (e.g. hobby, being happy with).
Additionally, findings of the CI study demonstrate difficulties in finding the right balance between the level of generality and specificity of question formulation in surveys for young children. As some of the described examples demonstrate, by adding a particular context or some explanation, a question became less prone to different interpretations and easier to comprehend.
7.6.2 Response scale types
Findings that relate to children’s recalling and judgement with regard to several types of response scales suggest that simple frequency scales with general verbal responses such as: ‘Always’, ‘Sometimes’, ‘Never’ or ‘Always’, ‘Often’, ‘Rarely’ (‘Not so often’), ‘Never’ are relatively the most applicable scales for survey questions for young children (eight and seven year olds).
Response scales with smileys should be avoided as a general response scale. Children interpreted the smiley-scale out of the question context, and their responses took into account their general level of happiness.
Regarding the Likert type scale, although children correctly understood the meaning of verbal responses, recalling and judgement with such scales was more complex for children than in the case of the frequency scale that is more compatible with the question formulation applied.
With reference to the frequency scales tested in this study, the simplest - three point scale (‘Always’, ‘Sometimes’, ‘Never’) can be recommended as the most applicable. This scale was mostly understood and suitable for seven and eight years old children. At the same time, a simple general four-point frequency scale (‘Always’, ‘Often’, ‘Rarely’, ‘Never’) was appropriate for eight years old children only. Namely, the use of this scale among seven years old children pinpointed some difficulties in understanding the differences between ‘Rarely’ and ‘Often’. The general four-point frequency scale referring to days (‘Every day’, ‘Most days’, ‘Sometimes’, ‘Never’) was appropriate for eight year old children and can also be used with seven year old children without major difficulties. However, the four-point frequency scales referring to number of days (‘Every day’, ‘Most days’, ‘Once or twice’, ‘Never’) or to number of times (‘Four or more times’, ‘Two or three times’, ‘Once’, ‘Never’) were not so applicable since problems in general understanding and differentiating between the scale points were noted.
Open-ended questions, in general, could be used among seven and eight year old children, especially if they required short and simple answers. However, they cannot be recommended as a preferred option for the questionnaire in the English language. Since it could be expected that younger children (seven year olds) will still have some problems with spelling in open-ended questions at that age, it is suggested that this should be substituted with more simple multiple-choice questions.
Generally speaking, findings suggest that response scales and the offered verbal points should, as closely as possible, match the question formulation. More specifically, if a question starts with the general formulation such as ‘Do you...’, ‘Have you...’, ‘Can you...’ children spontaneously expect and respond with dichotomous answers, (e.g. ‘yes / no’ or ‘I can / I cannot’). Thus, if a frequency scale is used, the recommended introductory formulation is ‘How often you...?’
7.6.3 Time frames and context
Findings related to specific time frames and specific context suggest that seven and eight year old children were not entirely capable of recalling their experiences in an exact period of time and space. Specifically, the results of this study showed that time frames specified as: ‘last week’, ‘during this school year’ and ‘since you joined the class’ were not appropriate. Children when answering do not always only take into account the time period mentioned in the questions and/or do not understand concept of specific period uniformly.
However, not putting constraints on any given time frames was related to different recalling periods among children when answering the same questions. Thus, based on the findings of this study it is hard to give a definite recommendation about (not) using the time frame in survey questions for seven and eight year old children. General advice would be to use general formulations or either very short time frames, such as yesterday, today, while in the case of slightly longer time periods (e.g. last week) to ensure that the time-frame reference is explained in detail and unified throughout the questionnaire.
Additionally, regarding the specification of the context, findings from this study showed that, although children understood the relevant words such as ‘at school’, sometimes in answering they also refer to their experiences in other places, not only to the place/context specified in the question. Thus, the general recommendation would be to avoid specifying exact contexts if this is not necessary.
7.6.4 Other recommendations
It is clearly demonstrated that the survey questionnaire planned for self-completion by seven and eight year old children should be as short as possible. Focus of the question should refer to only one experience or type of action, while the use of conditional question formulations (‘if – then’) should be avoided. The wording of questions should be simple. Only one type of instruction for selection and marking the answer (e.g. circling) should be used throughout the whole questionnaire. Additional introductory sentences and/or instructions that are not crucial should be avoided. A simple layout in the form of one question per page is recommended. Moreover, any unnecessary content or information should be avoided in a questionnaire for eight and seven year old children, since this can distract children from answering the questions. It is also recommended that questions should be written in capital letters using a simple font of a larger size without graphical solutions (e.g. bold type) for emphasis.
Potential Impact:
Cost and benefits of a ELSCYPWB
The principal policy impact of MYWEB is in providing a platform from which policy makers understand the needs for robust evidence in order to make sound policy decisions which benefit children and young people and which are financially sustainable. This is delivered through the cost benefit analysis that places the (high) costs of an accelerated cohort survey of children and young people’s well-being across Europe alongside the money spent in social policies, which have a direct impact on children and young people. Deliverable 7.2 has a more detailed explanation of the methodology and results of the cost benefit analysis.
Financial sustainability
The Delphi Survey and the costs and benefits of an ELSCYPWB
Over a third of respondents (41%) considered the survey to be financially sustainable. Yet, the majority of respondents indicated that improving data coverage is somewhat more important than constraining the cost. This is especially the case for respondents working in research (61.1%) compared to those working in policy (52.4%) and NGOs (50.0%). About a third of the respondents consider that improving coverage is top priority. Furthermore, experts strongly agreed with the following points:
• the economic benefits of improved well-being far exceed the cost of a longitudinal survey (93.8%, of which 50.6% strongly agree)
• a longitudinal survey will support policy makers to improve policy design and impact and make policies more efficient (97.9%, of which 58.6% strongly agree)
• evidence gathered by such a survey will contribute to improved well-being (95.6%, of which 45.3% strongly agree)
About half of the respondents agreed with the following sentence: “The economic value of improved wellbeing far exceeds the cost of implementing the survey. However, these economic benefits are not easy to demonstrate (i.e. they are not 'cashable') and therefore cost would be a barrier to implementation.” This was especially the case for policy makers (57.5%) compared to researchers (48.9%) and those working in NGOs (47.6%).
The various strategies identified by the experts to demonstrate policy relevance, draw key decision makers in the process and demonstrate the economic benefits of a well-being longitudinal survey to potential funders are quite similar. They include the following recommendations: clearly identifying specific policy questions the survey will address, demonstrating the need for an evidence-based decision making process, involving the media and increasing public awareness, undertaking cost analysis (SROI, cost of reinsertion vs early intervention, cost of no survey), and providing a sustainability plan.
Costs and benefits
What level of change would be necessary for the investment to be worthwhile?
It is estimated that a European longitudinal survey of children and young people’s wellbeing will cost approximately €722 million over twenty-five years to commission (see Appendix 1 of Deliverable 7.2 for the assumptions and computations that arrive at this figure). Assuming data from this survey is used to affect changes in member state government expenditure in this area, what level of change would be necessary for this investment to be considered worthwhile? To address this question, it is important to first consider member state expenditure on children and young people’s wellbeing and then to compare the cost of the survey to this expenditure. Such a comparison will give an indication of the scale of change necessary for the investment to be considered valuable.
Drawing on published data and analysis from the OECD and Eurostat, data on current spending across EU member states on children and young people’s wellbeing is presented in two ways, namely per capita and total expenditure. Error! Reference source not found. below gives an estimate of government expenditure on children by age group in twenty EU member states in 2011. This is the most recent year for which data is available. Error! Reference source not found. draws on analysis published by the OECD and therefore only covers those EU member states that are also members of the OECD. It provides aggregate, national government expenditure on education, childcare, cash benefits and tax breaks and other benefits in kind. To enable comparison, the OECD analysis is given in per capita spend in US dollars at purchasing power parity (PPP ).
Analysing data on total spend on children and young people’s well-being is less straightforward. To do this, the OECD per capita spend by age group data for 2011 has been combined with Eurostat population data (aggregated to the relevant age group, for 2011). This provides a broad estimate of the scale of national spending by several EU member states on public children and young people’s services. The spending included in this analysis relates to welfare spending (cash benefits, tax breaks and other benefits in kind) and education spending (childcare and compulsory schooling). It should be stressed that the underlying spending data is aggregated by OECD from the individual national budgets and there are a number of significant limitations around this data (the limitations of this approach are presented in the OECD publication, Doing Better for Children (Chapple and Richardson, 2009)). Table 2 presents the underlying estimate spend (nearest billion , USD PPP) and relevant population figures.
see Table 2: Estimated gross spend by member state on child wellbeing, 2011 (USD PPP) in PDF version of this report
The estimates presented in
Table 2 above suggest a combined expenditure of around $1 trillion in 2011 in US dollars (PPP) for the twenty EU member states that are also members of the OECD. We have estimated the likely cost of a European longitudinal survey on children and young people’s wellbeing at between €722 million over a twenty-five year period (see Appendix One). The cost of the survey is therefore an extremely small fraction of overall expenditure in this field (in the order of 0.003%). This suggests that only very small improvements in the effectiveness of member state expenditure on child wellbeing would need to be affected by the availability of longitudinal survey data for the benefits of such a survey to outweigh the costs. Figure 5 below provides data on the anticipated cost of the survey by member state, in year 1, as a percentage of spend on child welfare services . This further suggests that the costs of an ELSCYPWB would be a small fraction of overall spending in this area of social policy.
see Figure 5: Estimate of member state survey spend in year 1 as share of total spend on child welfare services in PDF version of this report
Although the cost of the survey, at member state level, is a very small share of the overall spend on child wellbeing services in year 1 (assumed 2017) and across the whole life of the survey (25 years), this in itself does not give a precise picture of the level of efficiencies that would be necessary for the benefits of the survey to outweigh the costs. The potential for such efficiencies is not evenly spread between member states; some (notably the UK, Germany and Ireland) already commission several longitudinal and other surveys, which provide data on child and youth wellbeing, while others do not currently have access to such data. An ELSCYPWB would complement existing data sources in those countries with a history of investment in this area whilst providing an opportunity for those member states that have yet to make such an investment. In addition, the potential benefits of direct member state comparisons through the use of a common research instrument and fieldwork time frame suggests that there are further benefits that cannot yet be anticipated or quantified.
The analysis set out here identifies evidence that longitudinal survey data are useful to and used by, policy makers in the UK. The UK is used as an example here because it is a member state with a long history of commissioning longitudinal surveys. We have also drawn on findings from the MYWeB research, which strongly suggests that policy makers, academics, practitioners, and others involved in child wellbeing across the EU believe that such a survey would be useful, would be used by policy makers, and would improve the quality and efficiency of public expenditure on wellbeing. We suggest that the costs of the ELSCYPWB would be a small fraction of the overall expenditure in child wellbeing services, suggesting that very small increases in the cost effectiveness of such programmes and services would be necessary for the investment in the survey to be worthwhile.
The project dissemination pursued a three-stage approach: during the 1st phase the main concern was to achieve visibility, during the 2nd phase – to spread knowledge about the project’s aims and its initial steps. The 3rd phase of the project included evaluation and reviewing of initial activities and feeding the results into more mature offerings.
The PDF version of this report contains figures and graphics
A summary description of objectives of the WP9
The key objectives of the dissemination and communication activities in the MYWeB project were: to construct a project web-site dedicated to facilitating dissemination activities in the public sphere and providing a secure 'intranet' facility to project partners and collaborators; provide ongoing dissemination of scientific findings of the research project to policy, practitioner and community audiences at regional, national and European level; to bring the objectives and findings of the project to the attention of public audiences through EU information portals (CORDIS); to provide ongoing dissemination of research findings to the academic community through: the website and individual publications in high quality academic journals; to disseminate key messages on 'quick wins' in the measurement of childhood well-being across the EU; to produce an accessible summary of the findings of the project aimed at children and young people.
A description of the main results and foregrounds
To disseminate findings and recommendations of the MYWeB project regularly throughout the duration of the project (for a detailed description see Deliverable 9.2: Dissemination Strategy). The dissemination strategy was prepared under management of WP9 leader Daugavpils University in collaboration with the coordinating institution Manchester Metropolitan University.
Logos
Logos were used on documents, promotional materials and outputs where appropriate. The MYWeB logo established a ‘brand’ identity to the project, making it easily identifiable. A number of suggestions for the project logo were developed by WP9 leader Daugavpils University and consortium members selected the logo by vote.
The EU logos were important components for outputs. The grant number FP7-613368 was prominently placed on outputs. Institutional logos were used where appropriate. As the lead institution, Manchester Metropolitan University placed its logo on all appropriate documents. In addition, partner institutions placed their own logo alongside MYWeB, EU and MMU logos. Any dissemination activities in the project acknowledged the European Community’s Seventh Framework Programme funding.
Information sheets
The information sheets outlining the MYWeB project’s objectives and main activities were used as a first handout to inform the relevant target groups about the project. Because the WP4 and WP6 included direct involvement of the children and young people, information sheets for teenagers and children were developed by the WP4 and WP6 leads. The cooperation with teachers started with the distribution of the leaflets on the aims and objectives of the MYWeB project.
Templates
Where appropriate, templates, logos and palette information were developed and uploaded to the resources page of the website intranet for partners to use in documents and presentations. Consistency of style of presentation helped partners to produce dissemination outputs.
Project website
The central element in the MYWeB dissemination plan for the 1st period was the project website (for a detailed description see Deliverable 9.1: Project Website). The website http://fp7-myweb.eu/ went live on the 31st March 2014 and it is evolving over the course of the project. During the 1st period, it was one of the main communication channels of the MYWEB project, i.e. the website was the main resource for partners and stakeholders involved in MYWEB, as well as for the general audience, to gain information on the project progress, on the main outcomes (e.g. deliverables), and on key events organized in the scope of the project. It consisted of two parts:
(1) Open Information Resources which provided an informative, easy to read and navigate, comprehensive overview of the project and disseminate information, documentation on the project and findings to a broad audience including scholars, students and the general public.
(2) Intranet database designed for the project partners who entered a password protected intranet space from where they were able to access key project working documents and timetables, event details, reporting requirements. Included within the project area was the description of work, files and presentations from meetings, Deliverables and Outputs areas.
During the 2nd period, on June, 2016 the new version of the project website was launched to provide ongoing dissemination of research findings to four target groups: (1) teachers and researchers, (2) children and young people, (3) practitioners, (4) policy makers.
The section ‘Teachers and Researchers’ includes four sub-sections: (1) Delphi survey; (2) Testing surveys; (3) Our project proposal; (4) Learning from young people. Under section ‘Policy Makers’ the results of Delphi survey, Policy briefs and posters are available. As per the “Practitioners” section, it includes information on Delphi survey and Testing surveys. The fourth section ‘Children and Young People’ was designed to bring the findings of the MYWeB project to attention of children and young people.
Statistics of accesses to the MYWeB website:
Conference Papers
Presentation at academic conferences and dissemination of the project objectives and results have started early in the project and will continue to be an important part of the dissemination activities. Conferences were useful forum to consult with the academic audience in a face-to-face capacity and to address issues relevant to the project. Presenting at conferences provided the opportunity to publicise the MYWeB project and disseminate the first outcomes of the project to a wider audience (see Template A2: List of Dissemination Activities).
Publications
One academic paper for each participating country has prepared for publication. Some of them have been already published, the others are submitted to print/in print (see Template A1: List of Scientific (peer reviewed) publications).
Workshops
The MYWeB project members have contributed to the diary of dissemination activities by recording information about individual contributions to workshops where the objectives of MYWeB have been publicised and presented. While targeted at smaller groups of participants, workshops provided opportunity to involve a much higher and more active level of engagement. Workshops helped to familiarise members’ institutions consortium with the MYWeB project (see Template A2: List of Dissemination Activities).
Social Media
To ensure a presence of MYWeB project on social media platforms the page has been created on Twitter: https://twitter.com/projectMYWeB.
Promotional Materials
The promotional materials of different forms (bookmark, UBS stick, pen/pencil etc.) containing MYWeB project contact details and summarizing the project rationale served to raise awareness of the project and encourage those interested to contact researchers and use the website.
Posters
Posters have been prepared to make the project more visible for the audience of the Policy Forum and of those conferences where MYWEB presentations were made or just poster session planned. A collection of posters from the MYWeb project is available from MYWEB website, see: http://fp7-myweb.eu/policymakers.html
EU information portal
The MYWeB project brought the objectives and findings of the project to the attention of public audiences through EU information portal CORDIS.
The potential impact and the main dissemination activities and exploitation of results
The MYWeB consortium targeted a specific audience from the children and young people themselves, with a dedicated part of the project website, up to EU policy makers. The dissemination included two policy-briefs produced by the team in each participating country, editing of one collected volume and the final project conference.
Policy-briefs
Each partner prepared two policy-briefs tailored appropriately for regional and national agencies in the relevant consortium state. Addressed to policy makers focusing on children and young people, employees of the public administration at all levels, non-governmental organisations and specialists working with children and young people, the policy-briefs highlight national relevant findings and policy implications. They cover a range of themes in relation to: indicators of children’s and youth’s well-being (Croatia), understanding of Estonian children and youth about well-being (Estonia), the importance of a longitudinal study on well-being of children and young people in Georgia, multilevel governance in German child and youth policy, young people’s well-being in Greece, the concept of well-being in European child and youth research, family, the most important domain important to children and young people well-being (Latvia), economic well-being - a transversal priority for Portuguese society, the concept of well-being and its lack of understanding and provision in Slovakia, differences and inequalities in the understanding of well-being among children and young people (Spain), children’s understandings of well-being: the value of consulting young experts (UK), children and youth's participation and well-being (Croatia), results of project MYWeB: experts' opinions about the all-European study of child and youth well-being (Estonia), what young people in Georgia need for well-being and happiness, gaps in current German child and youth well-being policy, children's well-being in Greece, how do young people understand well-being (Hungary), the main obstacles to implement child and youth policy in Latvia, aiming to tackle young people’s unemployment problems in Portugal, the domains and major assumptions of children and young people’s well-being in Slovakia, involving children and young people in research (Spain), the benefits of Europe wide longitudinal data to the United Kingdom.
Edited volume
The main objective of task 9.3 was to communicate MYWeB outcomes in the academia. Overall, academic dissemination in MYWeB involved publishing of an edited volume with the input of all partners. The edited collection with a provisional title ‘Children’s and Young People’s Well-Being: How a pan-European longitudinal survey could improve policy’ will be published.
Conference
The final conference “Children and young people’s well-being and research in Europe” organized in Barcelona on the 21st July, 2016 were intended: (1) to discuss the state of the art about children and young people’s well-being and related policies and identify research gaps from a European perspective; (2) to explain the interest of achieving a better knowledge on policy impacts with regard to children and young people’s well-being and explain the MYWEB project; (3) to present the main results of the EC’s funded project: “MYWEB. Measuring Youth Well-Being”; (4) to discuss the project with the relevant actors in the field: children and young people, academics and policy makers; (5) to disseminate the project through the media for it to reach EU citizens and well-being related professionals. Participants included researchers in children and youth well-being, policy makers related to children and youth policies, MYWEB Advisory Groups and Advisory Board members, children and young people’s representatives and MYWEB members.
Use and dissemination of foreground
The MYWeB consortium intend to carry forward its dissemination activities to provide direct evidence of the feasibility of a European Longitudinal Study for Children and Young People. The key objectives of the further dissemination is to stimulate feedback from stakeholders on the project’s deliverables, to encourage discussion among stakeholders of the project’s findings and recommendations; to facilitate and enable collaboration between different categories of stakeholders; and to strengthen the research and knowledge base of both researchers in children and youth well-being and policy makers related to children and youth policies. It is expected that the continued research on the methodological options will impact developing of longitudinal survey.
List of Websites:
http://fp7-myweb.eu
