Skip to main content

Methodologies and Data mining techniques for the analysis of Big Data based on Longitudinal Population and Epidemiological Registers

Article Category

Article available in the folowing languages:

Advanced analytics help researchers understand huge social changes across Europe

New data mining techniques will help researchers comprehend shifting demographics alongside economic and cultural changes.

Society

Societies across Europe face a raft of varied social changes, including the changing nature of the family unit, fluctuations in fertility rates, a decline in overall mortality due to increases in longevity and fluctuations in economic and social stability. Migration and ageing societies bring further opportunities and challenges. For the past decade or so, European research teams have been steadily creating vast databases and population registers to track and eventually analyse these monumental shifts. “The development of these long-term longitudinal databases and life-course and intergenerational analytical methods has made this one of the most dynamic areas of research right now,” says Diego Ramiro Fariñas, director of the Institute of Economics, Geography and Demography in Spain. The EU-funded LONGPOP, a Marie Skłodowska-Curie Innovative Training Network project, was dedicated to understanding this vast amount of data. The main goal was to analyse and understand the causes and consequences of the rapidly shifting social landscape across Europe. The project also set out to build a training platform to accelerate the number of researchers using these huge databases. A Europe-wide network was created and strengthened through the LONGPOP project, to allow researchers to share experiences, knowledge and work together on joint research projects to achieve these goals.

Creating new methods of data analysis

To be able to analyse such huge longitudinal registers and administrative linked data – which statistical agencies are increasingly producing – advanced data mining, analyses and statistical techniques were required. “It was evident that there was a lack of trained scholars and technicians who could deal with such kinds of data,” Ramiro adds. LONGPOP introduced new methodologies and data mining techniques, as they went through multiple data sources: from the digitisation and geocoding of historical maps and population registers, to modern geographic, sociodemographic and epidemiological information or social media data. “Identifying patterns of infant mortality in historical data and looking for current spatial clusters of emerging diseases within big cities are examples of applications of such techniques,” Ramiro notes.

Mental health links to disadvantage

LONGPOP analysed the intergenerational transmission of socio-economic conditions, the effect of the 1918 Spanish Influenza in Madrid, and links between population health levels and environmental factors such as pollution and heatwaves, amongst other topics. “We emphasise the growing importance of mental health in ageing European societies and focussed particularly on the link between neighbourhood disadvantage and mental disorders,” adds Ramiro. Initial results confirmed that persistent inequalities affect the health and survival of the poorest and least educated. The findings indicated links between neighbourhood crime and mental disorders, and that this varies among different European countries. Early childhood stressors determine how people react to detrimental conditions in later life. “All these results bring new and relevant insights into the complex relationship of different sociodemographic and epidemiological variables, and have important implications for policy and prevention,” Ramiro explains.

A boost to the young researchers involved

The comprehensive training network and the highly productive atmosphere of the network has benefited all fellows’ careers as researchers immensely. Through the collaboration with other affiliates of the project, most of them were able to finish their PhD research (or will do so soon), attend many renowned conferences in the field, and publish in high-impact journals.

Keywords

LONGPOP, society, shifts, demographic, crime, neighbourhood, data, analysis, mining

Discover other articles in the same domain of application