In the project description it was suggested that the latest state-of-the-art methods, particularly probabilistic latent semantic analysis and long short term memory networks (LSTM), was expected to give the best result. From the time the project description was written, until the start of the project in February 2019, there were significant changes in the field of natural language processing. Most notably, Jacob Devlin at Google did in November 2018 publish an article called “Bidirectional Encoder Representations from Transformers”. The researcher was one of the first that where able to use these improved methods.
The main scientific achievement in the project has been to be able to use transformer-based language models within the field of vaccine confidence. After collecting and coding the data (WP1), a model was trained to be able to analyse vaccine stance, sentiments and categories (WP2). In the end the data were analysed (WP3). The method has improved the accuracy of sentiment and stance analysis, and made it possible also to identify more complex and fine-grained sentiments regarding vaccine stance.
As the article “Categorizing vaccine confidence with a transformer-based machine learning model: analysis of nuances of vaccine sentiment in Twitter discourse.” (
https://medinform.jmir.org/2021/10/e29584/(odnośnik otworzy się w nowym oknie)) clearly shows, the project has been able to apply machine learning methods to gain an accuracy in vaccine sentiment and stance analysis that is beyond what was previously possible. The project has also been able to participate in releasing the first transformer model pre-trained on Covid-19 social media. This is described in the article “Covid-twitter-bert: A natural language processing model to analyse covid-19 content on twitter.”. The CT-BERT-model that was trained has contributed to several other research projects. The provided list under 1.1 includes many examples of how it has contributed
out interesting areas in implementing advanced machine learning services in commercial products, and it also provided useful knowledge for the tasks T4.1-2.
In the latter stages of the project, the focus was on applying and testing vaccine sentiment analysis. A lot of experiments were also carried out with regard to comparing large zero-shot models with encoder-based models. The most important work here was the publication of the article “Understanding the vaccine stance of Italian tweets and addressing language changes through the COVID-19 pandemic: Development and validation of a machine learning model” in Frontiers of Public Health. The article is a baseline for further research.
