Periodic Reporting for period 1 - MINDPICS (When a Profile is worth more than a Thousand of Hashtags: Automatic Inference of Personality Traits based on Images Shared in Social Networks)
Reporting period: 2016-07-01 to 2016-11-30
The social media, as a major platform for communication and information exchange, provides a rich repository of the opinions and sentiments of 2.3 billion users about a vast spectrum of topics. Such knowledge is playing an important role to understand and predict human decision making, while becoming essential for digital marketing, brand monitoring, and customer understanding, among others. Although social marketing budget is doubling each year, reaching 9 billion dollars in 2015 in US alone, the analysis of trends, topics and brands in social networks is based solely on textual posts. Despite the fact that 65% of users are visual learners, the knowledge embedded in the 1.8 billion photos uploaded daily in public profiles has been typically ignored. Based on this gap in coverage, during the MINDPICS project, a platform has been developed which applies the most modern machine learning techniques, based on Deep Learning, to understand near 100K images publicly shared per day, for the inference of relevant insights from social public profiles. In essence, the final visual-based personality trait model has been built using current state-of-the-art on image understanding, in the form of a working, validated prototype. Thanks to this project, a proper combination of different sources of knowledge (images, texts, geographical and temporal stamps) has enabled to roughly understand the whys of certain social user's demands and cultural-driven interests.
The key aspect of the contribution of MINDPICS has been the validation that personality trait estimations using the OCEAN model is a valuable source of knowledge for understanding the context of social profiles, based on the posted images. Interestingly, image-based experimental results actually correlate with previous cyber-psychology results based on texts. Moreover, for the Neuroticism trait it has been proved that results are even more accurate than using texts only. These results have opened a new avenue of research and development for further refining the personality model on this particular trait based on additional, mid-term experiments designed and validated with psychology experts.
Thanks to such a technological advanced, Visual Tagging has been able to better discover the hidden customers of a given brand, finding its logo on the pictures shared in public profiles. Centred on the Fortune 500 Fast-moving consumer good companies, a new procedure for acquiring clients has been implemented based on the commercial relationships between brand managers and digital marketing agencies. Also, after several interviews with key persons of marketing departments in multiple companies, Visual Tagging has better defined the KVP so that brands can now easily understand the offered service: access to product insights (How is the product being used, in what context, and with which other product), to competitive intelligence (How does the visual brand presence vs its main competitors, and whether the brand is leading vs following), to consumer insights and trends (Who and how are the fans/consumers/followers, what do they care about, and how can a brand make use of social media to laser-target the competitor's audience), and to customer engagement and acquisition (How can a company maintain and increase its number of fans, and How are the ones most active).
The key aspect of the contribution of MINDPICS has been the validation that personality trait estimations using the OCEAN model is a valuable source of knowledge for understanding the context of social profiles, based on the posted images. Interestingly, image-based experimental results actually correlate with previous cyber-psychology results based on texts. Moreover, for the Neuroticism trait it has been proved that results are even more accurate than using texts only. These results have opened a new avenue of research and development for further refining the personality model on this particular trait based on additional, mid-term experiments designed and validated with psychology experts.
Thanks to such a technological advanced, Visual Tagging has been able to better discover the hidden customers of a given brand, finding its logo on the pictures shared in public profiles. Centred on the Fortune 500 Fast-moving consumer good companies, a new procedure for acquiring clients has been implemented based on the commercial relationships between brand managers and digital marketing agencies. Also, after several interviews with key persons of marketing departments in multiple companies, Visual Tagging has better defined the KVP so that brands can now easily understand the offered service: access to product insights (How is the product being used, in what context, and with which other product), to competitive intelligence (How does the visual brand presence vs its main competitors, and whether the brand is leading vs following), to consumer insights and trends (Who and how are the fans/consumers/followers, what do they care about, and how can a brand make use of social media to laser-target the competitor's audience), and to customer engagement and acquisition (How can a company maintain and increase its number of fans, and How are the ones most active).
-Creation of a methodology and evaluate the performance for the social network users insight extraction process of personality traits.
-Implementation a know-how protection plan and Intellectual Property (IP) management of the key technological components of the social insight discovery product.
-Identification and analysis of new customers’ requirements and needs, by improving the market study, including a world-wide competitor analysis to take into account the competitors' likely strengths, weaknesses, future actions and current strategies.
-Implementation of a commercialization plan, including national and international strategies.
-Implementation a know-how protection plan and Intellectual Property (IP) management of the key technological components of the social insight discovery product.
-Identification and analysis of new customers’ requirements and needs, by improving the market study, including a world-wide competitor analysis to take into account the competitors' likely strengths, weaknesses, future actions and current strategies.
-Implementation of a commercialization plan, including national and international strategies.
Based on the interviews with marketing professionals and their feedback when using the platform described in the Figure below, Visual Tagging Services has improved the KVP during the MINDPICS project by generating not only the insights from users, but also quantitative analytics and ad targeting. The questions we learnt from these interviews can be summarized as:
- Product insights: How is the product being used? In what context? With what other product?
- Competitive Intelligence: How does the visual brand presence vs the competitors? Where the brand is leading vs following?
- Consumer Insights & Trends: Who & How are the fans? What do they care about? How can the brand use social media to laser-target the competitor's audience?
- Customer engagement & acquisition: How can maintain & increase the set of fans? How are the most active?
So the services Visual Tagging offers now definitively go beyond logo detection. We have complemented our analysis with other valuable visual information, like for example using our scene description technology for inferring the environment and context within which a brand product is being consumed, and using our facial expression analysis algorithms for identifying which are the consumers’ emotions when appearing with the brand.
With all these requirements defined and the feedback from marketing companies in mind, we next established the degree of competition in the market for automatic video/image tagging, is low to moderate, although it is continuously increasing. Indeed , as commented in the MINDPICS proposal, there are several companies working in social intelligence, which analyze the information coming from the social media (Social Bro, Brandwatch, …) but they are working mainly with hashtags. Today’s text based analysis platforms still do not use photos and consequently miss the visual signal.
- Product insights: How is the product being used? In what context? With what other product?
- Competitive Intelligence: How does the visual brand presence vs the competitors? Where the brand is leading vs following?
- Consumer Insights & Trends: Who & How are the fans? What do they care about? How can the brand use social media to laser-target the competitor's audience?
- Customer engagement & acquisition: How can maintain & increase the set of fans? How are the most active?
So the services Visual Tagging offers now definitively go beyond logo detection. We have complemented our analysis with other valuable visual information, like for example using our scene description technology for inferring the environment and context within which a brand product is being consumed, and using our facial expression analysis algorithms for identifying which are the consumers’ emotions when appearing with the brand.
With all these requirements defined and the feedback from marketing companies in mind, we next established the degree of competition in the market for automatic video/image tagging, is low to moderate, although it is continuously increasing. Indeed , as commented in the MINDPICS proposal, there are several companies working in social intelligence, which analyze the information coming from the social media (Social Bro, Brandwatch, …) but they are working mainly with hashtags. Today’s text based analysis platforms still do not use photos and consequently miss the visual signal.