The science of people watching … automatically
Automatically interpreting social interactions is a complex area of research. Get too close to the action and a researcher's presence can inhibit the subjects, interfere with the natural flow of the exchange, or completely discourage people from interacting at all. Observing from a distance is much less interfering but can make it difficult to follow interactions and exchanges, monitor nuances and even identify the various role players. Furthermore, in general, monitoring complex interactions within large groups is challenging. With support from the People sub-programme of the EU's Seventh Framework Programme (FP7), the 'Analysing social interactions at a distance' (ANASID) project aimed to do just what its name suggests. The project experimented with various technologies and devised methods to allow machines to automatically analyse social behaviour in crowded social gatherings. ANASID focused on developing automated techniques for identifying social groups and speakers within the group, as well as estimating the social aspects of group-based behaviour. From the original video data for the project — which, for privacy reasons, were processed to remove visible faces — 80 clips of 10 seconds each were annotated for conversing groups, head pose, body orientation and position. To collect data on people conversing in groups, ANASID organised a speed-dating event on campus. The team employed a number of means to record the interactions: two overhead cameras to record two speed dates at a time and android phones installed with a special recording application. By automatically measuring how people moved in the videos, ANASID was able to detect whether people were attracted to each other. Unfortunately, most of the audio equipment failed to record. To overcome these issues, the project acquired wireless microphone recording systems. These, along with wearable sensors and other audiovisual equipment were deployed for a much larger-scale experiment. ANASID also conducted another, more controlled lab experiment, which involved an encounter between 32 previously unacquainted people. From this data, ANASID was able to detect when someone was speaking using just a single accelerometer hung around the neck. ANASID managed to develop effective techniques for automatically detecting conversing groups, with an impressive accuracy of 87–92 %. The project also studied methods for detecting the speakers within groups talking between themselves. The findings of the ANASID project should help advance the study of social interactions by enabling non-intrusive monitoring of group dynamics.
