Abstract

In everyday life we engage in social interactions and use our body to communicate social actions and intentions in a seamless way. Our body, our behaviour and the behaviour of the people around us tell a lot about the social situations in which we are involved and of the social context in which we act. For example, the way people direct their attention while engaged in a conversation or the spatial distribution of a group during an activity can reveal information about the social context that characterises that particular social situation. Inspired by these observations, several researchers have investigated ways to automatically interpret social signals, using sensors and algorithms for detecting and modelling behavioural cues related to social interactions. With the increasing availability of sensors and technology in our environment, we believe that this research direction is highly relevant nowadays and presents an opportunity for further investigation of social signals in the HCI domain. In this dissertation, I discuss the design and evaluation of socially-aware systems, technologies that are sensitive to the social context in which they are deployed. The focus of this work is to explore interfaces that support engagement in co-located multi-user interactions, taking into account users’ nonverbal behaviour, including gaze, facial expressions, and body movements. In particular, the research goals are twofold: to understand which nonverbal cues and social signals reflect engagement in co-located group activities and to design systems that can utilise this information. The thesis presents an in-depth analysis of the state of art in designing socially-aware systems. Moreover, it presents empirical studies that describe the design, implementation and assessment (both in real-world and laboratory settings) of such systems. This research has involved the development and evaluation of prototypes in their real context of use. Specifically, the thesis focuses on socially-aware systems in the form of two multi-user technologies meant to be used in different social contexts: a responsive display deployed in a public setting, and an ambient display for supporting conversations in small group discussions. Finally, based on the findings and insights gained through these studies, the thesis provides generalised characteristics for socially-aware systems and presents MOSAIC, a theoretical framework for framing the complexity of the design space of such technology. The final aim of this work is to support HCI researcher and practitioners in exploring the opportunities and limitations of technologies that react and respond to the social context.