Unified Principles of Interaction

Project Information

ONE

Grant agreement ID: 695464

DOI

10.3030/695464

Project closed

EC signature date 12 July 2016

Start date 1 October 2016

End date 31 March 2023

Funded under

• EXCELLENT SCIENCE - European Research Council (ERC)

Total cost

€ 2 456 028,00

EU contribution

€ 2 456 028,00

Coordinated by

UNIVERSITE PARIS-SACLAY
France

Periodic Reporting for period 4 - ONE (Unified Principles of Interaction)

Reporting period: 2021-04-01 to 2023-03-31

Summary of the context and overall objectives of the project

Most of today’s computer interfaces are based on principles and conceptual models created in the late seventies. They are designed for a single user interacting with a closed application on a single device with a predefined set of tools to manipulate a single type of content. But one is not enough! We need flexible and extensible environments where multiple users can truly share content and manipulate it simultaneously, where applications can be distributed across multiple devices, where content and tools can migrate from one device to the next, and where users can freely choose, combine and even create tools to make their own digital workbench.

The goal of ONE was to fundamentally re-think the basic principles and conceptual model of interactive systems to empower users by letting them appropriate their digital environment. The project addressed this challenge through three interleaved strands: empirical studies to better understand interaction in both the physical and digital worlds, theoretical work to create a conceptual model of interaction and interactive systems, and prototype development to apply and test these principles and concepts in the lab and in the field. The conceptual model combines substrates to manage digital information at various levels of abstraction and representation, instruments to manipulate substrates, and environments to organize substrates and instruments into digital workspaces.

ONE has demonstrated the validity and fruitfulness of this approach through a number of case studies, and by developing a generative approach to the design of systems based on these concepts. Part of the work has focused on document management and editing and led to an ERC Proof-of-Concept (PoC) project called OnePub that will apply these principles to the development of a document editing and production platform. ONE has also inspired the PI to propose two major projects that were funded by the France 2030 national program: the CONTINUUM network of 30 interactive rooms, for which he is the scientific director, and the eNSEMBLE program on the Future of Collaboration that gathers over 100 research teams across computer science and social sciences, for which he is co-director.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

On the empirical side, we have published a series of user studies: use of digital tools by graphic designers, collaboration between designers and developers of interactive software, use of computational notebooks by scientists, use of document processing systems by legal professionals, use of desktop tools by sight-impaired users, use of LaTeX by users inside and outside academia.These studies have led to a better understanding of these users' needs and grounded the design of novel digital tools. This work was reported in several articles, including in flagship conferences (ACM CHI, ACM CSCW) and journals (ACM TOCHI) and received several awards.

On the theoretical strand, we have published, in collaboration with researchers from Telecom, work on a deeper understanding of interaction based on the concepts of information theory. This work has received a Best Paper and and Honorable mention awards at the flagship ACM CHI conference, as well as a Best Thesis Work award from Université Paris-Saclay and a Best Thesis award from Telecom Paris. We also revisited the relevance of a major law from psychology (Hick’s law) to Human-Computer Interaction, and published novel models for analyzing and predicting pointing performance. We have also advanced our deeper understanding of human tool use and how it applies to digital environments, and the development of the conceptual model of instruments and substrates, based on the concept of interaction as a first-class object. These concepts underlie most of the work conducted in the Engineering strand. Finally we have published a foundational paper with two other ERC grantees on the concept of Generative Theory of Interaction, to appear in ACM Transactions on CHI.

On the engineering strand, we have created various prototypes, including Enact for supporting designer-developer collaboration, Montage for video prototyping, Touchstone2 for experimental design (ACM CHI Best paper award), Tangler for a novel model of malleable software, VideoStrates for video production and editing, CamRay for remote collaboration across wall-sized displays, Textlets for text processing, FileWeaver for file management, iLaTeX for LaTeX editing, or Argus for a priori power analysis of experiments. These functional prototypes include novel interaction techniques based on the principles developed in the project and have been successfully evaluated them with users. This work was reported in numerous papers and presented in demo sessions at various conferences.

The PI has presented the project in keynote talks in France (DYSTOPIA conference, CNRS Humain & Numérique en Interaction, DATAIA-JST) and abroad (12th Biannual Conference of the Italian SIGCHI Chapter, ACM UIST Vision talk), and in a number of invited seminars (Wellesley College, Univ. British Columbia, Univ. Aarhus, Univ. California San Diego, ETH Zurich, Univ. Zurich, Stanford University, MIT CSAIL). The PI has also co-organized a workshop on “Rethinking Interaction” at the flagship CHI 2018 conference.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The results of the project go beyond the state of the art in several areas:

- We better understand the needs of expert users in a variety of areas through our empirical studies. These studies highlight the need for expert users to tailor software to their needs and for software to be more easily interoperable ;
- We demonstrated the value of using concepts from information theory to create interfaces that take better advantage of human and computer capabilities. We demonstrated that challenging users during interaction in order to get them to communicate information more effectively, rather than “helping” them with the most probable responde, leads to substantial increases in performance (up to 40%) ;
- We created a series of functional prototypes featuring novel interaction methods, demonstrating how these ideas apply to practical problems. These prototypes target different application areas, most notably for content creation. We introduced novel techniques for document editing and management that led to an accepted ERC Proof-of-Concept project ; and
- We developed the concepts of instrumental interaction and substrates and introduced the concept of Generative Theories of Interaction. A Generative Theory of Interaction identifies concepts and generative principles that bridge the gap between existing theories of human behavior and the design process of interactive systems.

This work has been recognized by publications in the best conferences and journals in the field, as well as by a series of awards (6 Best papers and Honorable mentions, 1 Best thesis).

ONE demonstrated how to design a new breed of interactive environments that integrate various forms of interaction and are more attuned to human skills and capabilities. We created prototypes demonstrating flexible and extensible environments that challenge the assumptions of current closed systems. We introduced a Generative Theory of instruments and substrates, featuring a small number of unifying principles within a single conceptual model to create a 'physics of digital information' that users can understand and appropriate.

Organization of the project