Community Research and Development Information Service - CORDIS

ERC

CREATIV Report Summary

Project ID: 321135
Funded under: FP7-IDEAS-ERC
Country: France

Mid-Term Report Summary - CREATIV (Creating Co-Adaptive Human-Computer Partnerships)

In the physical world, people can predict how objects will react, and develop skills based on that understanding. They can flexibly adapt objects to achieve different goals, taking advantage of their physical properties in creative ways. However, when they use today's interactive technology, they are faced with cumbersome interfaces, with few underlying principles that can be learned and reused in different situations. Transitioning from one software tool to another requires relearning myriad small details to accomplish the same task, but in an arbitrarily different way. Users do not own their software formats, so new software 'upgrades' can make old files inaccessible or force users to change their daily work practices.

The goal of the CREATIV project is to fundamentally re-envision the way people interact with digital information. Rather than trying to create ever more intelligent computers, our goal is to empower people. We want to create effective human-computer partnerships where people remain in control of the machine. To achieve this, we need to create a 'physics of interaction' that treats interaction as a first-class object, separate from data and functionality.

We are developing a set of key theoretical concepts based on 'instrumental interaction', which is how human beings use tools to accomplish tasks in both the physical and digital world; and 'co-adaptation', in which users both adapt their behavior to the system’s constraints, learning its power and idiosyncrasies, and appropriate the system for their own needs. Our work is based on indepth studies of creative professionals, extreme users who push the limits of technology. We observe, interview, and run participatory design sessions with music composers, artists, graphic and industrial designers, as well as scientists and engineers. We focus on the creative phase of their work, as they struggle to represent new ideas, and on the implementation phase, as they transform their ideas into a form that can be interpreted by a computer. We also work with these users to create novel forms of interactive technologies, often working at the boundary between physical and digital objects, such as interactive paper, or across surfaces, from tiny watches and smartphones to wall-sized displays.

The empirical studies and the technology explorations contribute to the theoretical principles we are exploring. We recently identified a key concept, called a 'substrate'. We noticed that creative professionals are expert at developing personal representations of their ideas, which balance constraints and flexibility. During the creative phase, they move smoothly back and forth between generating ideas and testing them under different constraints. However, when they shift to working on the computer, much of their creative expression is lost: they must now think in the software's terms and translate their ideas into a form that the computer can understand. We designed interactive paper technology in which the composer or artist identifies their own set of constraints and then explores specific ideas within those constraints. For example, Paper Tonnetz uses a hexagonal layout, like a honeycomb, to organize different sets of musical pitches. A C-major tonnetz will result in pleasing melodies wherever the pen goes, whereas a B-minor tonnetz may generate more eery or disturbing sounds. The key is that the musician can define both the structure of the muscial piece and then work through the individual musical phrases, in a form that resonates with their musical ideas and can also be read by computer-based composition software.

We created a fundamentally different type of substrate, called a WebStrate, that lets users choose their own multi-media editors as they work simultaneously on a single document. One author may prefer a graphical user interface such as Microsoft Word, whereas another may prefer a code-based interface, such as LaTex. With Webstrates, authors can choose their own editor, with their own tools, based on their own preferences and skillsets. If one person creates a specialized tool, such as to automatically number figures, she can include it with the document, so the author can use it as well. Webstrate capture both the data and the relationships among the data, each of which can manipulated separately with different kinds of tools.

We are also exploring new ways to support co-adaptation, to allow users to quickly and easily define their own gestures to issue personalized commands, and to take advantage of machine learning algorithms to allow users to draw gestures that are both accurate and expressive. Our ultimate goal is to radically transform interactive systems for everyone by creating a powerful and flexible partnership between human users and interactive technology.

Contact

Mireille Moulin, (Head of Administration and finance)
Tel.: +33 1 7292 5964
Fax: +33 1 7292 5936
E-mail
Record Number: 183310 / Last updated on: 2016-05-27
Information source: SESAM