Skip to main content
European Commission logo print header

Interconnected Embedded Technology for Smart Artefacts with Collective Awareness

Deliverables

ToyLabs ITR AB grew out of a project initially called ’SpyBalls’ (now developed as a product called ‘Spookies’) developed in a project course taught by the PLAY studio of the Interactive Institute in the HCI/Interaction Design course at the IT University of Goteborg in Sweden. Smart-Its research and technology were fundamental to the course, both as a conceptual and prototyping platform and were used to deeply develop principles and early prototypes for user testing of the SpyBalls application. The project was continued in 3 (1 is pending) master’s theses supervised by PLAY, in the filing of a Swedish patent (application number 0302073-2) in July 2003, and in the formation of the ToyLabs company start-up established in June 2003, who’s shareholders are: Tobias Rydenhag, Sara Backlund, Jesper Bernson, Lena Berlin, Anders Lidstrom, and Jonas Stromberg. SPOOKIES: Spookies are computational free-play plush toys with embedded electronics for use in active and creative play situations. Spookies come in seven different pairs (fourteen modules in total) that collectively make up a communication platform. Computational properties of various modules include: sound, image, and message transmission; motion, light, and distance detection; and a timer function. Pairs of modules can be used by children to, for instance, discover how far away a friend is, capture pictures, detect if someone is moving in the next room, or send a secret message. Configurations of multiple modules generate more advanced functions – for example, a light-sensitive module plus a melody player plus a timer module to trigger a song at sunrise. Spookies' unique flexibility relies on a modular architecture uniting two distributed wireless networks. One is based on radio communication for information transmission up to a distance of 200 meters. The other is based on Infra-Red (IR) communication for information transmission when the modules are physically connected. The two networks enable parallel communication where computation is distributed over all nodes. From the perspective of user experience, this is essential for enabling the intelligence of the system through physical and tangible interaction. A child can easily change the logic and information flow of the system by assembling Spookies in different ways, providing simple, physical end-user programming to children. Iterative prototyping and user participation have been essential to the development process – prototypes throughout have been evaluated in participatory sessions, informing both the research and product development in the company. SPIN-OFF: ToyLabs ITR AB is a spin-off from the PLAY studio of the Interactive Institute in relation to PLAY’s operations within the educational field as well as a PLAY studio research collaborator, particularly in relation to the Smart-Its research project.
This framework describes the principle structure and behaviour of smart artefacts that observe and collectively maintain a model of their environment. Individual artefacts in this framework have pre-configured domain knowledge and use embedded sensing and perception to infer dynamic knowledge concerning their state and environment. Dynamically associated artefacts, such as artefacts within proximity of each other, use collaborative inferencing to reason across partial knowledge held in participating artefacts. Collections of artefacts effectively constitute a distributed knowledge base. The framework has resulted from investigation of diverse application settings for smart artefact technology. The framework itself is independent of any target implementation platform, but has been tested, applied and demonstrated on the basis of Smart-Its devices. The framework has been validated in the context of two different types of application: - Artefact-centric observation of human activity, and collaborative reasoning across artefacts to support activity recognition. - Environmental monitoring embedded in artefacts, and collaborative reasoning to proactively identify critical situations.
The project has developed a context architecture for collective context information management of low power wireless sensor modules. This architecture is called smart-it. In our smart-it context architecture previously published SW architectures are merged together in a novel way to manage context information. The smart-it context architecture is the first context managing architecture for collective of low power sensor modules. In the context architecture the context information is handled from artefact perspective. The developed context architecture offers formal structures to develop context management SW for the application of ubiquitous computing with smart-it technology. It specifies SW architecture structure, API, and information interchange format. It supports also collective context recognition functionality. We think that our context architecture can make application developer’s implementation work easier in several ways. The advances of smart-it context architecture for SW developers: - Formal architecture structure for context management SW. - Specified API to access context information: access to context resources of smart-its network. - Formal context description for interchange. - Support for implementation of collective features. - Reuse and maintenance of code is easier. - Better readability of different developers’ code. The context architecture enables the formal way to create application specific recognition modules for use of applications. In several application experiments we found that artefact centric context information management is suitable for context aware application of ubiquitous computing. In general sense, the collectiveness of smart-its enables resource sharing (sensing, computation, memory and computation) to increase performance or to save energy. The single smart-its can also share some context with other smart-its. The collectiveness of artefacts enables new kind of application creation where people can work in ad hoc collectivities. By using collective context recognition we can achieve several benefits compared to centralized context recognition. Benefits are related to reliability, resource saving and recognition of more complex contexts. More reliable context recognition is one benefit, which can be implemented by using collective features. In that case, several smart-its recognize the same context equally in the same area and results can thus be summarized. Common measurement fact is that several measurements are more reliable than single measurement this is also valid in the case of context recognition. The multiple context recognition increases fault tolerance because one faulty smart-its does not prevent recognition. The work on context architecture has been published in various conference articles in international scientific conferences. The smart-its context architecture has also been presented in national seminars and conferences. In concept level, it is possible to apply the ideas of smart-it context architecture in current mobile terminals and embedded computers in the co-operation of device manufacturer. Negotiations are under way for implementing the context architecture in certain products on mobile phone platform.
Smart-Its P are using a lightweight, ultra-low power communication stack that is designed for fitting exactly the needs of a such environments as - Decentralised and cellular architecture and media access; - Ad-hoc behaviour and spontaneous book-in into a network; - Real time communication and synchronization; - Predictable processor load due to the network stack; - Low power consumption and small package of hardware; - Easy (no) administration and maintenance; - Support for spontaneous collaboration between peers. Data transmission is based on a stateless peer-to-peer protocol named AwareCon. It implements physical access to the transceiver including the coding on the medium and the data link layer (DLL). On the physical layer, a time division multiplex collision avoidance protocol is used, based on fixed slots and strict synchronization of all Smart-Its within communication range. Thereby a wake-up on traffic procedure is avoided, as it is problematic on a network channel. The access to network is regulated cooperatively by a very short arbitration. The network is very scalable as the arbitration allows several hundreds of nodes to co-exist within one network cell with only slightly degrading of overall throughput. The AwareCon introduces the novel concept of context awareness to networks and is therefore able to react better on changes in the setting and environment. Reduction of power consumption and better network performance are results of this novel method. The higher layer of the stack - named ConCom - allow for a efficient and common communication between Smart-Its devices but also between Smart-Its devices and Internet enabled devices. ConCom expresses and communicates information, especially context, in a way that is similar to a natural spoken language. ConCom works connectionless and without addressing and forms an efficient way to transport information between mobile devices with limited resources. It is especially applicative for highly mobile, dynamic and heterogeneous settings. The layer defines service expressions to express and access services between Smart-Its devices in peer-to-peer settings but is also used for providing development services. The main strength of this layer is the ability to provide an ad-hoc, location and context aware communication platform for spontaneous collaboration between Smart-Its devices. The ability for spontaneous collaboration in turn can be seen as the enabler for emerging functionality in creation of application setting build through the project. Here newer application build on top of available capabilities of existing devices made available to Smart-its peers through the ConCom communication language.
TecO's Smart-Its P is a tiny hardware/software platform for rapid prototyping of networked Ubiquitous and Pervasive Computing environments, for Ad-Hoc (Sensor) Networks, Wearable Computers, Home Automation and Ambient Intelligence Environments. The platform consists of ready-to-run hardware components, software applications and libraries for the hardware and a set of development tools for rapid prototyping. The platform was especially designed for long-term, unsupervised operation of such devices, ease of development and embedding in tiny spaces. Smart-Its P addresses basic problems that arise when building systems with computers integrated into (everyday) objects and environments. This particular platform was designed following study of users of such systems. The target developer skill-set therefore varies, in that the devices can be used and configured by non-experts with very limited programming expertise, as well as experts of embedded systems, who possess the knowledge of extending the hardware and/or software of these systems. Developers not familiar with embedded systems, hardware, low-level or network programming are still facilitated with a set of utilities and tools for creating Ubicomp artefacts, embedded wireless sensor nodes and other related technology components. These tools include the following features: - Ready-to-run set of hardware components coming with embedded software that can be used without any knowledge of hardware or low-level software. - Modular hardware design that still enables the developers to change hardware. - Integration hardware that allows users and developers to access embedded devices from the Internet. - Additional hardware modules that can be used for further custom hardware development. - Software that allows writing Smart-Its P programs for the embedded devices or for PC based services in various languages (C, Java, Delphi). - Tools to control supervise and configure the Particle hardware. - Tools to support development including tools to program and configure Smart-Its-P remotely over the Internet. - Services for simplest access to Smart-Its P output as sensor and context information through querying a database or using the data in programs as Excel or Matlab. Various versions of the Smart-Its P hardware are designed throughout the project. All of them share characteristics such as ultra low power consumption (<30mW max at full operation, some µW to 1mW in average), mechanical robustness and high number of I/O lines. They differ in size (from 15x25mm to 45x27mm) and available resources (from 1.5k Data and 32k Program to 129k Data and 128k Program RAM) but share the same operating system, development environment, programming languages etc. For separation of concerns, Smart-Its P hardware is split into two modules: The communication part and the sensor/actuator part. Both parts share a standard interface connector that enables developers to extend the system with own sensors, actuators, resources (e.g. additional memory) or more computing power. On the software level the operating system shields the developer from system details. The communication board can be used either by application developers without knowledge of embedded system technology by using the operating system’s high-level API or by experts. Experts have access to system functionality - especially the communication - down to the RF physical layer. For example, field strength control or transceiver modes that can be adjusted by software while running the protocol enable the testing of new physical layer or Link Layer protocols without the need for assembling hardware. Various sensor boards provide different kind of sensors and actuators including acceleration, audio, light, temperature, force/pressure etc. Additional boards provide standard interfaces for additional sensors and serial lines for attaching the system to other type of computers. Development support hardware includes power supply hardware and test-boards for simple functionality test. The software consists of a small but powerful OS that provides a very simple to use driver concept and APIs for accessing sensor, actuator and network functionality. Developers can either program application programs and download these wirelessly to the devices or write services, that run on dedicated times on the embedded system. Standard services of the operating system are: - Time Server: synchronizes time within a network with a maximum deviation of 4µs. - Download Service: for installing new services and programs wirelessly on the device and several services to compute context information from sensor values. Several standard applications are provided as well, providing a base functionality platform that can be reused in various projects. The platform is now in use by several developers all over the world, mainly in Europe but also including the U.S. and Japan, and is integrated in upcoming research and development project.
'Lancaster Smart-Its' are a hardware/software toolbox for rapid assembly of embedded devices and smart artefacts. The toolbox is an extensible device building device that enables prototyping with different configurations of wireless network, sensors, actuators, and software for customisation to particular physical design contexts. The primary design rationale for DIY Smart-Its is to achieve short cycles between design idea and lab prototype, and ease of reproduction by outside developers. This was motivated by the dilemma observed in related R&D, i.e. that applications of smart artefacts cannot be developed without suitable hardware, while the design of suitable hardware depends upon an understanding of how the hardware fits into the application environment. However, rapid prototyping of smart devices clearly has uses beyond R&D, e.g. in design processes and education. To achieve the design goals underlying DIY Smart-Its, a purposefully simple implementation of the device architecture: a core board provides processing and communication, and add-on boards integrate physical interaction components. The core board includes a PIC microcontroller, 8KByte FRAM, and a transceiver module for wireless communication operating on 433.92MHz with raw data rates of up to 160KBits/s in half duplex mode. The core boards are available in a compact form factor for embedded operation and in an extended version that provides a serial interface and larger battery pack. The add-on boards are connected over a standard interface to a core board to provide physical interaction components that are controlled by the core MPU. In the simplest case an add-on module can be a single sensor but more commonly they integrate a group of sensors and/or actuators. A variety of add-on boards are available as part of a growing toolkit but additional ones can be build by adapting a generic board design or by designing dedicated boards. DIY Smart-ITs are programmed in C. Application developers are provided with libraries and tools that provide programming abstractions at multiple levels. At the lowest level, drivers shield the developer from hardware detail, both on component level (sensors, actuators, memory, power supply, processor) and on board level (physical connections between components). At application level, application interfaces and code libraries are provided for the set of tasks a Smart-Its application typically involves. These are acquisition of sensor data, processing of sensor data to application-specific context, communication of context events to other devices, and reaction to context events. DIY Smart-Its have been developed and evaluated in conjunction with so-called hackfests: workshops with external developers in which participants had the opportunity to develop simple context-aware embedded devices. More than 40 participants attended the hackfests from more than 10 different institutions active in ubiquitous computing and interactive systems research. None of the participants had had prior practical experience in hardware design, yet all succeeded in building their first functional device within one day; and more complex systems composed of multiple Smart-Its were constructed in two to three days. The particular advantages of DIY Smart-Its observed in workshops include: - Significant reduction of the development time: instead of several weeks, developers are now able to construct system demonstrators within days. - Significant reduction in knowledge and skills required from developers: even developers without expertise in hardware design are now able to construct complex sensor-based context-aware devices and systems. - Support for reuse: existing hardware and software components can easily be reused for new systems; similarly, hardware and software components can be exchanged between systems. - Support for adaptation and modification: existing Smart-Its devices can easily be modified by adding or removing sensors together with the respective software modules. Developer workshops have been effective not only as evaluation tool but also for dissemination of the technology. DIY Smart-Its have been adopted widely by research groups in the area of ubiquitous computing. The use potential however extends much further, for example for research into creativity and problem-solving with tangible devices, and for development of new educational media.
An important premise of ubiquitous computing is that the use of sensors and elaborate perception techniques will play an important role in order to derive interesting and high-level context. The Smart-Its project therefore served as an enabler to explore the use of multiple sensors, distributed throughout the environment to make applications aware of the situation of the environment and the users. Therefore the research drivers started from an application perspective in order to investigate artefact-based collective sensing, perception and context recognition in diverse applications. This has on the one hand motivated the design of innovative applications in highly diverse application domains and on the other hand resulted in the development of several conceptual frameworks. As an example we briefly summarise two innovative applications investigated during the project: - Proactive Instructions for Furniture Assembly demonstrates how physical artefacts with embedded sensing can collectively recognize states of their assembly and user actions. It illustrates embedding of user instructions as a naturally artefact-centred application, and it highlights closure between plans formulated in the virtual world, and activity in the real world. - A-Life illustrates distributed sensing and context-awareness in an alpine rescue scenario. It demonstrates fusion of distributed sensors, perception of vital context information, and use of collective context to support rescue of avalanche victims. Based on technology design experiences and exploration of application and use, the project developed several conceptual frameworks as for example the following: - The Perception Framework provides layered abstractions for perception and recognition functionality. It constitutes a design and implementation framework for application that are based on distributed sensing, perception and context. - The Semantic Proximity Hierarchy is concerned with contextual relationships and provides a framework for notions of how close entities are in context space. - A Sensing Taxonomy was developed as a first step toward a design framework that helps artefact designers and application developers understand and evaluate sensing opportunities.
As emerging technologies such as SMART-ITS (Interconnected Embedded Technology for Smart Artefacts with Collective Awareness) disappear into everyday objects, supporting possibilities for collective awareness among ‘smart’ artifacts in our everyday life and intimate environments, the relations among users and the things in their surroundings is transformed. There are new interaction techniques, interfaces, actuation, multimodality and interactive behaviours that not only need to be designed, but investigated in relation to new perceptions and possible experiences in use. Within the Smart-Its project, a number of applications have been developed to explore aspects of the computational platform and also to investigate the implications of Smart-Its on design practice and user experience. SMART-ITS RESTAURANT One such application that demonstrates the development of interaction techniques, design methods, and evaluation of user experience is the ‘Smart-Its restaurant’ application. Taking a specific work and everyday context, the application was developed for the highly dynamic flows and objects in a restaurant setting. A set of working demonstrators with Smart-Its augmented artifacts was developed with scenario-based design methods – the resulting application was developed as an exhibition in order to involve hands-on exploration of smart objects and the application was also evaluated in a user test. The prototypes demonstrated how sensitive food items may keep track of their status during their lifecycle by communicating in internal and external networks, how the handling of artifacts individually or collectively could be reflected in dynamic behaviours (for example, in dynamic menus) and how several objects can collaborate and share data to support logistics and workflow of users. Taken together, this suite of demonstrators showed how a variety of objects and tasks can be augmented and supported by sensors, communication and computation. By mixing real sensor data and visual scenarios we created a complete vision of an “intelligent environment” of the future, which visitors could engage and interact with. SMART-ITS APPLICATION DEVELOPMENT In the Smart-Its project, a variety of applications were developed among the project partners (f.ex. Holmquist et al in documents section) which begin to map out a design space of the possibilities for collective awareness and ‘intelligence’ in environments and objects. Typically, in research projects developing smart artifacts or ubiquitous computing applications, it is necessary to build a lot of custom hardware and software from scratch. The Smart-Its platform allows designers and researchers to construct responsive or ‘smart’ environments with comparably little overhead. Future interactive systems will require a multitude of sensors and distributed computation – with Smart-Its and other physical prototyping toolkits, which are starting to appear, it is possible to explore such systems today without building everything from scratch. Application and user experience work in the Smart-Its project collectively constitute a body of design methods, interaction techniques, and user experience evaluations as a body of knowledge for working comprehensively with the technology possibilities of such ubiquitous computing platforms from the design and user-centered development perspective.

Searching for OpenAIRE data...

There was an error trying to search data from OpenAIRE

No results available