Fingis

Open-ended smart products

With a growing number of smart speakers, thermostats and smart fridges on the market, connected devices are now a part of everyday life. But for the people entering this next generation of household products, making the upgrade also means letting brands reach closer than ever before into their home with products often very opaque about the way they work. As connected products get more advanced, it becomes increasingly difficult for people to understand the objects around them. Unfortunately, the default approach for the makers of these products is to treat people as passive consumers of their own predefined vision of smart living, prioritising convenience over control in the process. This puts users in situations where they can easily be manipulated or have their data exploited by their own products. Could we instead see consumers as collaborators capable of shaping products as they live with them?

What if people had more influence on defining what smart living meant to them, would they want the intercom to prioritise safety or hospitality in their home? Would they accept to have a camera in their kitchen if it was to bake better bread? The Fingis propose a model of how we might live with technology and relate to it as it grows into the things that make a home.

Each of the three Fingi gives appliances new sensing capabilities that inform the entire ecosystem's intelligence:

• The Purposing Fingi senses what is happening inside an appliance. How it is used, for how long and so on. It can also control the device to implement modes to help a person to use it more, less or differently depending on the mode the Purposing Fingi is set on.
  
• The Social Fingi senses how similar appliances are being used in other homes and establish a bridge between two local Fingi ecosystems. This bridge opens possibilitie for applications such as radios to radio communication, or the coordination of the lights in different apartments for example.
  
  
• The Observing Fingi senses what is happening in the surrounding environement of an appliance with a camera and a microphone.
  

Making AI tangible

The Fingis collect data from the appliances they are plugged into and use AI to uncover new ways to use and combine them. These new features can range from automating routine tasks to coordinating multiple appliances, or even discovering alternatives uses for your home’s devices. As the Fingis open more and more possibilities to interact with the appliances they affect, they turn into a personal home improvement assistant, helping people to get creative with their newly augmented appliances and build the smart home of their choosing.

Like mushrooms popping up on the surface of an organism colonised by a fungus, the Fingis are the physical manifestation of the AI that exists in a smart home. This metaphor invites people to look at AI as an ecosystem of interdependent actors that can affect each other in different ways depending on the way they configure and interact with them.

How it works

While primary interactions with the Fingis happen directly on the appliance they sit on, the companion app allows people to keep an overview of the system and manage the way Fingis collect and use the data they collect.

When an appliance is connected to the Fingi network, it can be visualized in the app. There, people can see all the devices in their network and the way they interact with each other.

The Purposing Fingi can be used to give the system individual directions on how to improve each appliance on a case by case basis.

The app allows people to monitor and delete data points registered by their Fingis. Over time, patterns in the data can lead to update suggestions pushed by the Fingis through the app.

Exploring the system with scenarios

Throughout the project, tangible stories of how an ecosystem of connected appliances would behave were used from the start to explore the concept, let co-creation participants play with it, and ultimately refine the rules of the system.

SCENARIO 1

Toast FM

After detecting the morning routine, the toaster develops the ability to automatically turn on the radio in the morning.

SCENARIO 2

Training a connected Kettle

Coupled with an intercom, a kettle learns that it is common etiquette to prepare some tea when receiving guests.



But as it turns out, not every person ringing the door is a guest. An observing Fingi can be added to the intercom to allow it to recognize visitors from deliveries.

SCENARIO 3

Living with a Social Radio

Two radios augmented with social and observing Fingis are connected to allow two friends to create a shared audio experience between their kitchens.

Later on, after one of the radio hears its owner ask his friend some help with his plants, the augmented radio takes note from the way the friend has been using her radio to play pleasant music for her green tenants and mimics this behavior.

SCENARIO 4

The intercom records the voice of its owner when he talks to his visitors when they ring the bell. Using the nearby radio’s speakers, the intercom then offer to welcome expected guests and let them in when they arrive a the door.

The scenarios in action

Process over 10 weeks

The Fingis are the results of weeks of desk research, interview sessions, exploratory tinkering and multiple sacrificial prototypes tested with real people.

"I don't even see the tatoos on my arms anymore"

- Brian, tattoo artist

Using analogous research to understand the concept of object evolution

From looking at these seemingly unrelated cases together, I realised that much like how we don’t think about the algorithms behind our favorite apps or about what our products do with our data, we have a tendency to overlook the details of the things around us when they stop to be at the forefront. Instead, people have a tendency to simplify their perception of a tool to the main purpose they use it for. This is why people think of Facebook as a tool to stay in touch with their friends and not as a platform used by businesses to develop and tap into the online profiles of nearly everyone on the planet with an internet connection. Both can be said to be true, but very few people think of the latter when they use Facebook to get news from their friends everyday.

On the other hand, looking at the way the designers of customizable bicycles and furniture involve their customers in the fabrication of their products made me understand that things like building habits or encouraging participation can be powerful tools to make people care more about their own products. These practices are effective because they take customers on a path to develop an holistic understanding of their products until they know them nearly as well, or sometimes better than their makers. That’s why I decided to design an intelligent product built around the process of making users regularly contribute to the parameters of its system and AI, with the assumption that this would empower consumers to always be able to understand, and control their complex products when they are using them, even as they start to evolve.

Read the complete article about the research phase of the project

Early concepts
Collective Intelligence

As I was looking for ways to help people understand what the things around them would be able to do in the context of a smart home, I started to play with the idea of making appliances collaborate to achieve more than they could on their own. My hypothesis, based on what I learned about the way people perceive the tools they use, was that something as complex as multi-device interaction and collective intelligence could be brought down to a person’s level of understanding if each object in the system remained in character, with its capabilities limited to what it was already able to do on its own. Following these principles, the primary interface of a smart home would be the electric appliances that we already interact with everyday.

Then, to study systems based on decentralisation and networked intelligence, I looked at how fungi that grows in forests soil are enabling trees to communicate with each other and even exchange nutrients through a network made of mycelium. This natural communication network is able to make an entire ecosystem more resilient just by complementing the way trees naturally absorb nutrients to grow by giving them new channels to exchange and receive nutrients without altering the way they do it normally. In a similar way, I wanted to design a system where appliances would be able to serve people new needs for smart living while remaining the same devices they are now, using collaboration to complete each other and do more.

First prototype

To test on the potential of this idea, I prototyped a few scenarios of interactions between household appliances to explore the different forms of collaborations that could exist within a home. Through this series of experiments, I found that a lot could emerge from, on one hand making it possible to read and combine information about the way each appliance is being used, and on the other hand making it possible for the system to completely control appliances and let them coordinate with each other. As I was experimenting with scenarios within these constraints, I realised that the possibilities opened by a system built on these principles and the questions that it would raise were well worth exploring on their own given the time frame of the project. I decided to focus on defining what the experience of managing an ecosystem of appliances would feel like and what people would want to use it for.

A fan reacts to the kettle releasing steam and blows it away.

Creating the model of interaction with people in their home

To test a few models of interactions that people could use to control a network of appliances, I created a low fidelity prototype of the Fingis and asked people to use them in their home and connect their appliances to one another. At this stage, I had the idea of using multiple types of enablers to give different capabilities to each appliance. This would provide people a set of physical controls to program and use the system. However, I wanted to try different approaches to find how to make this set of enablers work in a coherent way and iterated the composition of the kit through a series of sessions.

Tools and props used during co-creation sessions

From trying different configurations for the Fingis kit, I realised that giving people too much control over the system was not always ideal. When the participants could create and configure how each appliance would interact with one another, the burden of imagining every possible combination would fall on their shoulders and they would simply find the system too complex to even see its value. Instead, I found that it was preferable to refrain from asking people to focus on the relationship between each appliance in the system, but rather to let them consider each appliance individually and to frame the different Fingis as tools that give each appliances a set of goals and capabilities.

With this approach, the system would then use AI to automate the process of generating new ways to use the connected appliances around the home and to let people pick from the updates and combinations they liked. When people's people's focus was placed on individual elements of the network rather than on the whole picture, this model of interaction gave the participants a different way to think about their appliances ecosystem. It invited them to explore the system by manipulating each actor individually to then observe how different parameters would affect the overall system over time.

But while using AI to generate potential applications enabled by the Fingis made the concept more approachable for people, it also brought up a lot of questions about the dynamic of the relationship that this would create between people and their network of appliances. If the things in our home had the power to directly change the way we use them, what type of information would people want their appliances to collect in order to create these modifications? Would they accept that their appliances communicated with the outside world to make better recommendations? How much of their life at home would they be willing to delegate to an AI and when would it become too intrusive? These are some of the questions that I also explored during the co-creation sessions using different research tools and prototypes to walk participants through different scenarios and define some of the characteristics of the Fingis' AI. And in the end, while the final outcome of the Fingis is a take on some of these questions, the goal of this project was also to give people another perspective on the inner workings of the connected home and the intelligence that may soon drive the objects around us.

Thanks to Anders Højmose for being an amazing advisor all along this project and to the many others who helped along the way.