HCI, Interaction Design, Product Design
RCA/IC IDE - Group Major (3 Members)
Contribution: Research, Experiment, Concept, Prototype, Design
Work with:
Awards:  Winner of 2020 Core77 Interaction Design
Winner of London Business School 2020 Health Tech Challenge
Dots is an inclusive interface for future spatial computing, which empowers the disabled people to design their way to interact with MR and IoT based on their body conditions.​​​​​​​
Rising of ubiquitous technology like Mixed Reality (MR) and Internet of Things (IoT) cause the user interface starts to shift from the touch screens to the surrounding environment. In this context, the traditional inclusive interaction design method may no longer applicable and a new approach of interface design for accessibility need to be proposed. By conducting qualitative analyze of experiments and user research, this study provokes novel insights of customizable interaction design and present an inclusive natural user interface (NUI), which enables the people with physical disabilities interact with the spatial computing environment.
/ Watch Video
/ Future Trend
Over the last few years, the development of the Mixed Reality (MR) and the Internet of Things (IoT) has revealed the possibility of a screen-free future, marked we are entering an era of ubiquitous computing. The user interface starts shifting from the touch screens to the surrounding environment. The interaction between human and machine will change a lot. And more spatial interaction will happen. 
/ What has been missed?
Spatial interaction requires more ability of body movement, which brings an increasingly challenging context to the design. However, most of the existing technologies only rely on limited body parts, mainly hands, to do spatial interaction, which decreased the accessibility of technologies. The traditional dominant method of existing gesture recognition which based on supervised machine learning may also fail due to the differences between different disabled peoples
So, how might we involve EVERYONE into the exciting future?
In this context, the traditional inclusive interaction design method may no longer applicable. And a new approach of interface design for accessibility need to be proposed. ​​​​​​​
/ Inclusive Design  vs  Customisable Design ?
Disabilities are often highly individual, which leads to a lack of generality in inclusive design. For better adoption, a large number of inclusive design projects are aiming at smaller groups of people, which makes a high amount of different systems had to be designed for users with diverse conditions. By contrast, if a system with an immutable form or interactive model designed to adapt to people with different characteristics, it could be hard to maintain the same level of efficiency and suability for different users.

Can we, instead of making different users adapt to one system, make a flexible and customisable interface to adapt to different users
/ Method and Approach
To explore the possibility of an inclusive natural user interface for people who have physical disabilities, several experiments were conducted. The goal of these experiments is to extract a full body movement-based interaction pattern for the spatial interaction.

We gave participants four 3D object manipulation tasks and set different limitations of using their bodies.
/ Two-point System
From the observed behavior in the experiment, we found that all the interactions could be described as the relative motion of two points in 3D space. And the result have no difference between the disabled participants and the able bodied participants.
/ Meet Dots
1. Attach to anywhere
Depending on users body conditions, they can attach two dots to any of their body parts, as long as these two parts can accomplish at least one interaction patterns. It is also possible to utilise the environment like attaching one dot on the table. And the exact attaching points depend on the specific task that users wish to do. 
2. Design your own spatial interaction
By connecting Dots with Mixed Reality and Internet of Things devices, users can accomplish multiple tasks using their bodies. We empowered everyone, especially disabled people, to interact with future technologies by letting them design their customized interaction ways based on their body conditions and their specific situations.
Back to Top