On the need for flexible and affective voice interaction in smart buildings

Abstract

The rapid growth of consumer-oriented Internet of Thingsdevices, encompassing smart home applications like energy management, entertainment, security, and environmental control, has created a complex ecosystem of smart devices. Platforms such as Home Assistant (HA)simplify device integration and control, enabling users to query and command all devices from a single hub. However, users still encounter important challenges and a steep learning curve to fluently interact witheach device. This study explores the initial user experience of individualsadopting HA for the first time to accomplish certain tasks involving interaction with smart devices. We conducted a comparative observationalstudy asking participants to perform increasingly complex tasks with HAand by using a voice assistant as an interface to HA. Finally, we askedparticipants to reflect and brainstorm about features they would like thesystem to exhibit so they could accomplish the tasks more easily. Wereport key usability findings regarding observed and self-reported challenges, positive experiences and potentially new voice interaction scenarios to simplify HA functionality. Our findings offer insights for designingintuitive and adaptable interfaces for smart home platforms.