Ideally, for lively discussions to occur during online meetings, the participants should turn on both their camera and microphone. In practice this is not always possible, and meeting participants may opt to use a text chat to communicate their ideas and reactions instead. However, text messages are also time-consuming and labor-intensive to type as well as omit many of the emotional cues available through visual and audio channels. To address these issues, we propose EmoScribe Camera, a virtual camera system that generates images of automatic text captions in real time and outputs them as a software-based virtual camera that simulates a physical camera. We report on the results of a user study evaluating the efficacy of EmoScribe Camera as an alternative communication channel during online conferences when participants have their camera and microphone turned off.

Ari Hautasaari*, Minami Aramaki*, Rintaro Chujo, and Takeshi Naemura. 2024. EmoScribe Camera: A Virtual Camera System to Enliven Online Conferencing with Automatically Generated Emotional Text Captions. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA '24). Association for Computing Machinery, New York, NY, USA, Article 121, 1–7. https://doi.org/10.1145/3613905.3650987  (*: equal contribution)

Paper [PDF]

Minami Aramaki

is a former master's student at the Graduate School of Interdisciplinary Information Studies, The University of Tokyo. Her main research area is Human-Computer Interaction, with a particular focus on Human Communication.

Dr. ARI HAUTASAARI

is a Project Associate Professor in the Interfaculty Initiative in Information Studies, The University of Tokyo. His research interests are in multilingual and socio-emotional computer mediated communication (CMC), computer-supported cooperative work (CSCW), and emotional value exchange in online C2C markets.

RINTARO CHUJO 

is a master student at Graduate School of Interdisciplinarity Information Studies, The University of Tokyo. His main research area is computer-supported cooperative Work (CSCW), and Human-Computer Interaction especially Human Communication.

Dr. Takeshi Naemura

is a Professor in the Interfaculty Initiative in Information Studies, The University of Tokyo. He received his Ph.D. in electronic engineering from the University of Tokyo in 1997. He was a visiting assistant professor of computer science at Stanford University supported by “Japan Society for Promotion of Science (JSPS) Postdoctoral Fellowships for Research Abroad” from 2000 to 2002. His research interest includes Virtual Reality and Human Interfaces. 

This research is part of the results of Value Exchange Engineering, a joint research project between Mercari, Inc. and the RIISE.