Tactile interface to support everyday conversations

What’s it about?

Everyday conversation use a multitude of channels of communication. People with impaired recognition, e.g. Vision, are disadvantaged by this in the interpretation of a communication situation, as they have no access to visual non-verbal information, such as facial expressions. This can lead to a situation where a communication situation is analysed only to a limited extent.

The research professor for socio-informatics at THWS is working on a solution to this. The aim is to support the communication situation by an interface which recognises emotions, transforms them into tactile stimuli and transmits them with the help of a vibrating glove. The solution is based on a coding system for facial muscle movements and is a common standard for detecting emotions based on facial expressions. A new feature is the transmission of essential information via a vibrating glove. In this way, an impaired speaker can be given the facial expressions of the listener during his or her own speech act.

Publications

• Kushnir A. & Müller N.H. (2020). Haptic Feedback in Everyday Conversation Situations. DOI: 10.1007/978-3-030-50726-8_31
• Kushnir A. & Müller N.H. (2020). Development of a Wearable Vision Substitution Prototype for Blind and Visually Impaired That Assists in Everyday Conversations. (Best Paper Award)

Who’s involved?

Anna Kushnir, 3rd semester of the Information Systems Master’s

Supervised by: Prof. Dr. Nicholas Müller