My research lies in the field of human-computer interaction (HCI), and is specifically focused on the electronic and acoustic design of new interfaces to facilitate more natural interactions between humans and smart devices. As computing technology has continued to advance, the limitation on the performance of smartphones, tablets, and video screens is now set by a user’s ability to navigate the large amount of visual information presented on the display of the device rather than the available computing power. There is strong evidence to suggest that devices that combine additional modalities such as spatial sound and touch can improve user’s ability to perform search tasks, and give a greater sense of immersion. However, conventional means of incorporating these additional modalities, especially spatial sound, requires physical installations of loudspeaker and/or microphone arrays that are not conducive to the compact designs of modern displays, and can affect the aesthetics of the space in which they are deployed. This challenge raises two fundamental questions:

1)     How can we design systems that incorporate additional modalities such as spatial sound and touch into display devices without disturbing the aesthetics of the device, or the built environment?

2)     To what extent can devices employing these new systems improve the performance of a human/machine during an interaction, and/or create a greater sense of immersion for the user?

My research addresses these challenges by utilizing the bending vibrations of flat surfaces to serve as the primary audio interface – either by capturing vibration signals induced in the structure by acoustic sources, or by exciting the surface to produce sound-radiating vibrations. Similar methods may also be employed to capture touch inputs for haptic feedback. Combining these capabilities gives the potential to transform the traditional display into a versatile interface capable of employing features such as noise cancellation, texture control, audio source/image pairing, and others outlined in the figure to the left. I call these interfaces smart acoustic surfaces. My research in this area has been supported by NSF grant 2104758.