Associate professor of computer science and engineering Ellen Gethner and PhD student Shannon Steinmetz, along with College of Arts & Media faculty Cecilia Wu and Jeff Merkel, have turned their synaesthetics research into an art exhibit as part of Emmanuel Art Gallery’s “I cannot heave my heart into my mouth” College of Arts & Media faculty exhibition.

The exhibit, titled Synaesthetic Mindscape, is a multisensory experience.

Description: This Embodied Sonic Meditation research concerns the synesthetic

connection of sound triggering color through sensory cross-modality, with an overlaid immersive virtual reality of 3D sound and visual composition. The goal is to simulate the embodied synesthete’s experience by creating an immersive audio-visual virtual reality (VR) installation for the audience to explore through their own bodily self-consciousness.

The current stage of this research is focusing on fast prototyping a multimedia art piece that integrates with surrounding sound and computer-generated graphics, using VR technology to mediate the audience’s experience.  Through observing the audience’s interactions with the system, we will refine the design of the VR meditation/navigation technology and improve the future audience embodied sonic meditation experience. Ultimately we aim to make this an interactive three-dimensional synesthetic VR world, so that the audience can , in real-time, shape sound as an object, and physically walk and explore inside of the visualized sonic world.  

This is an ongoing Office of Research Services Creative Research Collaborative project between the College of Arts & Media and the College of Engineering, Design and Computing.

“I cannot heave my heart into my mouth” faculty exhibition runs from January 23 – February 19 at the Emmanuel Art Gallery on the Auraria Campus.