Leveraging cognitive synchronization to create a collaborative music production experience with immersive visualization in virtual reality.
Hyperscanning is a method to detect whether brainwave synchrony exists between two or more individuals, usually due to behavioral or social interactions. It is typically limited to neuroscience studies and is rarely used as an interaction or visual feedback mechanic. In this work, HyperDrum is proposed: a way to leverage cognitive synchrony to create a collaborative music-production experience with immersive visualization in virtual reality. Participants wear electroencephalography (EEG) head-mounted displays to create music together using a physical drum. As the melody becomes more in sync, hyperscanning evaluates the degree of synchrony. The produced music and visualization reflect that synchrony level while simultaneously training participants to create music together, enriching both experience and performance. HyperDrum’s main goal is twofold: blend cognitive neuroscience with creativity in VR, and encourage connectivity between humans using both art and science.
Figure: (left) two participants engaged in HyperDrum, and (right) interactive virtual environment for hyperscanning
Read more about the paper here.
Download the video preview file here
Access the pdf file here