Leveraging cognitive synchronization to create a collaborative music production experience with immersive visualization in virtual reality.
Hyperscanning is a method to detect if brain wave synchronicity exists between two or more individuals, which is usually due to behavioral or social interactions. It is usually limited to neuroscience studies and very rarely used as an interaction or visual feedback mechanic. In this work, we propose HyperDrum, which is about leveraging this cognitive synchronization to create a collaborative music production experience with immersive visualization in virtual reality. We let the participants wear electroencephalography (EEG) head-mounted displays to create music together using a physical drum. As the melody becomes in synced, we perform hyperscanning to evaluate the degree of synchronicity. The produced music and visualization reflects the synchronicity level while at the same time trains the participants to create music together, enriching the experience and performance. HyperDrum’s main goal is twofold; to blend cognitive neuroscience with creativity in VR, and to encourage connectivity between humans using both art and science.
Figure: (left) two participants engaged in HyperDrum, and (right) interactive virtual environment for hyperscanning
Read more about the paper here.
Download the video preview file here
Access the pdf file here