In a hongi, noses and foreheads press together and breath is shared. You cannot do that with a hologram, unless you bolt a shape memory alloy actuator to the headset and let it press back. Part three of the volumetric capture series, where the work gains a sense of touch.
By the time First Contact – Take 2 had been exhibited, we knew the volumetric encounter worked. Visitors sat across a table from a life-sized recording of a Māori woman and came away moved. But there was an obvious ceiling to it, and it was cultural, not technical. In te ao Māori, you have not properly met someone until the hongi: noses and foreheads pressed together, the breath of life shared. Our virtual encounter walked right up to that moment and then had nothing to offer. A hologram cannot touch you.
So the next iteration set out to fix exactly that, and Haptic HONGI is probably the strangest and most affecting thing I have helped build: an AR experience where, when the volumetric woman across the table leans in to greet you, something gently presses against your nose and forehead. For a moment, you have met.
Tania
This iteration was again led by Mairi Gunn, and it centres on Tania Remana (Ngāpuhi), a multimedia artist and performer. Tania was recorded as a volumetric video: she welcomes the visitor, talks with them, and then slowly leans forward and invites them to hongi. Everything hangs on her performance. The technology's job is to get out of the way of it.
Making a hologram press back
The haptic unit is a small standalone module that rides on the headset, at this stage a HoloLens 2. It is built around a Shape Memory Alloy (SMA) strip, a bimetallic material that changes length when current heats it. The strip sits in a 3D-printed housing in a loop that can warm up to around 60°C when activated, so the moving ends are capped with small sculpted pieces of makeup sponge: soft on the skin, and a safe buffer so the warm alloy never touches your face directly.
The control chain is deliberately simple. An ESP8266 microcontroller drives the SMA's controlling unit; an Android tablet acts as a portable Wi-Fi hotspot that both the microcontroller and the AR headset join. When the application running on the headset reaches the moment in Tania's performance where she leans in, it issues a command over the network, the microcontroller fires the actuator, and the sponge presses your nose and forehead in time with her movement.
It is a few grams of alloy and sponge, and it completely changes the experience. Without it, you watch someone offer you a greeting. With it, you receive one. The timing is everything. The press has to land exactly when your eyes tell you her forehead reaches yours, and we spent a disproportionate amount of the build tuning that.
A table that could be in two places
This was also the period when the world had opinions about people sharing breath, and exhibitions could not rely on everyone being in the same room. So the playback side grew a second audience: alongside the in-studio visitor wearing the AR headset, a camera feed of the experience could be piped into Mozilla Hubs, letting remote viewers join the room as the encounter unfolded.
That pandemic-era subtext never left the piece. We had built a machine for the safest possible hongi, one across time and space, during the exact years when the real thing was hardest to offer.
The thinking behind this stage of the project is written up in the paper First Contact – Take 2: Using XR to overcome intercultural discomfort, and its expanded version in Virtual Creativity 11(1), 2021.
The next and final post in this series jumps to Sydney, where Haptic Hongi – Reiterated went to SIGGRAPH Asia 2023 on a new headset, with a leaner capture pipeline, and where I got to watch four years of this work compress into a ninety-second queue conversation.
Evolution of Volumetric Capture: