Four years of research assumed everyone owns a headset, then came a pandemic in which nobody did. XRTB puts the VR person on the Zoom grid.
Every system in this series so far has the same hidden requirement. RemoteFusion needs a Magic Leap on one head and a Vive on another. The CHI study needs both, plus eight depth cameras. The training work needs three Azure Kinects and a machine that can render volumetric video in real time.
Then 2020 happened, and suddenly everybody in the world was doing remote collaboration daily, with a laptop webcam.
That gap is the entire motivation for this last post. Not "how do we make the ideal system", but "how much of this survives contact with what people actually own".
The wrong problem to solve
The obvious move would have been to make the research cheaper: strip the system down until it runs on commodity hardware. That path leads somewhere bad, because you end up with a worse version of a video call and a lot of extra setup.
The better framing turned out to be asymmetry. It is fine for the system to be lopsided. One person having a headset is genuinely useful even if nobody else does, as long as their advantage is legible to everyone else on the call.
So XRTB does not try to bring Zoom into VR. It brings VR into Zoom.
Asymmetric by design: the headset user is in a 3D environment, everyone else is on a normal call.
Becoming a webcam
The mechanism is deliberately unglamorous. The VR application exposes a virtual camera, which the operating system presents to Zoom or Skype as though it were an ordinary webcam. Zoom needs no plugin, no modification and no awareness that anything unusual is happening. It sees a camera; it shows a video feed.
The virtual camera as a physical object in the scene: point it, and that is what the call sees.
What makes this more than a screen share is that the virtual camera is an object in the 3D world. The presenter can pick it up, reposition it, orbit it around a model, hand it a new angle. Framing becomes something you do with your hands rather than a setting in a menu.
The conference window lives inside the virtual environment, so the presenter can see the people they are talking to.
The reciprocal half matters just as much: the Zoom call is rendered inside the virtual environment, as a panel the presenter can position. Without it you get the demo problem, where the person in the headset is talking confidently to an empty room and has no idea their audience is confused.
Desktop mirroring, so ordinary 2D material still has somewhere to live.
The environment also supports multiple presenters sharing the same virtual space while webcam participants join normally, which makes it a genuine hybrid rather than a one-person trick.
Where the series lands
There is something slightly deflating about ending four years of research with "and then we made it pretend to be a webcam." It is also, I think, the most useful thing in the series.
The earlier work answered questions about what to send: reconstruct the room rather than stream it, explicit cues beat implicit ones, audio orients and vision pinpoints, the full volumetric recording teaches best. Every one of those findings is real, and every one is stuck behind hardware that most people will never buy.
Meeting people on the tools they already have is not a compromise on that research. It is the only way any of it reaches anyone. The capstone of the series is the one that gives the technology up.
The paper is XRTB: A Cross Reality Teleconference Bridge to Incorporate 3D Interactivity to 2D Teleconferencing, CHI Extended Abstracts 2021, with Max Collins, Huidong Bai and Mark Billinghurst. Code is on GitHub.
That closes out this series. Every system in it shares cues people send on purpose: a reconstructed room, a gaze ray, a pointing hand, a voice placed in space. The obvious next question is what happens if you share the cues people cannot help sending, which is where Involuntary Signals picks up.