brain_deer.plugins.vr_stream.plugin

VR 3D Stream Plugin for BrainDeer 🧠🦌

Goal:

Stream the BrainDeer 3D view (VTK planes/volume) to an external VR client (e.g. a Unity app running on a Meta Quest 3) and accept basic camera/scale control commands from the headset.

High‑level design:

  • This plugin lives entirely on the BrainDeer (Mac) side.

  • It exposes a small WebSocket server over TCP (default: ws://localhost:8765). The Quest/Unity app connects to this server over the network (Quest Link over USB still exposes a network interface).

  • The plugin repeatedly captures the BrainDeer 3D widget as PNG frames and pushes them to any connected WebSocket client.

  • It also listens for JSON control messages from the client to adjust the 3D camera (position/zoom) so you can move and resize the brain in VR.

Unity‑side (high‑level outline, not part of this file):

  • Use a WebSocket client (e.g. WebSocketSharp or native WebSocket support) to connect to ws://<Mac-IP>:8765.

  • On each received binary message:

    • Decode PNG bytes into a Texture2D.

    • Apply the texture to a quad or sphere in your VR scene.

  • To control the BrainDeer 3D camera, send JSON messages like:

    { “type”: “camera”, “azimuth”: 5.0, “elevation”: -2.0, “zoom”: 1.02 }

    or:

    { “type”: “reset_camera” }

This plugin focuses on the BrainDeer side; you are free to design the Unity scene to your liking.

Classes

VRStreamControlWidget

Small control panel for the VR streaming plugin.

Plugin

VR 3D Stream plugin implementation.

Module Contents

class brain_deer.plugins.vr_stream.plugin.VRStreamControlWidget(plugin: Plugin, parent=None)

Bases: PyQt5.QtWidgets.QWidget

Small control panel for the VR streaming plugin.

Lets you:

  • Start/stop the WebSocket streaming server

  • Set port and FPS

  • Choose whether to stream a white or transparent background

class brain_deer.plugins.vr_stream.plugin.Plugin(api)

Bases: brain_deer.plugin_api.v1.PluginBase

VR 3D Stream plugin implementation.

start_streaming()

Start WebSocket server and frame capture loop. Returns (ok: bool, message: str) for UI.

stop_streaming()

Stop WebSocket server and capture loop.