Unity-SFU-Integration

Test project for implementing WebSocket / WebRTC's SFU (Selectable Forwarding Unit) network architecture on the Unity. This project was created for use in Oculus Quest 2, but supports other platforms (PC, Android mobile) to test the connection from multiple clients.

Features

Socket

Each protocol's client (WebRTC or WebSocket) inherits SfuClient class and could calls the Connect, Close, Send, OnMessage function via the same API.

• Broadcast
• Unicast (by user id)
• Multicast

WebRTC (WebRTC Client)

• DataChannel
• Audio
• Video (Not tested yet)
• Trickle-ICE
• Vanilla-ICE (No plans at the moment.)

WebSocket (WebSocket Client)

• Binary
• Text (No plans at the moment.)

Network Event (with user id)

Socket

• OnOpen
• OnClose

Room

• OnJoin
• OnExit

NetworkId

NetworkIds are defined as a combination of public id and private id. public id is set additionally when spawning network manageable gameobject (like Prefab).

NetworkObject

• Could select default protocol for Synchronise (WebRTC or WebSocket)

NetworkObjectGroup

All network-manageable gameobjects (NetworkObjects) are listed and managed in the respective NetworkObjectGroup. Thus, pre-existing NetworkObjects and any spawnable gameobjects are managed in their own NetworkObjectGroup and initialized through the NetworkObjectGroup.

Synchronise the unity's component and Avator

• Transform (threshold base)
  • 16bit Float
  • 32bit Float
  • With Rigidbody
  • With Interpolation
  • Without Interpolation
  • Nesting synchronisable gameobjects
  • Updating based on world space transform
  • Updating based on relative transform
• Animator
  • With Interpolation
  • Without Interpolation (Poor test ...)
• Avator
  • Headset
  • Hand/Finger
  • Elbows/Knees (Maybe I need to add IK first to support it ...)

Interactor

• Grab
• Ray
• Poke

Overview

demo.0.mp4

demo.1.mp4

demo.2.mp4

Get Started

Project Setup

• Clone this repository with the following command.

git clone https://github.com/TLabAltoh/Unity-SFU-Integration
cd Unity-SFU-Integration
git submodule update --init

• Select Multi Pass in ProjectSettings/XR Plug-in Management/Oculus/Stereo Rendering Mode for UI canvas and hand tracking rendering.

• Search asset that named Config on the Project view and set your server's ip/port.

• Open Assets/Samples/VRProject/Scenes/MAIN.unity

Oculus Quest or PC

You may not need to check this as the bellow is the default setting.

• Replace StandaloneInputModule with CanvasModule

• Please confirm that the following setting

Gameobject.name: "Canvas (Screen Space)"

Gameobject.name: "TVRCameraRig"

Android Mobile (not Oculus Quest)

• Replace CanvasModule with StandaloneInputModule

• Please confirm that the following settings

Gameobject.name: "Canvas (Screen Space)"

Gameobject.name: "TVRCameraRig"

• Play Assets/Samples/VRProject/Scenes/MAIN.unity on the Unity Editor or build app

Note

The first joiner is treated as the host, but World Space UI operation is only supported in Oculus / PC mode. The Android mobile client must join after the host joins.

How to move character

PC: just same as Unity Editor's Scene View.
Oculus Quest: Headset's position tracking
Android Mobile: Joystick at bottom left of screen.

Server Setup

• Clone and run the server repository with the following command.

git clone https://github.com/TLabAltoh/rust-server-for-multiplayer.git
cd rust-server-for-multiplayer
build-*.bat
run-*.bat

server repository is here

Issue

Poor test

This project has only been tested on a local network and not on a dedicated server. The server was hosted on a general Windows PC.

Poor documentation

All the features are implemented on the sample scene. And the project has continuously do destractive update. So documentation will be made after project architecture is stable.

Credit

• Oculus quest 2 by Nosakhae is licensed under Creative Commons Attribution.
• Realistic Human Lungs by neshallads is licensed under Creative Commons Attribution.