Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sfu-to-sfu talk MSC3401 and add new features #11

Merged
merged 126 commits into from
Aug 17, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
126 commits
Select commit Hold shift + click to select a range
c6a93ed
remove http API; add basic mautrix VoIP hooks
ara4n May 31, 2022
1795332
add sample config
ara4n May 31, 2022
5fe938d
rename sample config
ara4n May 31, 2022
a81ddea
.gitignore
ara4n May 31, 2022
1ac4be0
typo
ara4n May 31, 2022
1368d31
go fmt
ara4n May 31, 2022
8d8303d
merge main
ara4n May 31, 2022
e7c757a
Merge branch 'master' into matthew/matrix
ara4n Jun 2, 2022
0a0e308
theoretically hook up matrix to the focus (WIP)
ara4n Jun 3, 2022
36b5b09
we can infer device_id from /whoami
ara4n Jun 3, 2022
0586e38
pin to right go-mautrix
ara4n Jun 3, 2022
f604f23
fix build
ara4n Jun 3, 2022
5af7532
fix dep
ara4n Jun 3, 2022
782bcd2
hook up SDP-over-DC signalling again
ara4n Jun 3, 2022
e9844a4
hook up sfu->client ice candidates
ara4n Jun 3, 2022
9129b4e
hook up client->sfu candidates
ara4n Jun 3, 2022
79f887a
whack up the debug
ara4n Jun 5, 2022
5f1dc72
handle todevice msgs correctly
ara4n Jun 5, 2022
6e1eb29
fix stupid map lifecycle bugs
ara4n Jun 5, 2022
6599350
fix up sessions
ara4n Jun 5, 2022
6cb5842
unmarshal DC correctly
ara4n Jun 5, 2022
e2e8a82
debugging
ara4n Jun 5, 2022
ba82de8
experiment: stop namespacing on streamIDs
ara4n Jun 5, 2022
1dd777b
moar logging
ara4n Jun 5, 2022
fba15af
don't barf on candidate completion
ara4n Jun 5, 2022
95cffd1
more debugging; handle stream lookup failure correctly
ara4n Jun 5, 2022
8bd0d04
reenable ICE
ara4n Jun 5, 2022
0053c43
only send ICE candidates after we've calculated an answer
ara4n Jun 5, 2022
fb61684
remove debugging
ara4n Jun 6, 2022
f3d9cdb
Delint
SimonBrandner Jul 30, 2022
46c7e3a
Add more logging for DC
SimonBrandner Jul 30, 2022
b7add84
Add useful logging
SimonBrandner Jul 30, 2022
0305ee0
Update `.mod` files
SimonBrandner Jul 30, 2022
2b4bafb
Fix types
SimonBrandner Jul 31, 2022
831d7b0
Be more precise about `m.call.negotaite`
SimonBrandner Jul 31, 2022
a3ec7d8
Delint
SimonBrandner Jul 31, 2022
ca48340
Delint
SimonBrandner Jul 31, 2022
5291787
Make basic group calling work
SimonBrandner Jul 31, 2022
d0cb49b
Handle events we don't care about nicer
SimonBrandner Aug 1, 2022
8393552
Handle `m.call.hangup` and `m.call.select_answer`
SimonBrandner Aug 1, 2022
699e362
Allow lookup of tracks by info about them
SimonBrandner Aug 1, 2022
d592bf3
Make leaving confs work
SimonBrandner Aug 2, 2022
0caefb1
Move golang files into `src`
SimonBrandner Aug 2, 2022
209be90
Reogranize source code
SimonBrandner Aug 2, 2022
7d57680
Add newline
SimonBrandner Aug 2, 2022
ab61cda
Remove `static` dir
SimonBrandner Aug 2, 2022
8eff32f
Delint `README.md`
SimonBrandner Aug 2, 2022
484aed8
Add copyrights
SimonBrandner Aug 2, 2022
3659366
Extract `onTrack`
SimonBrandner Aug 2, 2022
c690431
Extract `sendDataChannelMessage()`
SimonBrandner Aug 2, 2022
2d8cf74
Add support for `publish`
SimonBrandner Aug 2, 2022
5b62e45
Extract `sendDataChannelError()`
SimonBrandner Aug 2, 2022
a78c8ec
Correctly remove tracks when a person leaves a call
SimonBrandner Aug 3, 2022
37b1c35
Update scripts
SimonBrandner Aug 3, 2022
6d91759
Move scripts into a separete dir
SimonBrandner Aug 3, 2022
1ad36e9
Update README
SimonBrandner Aug 3, 2022
ee5a4c3
Select by both `streamId` and `trackId`
SimonBrandner Aug 3, 2022
805244e
Improve logging
SimonBrandner Aug 3, 2022
cb7c76f
Correctly log `select`
SimonBrandner Aug 3, 2022
8398119
Log `userID`s instead of `callID`s
SimonBrandner Aug 3, 2022
cd3076b
Fix terminate call log
SimonBrandner Aug 3, 2022
b999bda
Fix more logs
SimonBrandner Aug 3, 2022
0d373b0
Don't error if we found no tracks
SimonBrandner Aug 3, 2022
5657c6d
Try to fix race condition where track is later than state event
SimonBrandner Aug 3, 2022
060a528
Fix `tracks` mutex
SimonBrandner Aug 3, 2022
c591a4a
Handle no tracks to remove better
SimonBrandner Aug 4, 2022
6063f2c
Better logging
SimonBrandner Aug 4, 2022
646efa7
Add `unpublish`
SimonBrandner Aug 4, 2022
ca546c6
Remove old call with device id when we get a new one
SimonBrandner Aug 6, 2022
4778573
Fix `addSubscribedTracksToPeerConnection()` being called for incorrec…
SimonBrandner Aug 6, 2022
928604e
Add comment
SimonBrandner Aug 7, 2022
b8479f0
Remove empty line
SimonBrandner Aug 7, 2022
24a3e0b
Support profiling
SimonBrandner Aug 7, 2022
e5bec68
Better cpu and memory profiling
SimonBrandner Aug 8, 2022
d80dada
Implement sessions ids
SimonBrandner Aug 8, 2022
f0282d8
Theoretical perf improvement
SimonBrandner Aug 9, 2022
ebf4e10
Use gorutines
SimonBrandner Aug 9, 2022
becdee7
Lock later
SimonBrandner Aug 9, 2022
7eb8480
Move `getConf()` to the correct place
SimonBrandner Aug 11, 2022
c7cda4d
Implement timeouts
SimonBrandner Aug 11, 2022
6149d7f
Remove some debug logs
SimonBrandner Aug 11, 2022
d3b4ba3
Add vscode to gitignore
SimonBrandner Aug 11, 2022
80e9192
Fix timeouts
SimonBrandner Aug 11, 2022
3f56374
Make logging time optional
SimonBrandner Aug 11, 2022
4845a7e
Fix warning
SimonBrandner Aug 11, 2022
bdd0bbe
Remove unused packages
SimonBrandner Aug 11, 2022
9c1e479
Fix disabling loggin
SimonBrandner Aug 11, 2022
4dfd79e
Don't log some events
SimonBrandner Aug 12, 2022
b80edd6
Improve logging
SimonBrandner Aug 12, 2022
02e553c
Switch to data-channel for sending metadata
SimonBrandner Aug 13, 2022
02825ea
Update README
SimonBrandner Aug 13, 2022
e513dd1
Add new lines
SimonBrandner Aug 13, 2022
e473cd9
Remove cascade - will be readded later
SimonBrandner Aug 13, 2022
7dbe06b
Follow naming conventions
SimonBrandner Aug 13, 2022
052aacc
Add `WriteRTCP()`
SimonBrandner Aug 13, 2022
8d730e0
Rename `config` to `Config`
SimonBrandner Aug 13, 2022
cc822e2
Remove unused field
SimonBrandner Aug 13, 2022
8e1c608
Improve typing
SimonBrandner Aug 13, 2022
59907f5
Rename type
SimonBrandner Aug 13, 2022
22b3d81
Move types into mautrix-go
SimonBrandner Aug 13, 2022
ccf28d5
Use `ToDevice` prefix
SimonBrandner Aug 13, 2022
b528c03
Move types to mautrix-go
SimonBrandner Aug 13, 2022
250e2c9
Remove comment
SimonBrandner Aug 13, 2022
839ae57
Remove type
SimonBrandner Aug 13, 2022
e82cc39
Improve logging in utils
SimonBrandner Aug 14, 2022
71c2a0f
Don't timeout already ended calls
SimonBrandner Aug 14, 2022
9abb6d9
Break out of `WriteRTCP` loop on error
SimonBrandner Aug 14, 2022
0bb1871
Add somu mutex locks and unlocks
SimonBrandner Aug 14, 2022
5c45ebf
Fix incorrect mutex usege
SimonBrandner Aug 14, 2022
8d45581
Fixup same small issues
SimonBrandner Aug 14, 2022
f528225
Don't assume everyone is Šimon
SimonBrandner Aug 15, 2022
4bf1c55
Don't kill the SFU when receiving invalid messages
SimonBrandner Aug 15, 2022
f0cfb9e
Improve log
SimonBrandner Aug 15, 2022
1e127f0
Separate `DataChannelHandler()`
SimonBrandner Aug 15, 2022
80be219
Don't panic where we don't need to
SimonBrandner Aug 15, 2022
6a2bf28
Comment `GetRemoteMetadataForDevice()`
SimonBrandner Aug 15, 2022
a0504d0
Rename function
SimonBrandner Aug 15, 2022
02bcae8
Add `NewFocus`
SimonBrandner Aug 15, 2022
42ebd11
Fix public/private
SimonBrandner Aug 15, 2022
8698c08
Rename var
SimonBrandner Aug 15, 2022
7a47a55
Send `UsernameFragment`
SimonBrandner Aug 15, 2022
223fbd7
Update
SimonBrandner Aug 15, 2022
e0255e4
Restructure to use `OnEvent`
SimonBrandner Aug 15, 2022
b7e8cbc
Update `go.mod`
SimonBrandner Aug 15, 2022
cc0c2e5
Send end of ICE
SimonBrandner Aug 17, 2022
0f40571
Remove UsernameFragment
SimonBrandner Aug 17, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
config.yaml
dist/
*.pprof
.vscode/
195 changes: 31 additions & 164 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,177 +2,44 @@

## Why

`SFU-to-SFU` is an example of a cascaded decentralised SFU. The intention is to be a implementation of Matrix's [MSC3401: Native Group VoIP signalling](https://github.com/matrix-org/matrix-spec-proposals/blob/matthew/group-voip/proposals/3401-group-voip.md).
This example is self contained and doesn't require any external software. The project was informed by the following goals.

* **Easy Scaling** - SFU count can be grown/shrunk as users arrive. We don't scale on the dimension of calls making things easier.
* **Shorter Last Mile** - Users can connect to SFUs closest to them. Links `SFU <-> SFU` are higher quality then public hops.
* **Flexibility in WebRTC server choice** - All communication takes place using standard protocols/formats. You can use whatever server software best fits your needs.
* **Client Simplicity** - Clients will need to be created on lots of platforms. We should aim to use native WebRTC features as much as possible.

The SFUs themselves have no concept of conference calls/rooms etc... All of this is communicated in the Matrix room. The SFUs themselves just operate off of
pub/sub semantics. The pub/sub streams are keyed by `foci`, `call_id`, `device_id` and `purpose` these keys come from [MSC3401](https://github.com/matrix-org/matrix-spec-proposals/blob/matthew/group-voip/proposals/3401-group-voip.md).

Lets say you have a Matrix room where user `Alice` wishes to publish a screenshare to `Bob` and `Charlie`.

```
* `Alice` establishes a session with a SFU
* `Alice` publishes a screenshare feed with `call_id`, `device_id` and `purpose`
* `Alice` publishes to the matrix room with the values `foci`, `call_id`, `device_id` and `purpose`

# Connecting directly to publishers FOCI
* `Bob` connects directly to `foci` and establishes a session.
* `Bob` requests a stream with values `foci`, `call_id`, `device_id` and `purpose`.

# Connect to FOCI through different SFU
* `Charlie` connects to a SFU they run on a remote host.
* `Charlie` requests a stream with values `foci`, `call_id`, `device_id` and `purpose`.
* `Charlie`'s SFU connects to `foci` and requests the stream.
* `Alice`'s stream arrives to Charlie via `Alice -> FOCI -> Charlie's SFU -> Charlie`
```
`SFU-to-SFU` is an example of a cascaded decentralised SFU. The intention is to
be a implementation of Matrix's [MSC3401: Native Group VoIP
signalling](https://github.com/matrix-org/matrix-spec-proposals/blob/matthew/group-voip/proposals/3401-group-voip.md).
This example is self contained and doesn't require any external software. The
project was informed by the following goals.

* **Easy Scaling** - SFU count can be grown/shrunk as users arrive. We don't
scale on the dimension of calls making things easier.
* **Shorter Last Mile** - Users can connect to SFUs closest to them. Links `SFU
<-> SFU` are higher quality then public hops.
* **Flexibility in WebRTC server choice** - All communication takes place using
standard protocols/formats. You can use whatever server software best fits
your needs.
* **Client Simplicity** - Clients will need to be created on lots of platforms.
We should aim to use native WebRTC features as much as possible.

This implements the MSC only roughly - given the current experimental nature of
this projects, it deviates in certain areas from the MSC.

## How
### Establishing a session
Client sends a POST with a WebRTC Offer that is datachannel only. Server responds with Answer.

Server will open a datachannel called `signaling`. Clients can send publish/subscribe now.

`POST /createSession`

`Request`
```
o=- 6685856480478485828 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0
a=extmap-allow-mixed
a=msid-semantic: WMS
m=application 9 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 0.0.0.0
a=ice-ufrag:gLSF
a=ice-pwd:xuxSHK0uJuSb607uYunnzlCQ
a=ice-options:trickle
a=fingerprint:sha-256 C2:1F:9B:A1:C2:DF:7E:13:E4:F9:64:F5:EC:4D:17:A1:89:21:0E:32:61:2A:B7:A5:A7:2A:7C:06:AC:FB:B2:A1
a=setup:actpass
a=mid:0
a=sctp-port:5000
a=max-message-size:262144
```

`Response`
```
o=- 1712750552704711910 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0
a=extmap-allow-mixed
a=msid-semantic: WMS
m=application 9 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 0.0.0.0
a=ice-ufrag:90cu
a=ice-pwd:PARVC6h9kLvvgCqxSocjrXYZ
a=ice-options:trickle
a=fingerprint:sha-256 7F:79:0F:50:FF:D1:3F:DF:CA:BD:06:89:2B:C8:05:2E:EC:7D:EF:66:AF:A8:6E:D8:70:C6:74:68:E6:5C:47:D7
a=setup:active
a=mid:0
a=sctp-port:5000
a=max-message-size:262144
```

### Publish a Stream
A user can start publish a stream by making a JSON request to publish with a new Offer. With the following keys.

* `event` - Must be `publish`
* `id` - Unique ID for this message. Allows server to respond with with errors
* Stream Identification - `call_id`, `device_id`, `purpose`
* `sdp` - Offer frome the Peer. Any new additional tracks will belong to the stream.

```
{
event: 'publish',
id: `ABC`,
call_id: 'AAA',
device_id: 'BBB',
purpose: 'DDD',
sdp: `...`,
}
```

** Errors **
* Stream already exists
* Server over capacity

The server will respond to the `subscribe` with the answer.

```
{
event: 'publish',
id: `ABC`,
call_id: 'AAA',
device_id: 'BBB',
purpose: 'DDD',
sdp: `...`,
}
```


### Subscribe to a Stream
A user can subscribe to a stream by making a JSON request to subscribe with a new Offer. With the following keys.

* `event` - Must be `subscribe`
* `id` - Unique ID for this message. Allows server to respond with with errors
* Stream Identification - `call_id`, `device_id`, `purpose`

```
{
event: 'subscribe',
id: `ABC`,
call_id: 'AAA',
device_id: 'BBB',
purpose: 'DDD',
sdp: `...`,
}
```

The client will respond to the `subscribe` with the answer.

```
{
event: 'subscribe',
id: `ABC`,
sdp: `...`,
}
```
### Configuration

** Errors **
* Stream doesn'texist
* Server over capacity
* `cp config.yaml.sample config.yaml`
* Fill in `config.yaml`

### Unpublish a Stream
```
{
event: 'unpublish',
id: `ABC`,
call_id: 'AAA',
device_id: 'BBB',
purpose: 'DDD',
}
```
### Running

### Unsubscribe to a Stream
* `./scripts/run.sh`
* Access at <http://localhost:8080>

```
{
event: 'unsubscribe',
id: `ABC`,
call_id: 'AAA',
device_id: 'BBB',
purpose: 'DDD',
}
```
### Profiling

## Running
* `./scripts/profile.sh`
* Access at <http://localhost:8080>

* `go run *.go`
* Access at http://localhost:8080
### Building

* `./scripts/build.sh`
* `./dist/bin`
* Access at <http://localhost:8080>
128 changes: 0 additions & 128 deletions cascade.go

This file was deleted.

4 changes: 4 additions & 0 deletions config.yaml.sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
homeserverurl: "http://localhost:8008"
userid: "@sfu:shadowfax"
accesstoken: "..."
timeout: 30
Loading