SteamVR input changes testing

[Frooxius]

Hello everyone!

I've got another test, that should be relatively quick and small.

Build Version

2024.10.22.1133

What is being tested

There are two main things:

1. We switched our internal project to modern SDK style, which changes the whole build pipeline (I don't expect this to cause any major issues)
2. I've updated SteamVR SDK to latest version and rewrote some of the controller handling to fix issues with Steam Link and hands (see this issue: Resonite doesn't switch SteamLink Hand Tracked & Controllers properly & Crashes #2945)

How to test

• On Steam, switch to the prerelease branch and make sure the build updates before you run it
• Make sure your build version matches the one at the top of this message

You can test alone, I don't expect any multiplayer/networking issues.

This is specifically affecting SteamVR. If you're using desktop or native Oculus support, this won't affect you and your testing will likely not be very relevant (you can still try to build, but it doesn't touch these systems so I don't expect as much).

Here's a list of things you should try out (or at least some of these) and focus on while testing:

What to test

• Test with SteamVR and as many different controllers as you can - Vive Wands, Index, Touch controllers (over Steam Link or Virtual Desktop and such), WindowsMR
• Start Resonite with the controllers off, then turn them on
• Start Resonite with one controller on and then turn the other one on
• Start Resonite with the controllers already on
• If using Steam Link beta, switch to hands from controllers and vice versa
• Turn controllers off and then back on
• Switch from VR to desktop and back
• Do a few basic things - make sure the buttons work, you can move around, click things and so on

What to watch out for

These are good things to watch out for when testing:

• Controllers & buttons/bindings suddenly not working
• Controllers being swapped (left & right)
• Controllers/hands being misaligned
• Crashes and stuff exploding

What NOT to test & report

Generally things that do not involve this system - any existing unrelated bugs with components and other systems.

• If you're unsure, try to test the same scenario on the main build. If it happens there as well, don't report it as part of this testing

If you find something that got broken

• Create a GitHub issue for a pre-release bug: https://github.com/Yellow-Dog-Man/Resonite-Issues/issues/new?assignees=shiftyscales&labels=prerelease&projects=&template=prerelease.yml
• Make sure to include the log
• Please include description of what happened. Be succinct, but exact.

If you find that stuff works okay

Please report it too and include details on what you generally tested, so we know things are okay. You don't need to make a new issue for this - just a commend in this Discussion is fine.

This lets us know that a bunch of stuff was tested and get better confidence that stuff won't break with release.

Please include which SteamVR controllers you've tested and whether you used Steam Link, Virtual Desktop and such.

How long will testing go for?

Once we get good positive results on variety of controllers, we'll release the build next available chance.

If there's issues, we'll keep testing and fixing until the blockers are resolved.

[gameboycjp]

I didn't have too much time to test until my headset ran out of battery, so my testing is very light. I used a Quest 2 with Steam Link. I was able to lock my headset and unlock it with controllers and retain control, and I can confirm hand tracking worked. During all of that, I was putting together a small utility, so I was knee deep in the inspector.

[Frooxius]

Thank you for the report!

[Stellanora64]

Just tested with my reverb G2 for WMR and everything seems fine, I even tried my custom steam vr key binds, and they worked without issue.

I also tested connecting and reconnecting controllers, before and after resonite's start up, and had no issues.

[Frooxius]

Thanks for the report!

[BigRedWolfy]

Just tested with Quest pro with quest pro controllers, doing the switch worked properly. Tried changing from controllers to hands in the steamvr menu, then closing the steamvr menu with hands, and Resonite kept running. ran out my home world and used my menus and no abnormal activity.

[BigRedWolfy]

Looking to try this one I get home to see if it switches when using Index Controllers since I have previously not been able to switch

[Frooxius]

Thank you for reporting!

[Nytra]

index controllers work with quest headset on prerelease, i am using them now

edit: using them through virtual desktop

[Frooxius]

Did anybody test this with Vive wands controllers? These are the ones I'm most concerned by, since they have a bit unique logic due to being able to swap left and right.

[Nytra]

@Frooxius

It is still being really weird. After turning the right controller off and then on again it doesn't track movements anymore, and they are still swapped so the left controller buttons affect the right hand. Sometimes it is possible to fix them after some combinations of turning each one off and on.

DESKTOP-H976HO2 - 2024.10.22.252 - 2024-10-22 05_34_17.log

[Nytra]

It does seem like the worst thing to do is start the app with only the left controller turned on, and then turn the right one on which leads to the broken finger poses and both controllers' buttons affecting the left hand, which actually seems to be unfixable

DESKTOP-H976HO2 - 2024.10.22.252 - 2024-10-22 05_53_59.log

[Frooxius]

@Nytra I rewrote the handling in 2024.10.22.1133

Can you test that one please?

[Nytra]

@Frooxius Everything seems to work now!

[Frooxius]

Ah awesome! Thank you so much!

[ko-tengu]

This build seems less stable for me compared to release using a Quest 3 and Steamlink streaming. Even with hand tracking disabled, disconnecting and reconnecting the headset has a high chance of causing a unity crash. The current release build at worst loses connection to the controllers, which is a preferable failure state.
crash.dmp
error.log
Player.log
BUFU - 2024.10.10.1348 - 2024-10-11 13_22_44.log

[Frooxius]

Do you have any stats on this? How many times does the prerelease crash vs the main build with this scenario?

We had reports of the crashes which seem about the same as the main built and prerelease.

Can you provide logs both for main build and this build for the same scenario?

[ko-tengu]

I can not make the current release build crash from simply turning the headset on and off, so 100 v zero.

This is different from the crashes due to switching between hand tracking and normal controllers, which I can trigger equally as often on both pre-release and release.

[Frooxius]

@ko-tengu How many attempts was that on the current release build and on prerelease?

Crashes can be random, so if you tried it just once or twice, that's not enough data to determine if this crashing is a lot more common on prerelease vs main.

Are all the conditions otherwise same for each build? E.g. same SteamLink version, same test pattern?

Are you able to provide logs from the main build as well for this scenario so we can compare?

[ko-tengu]

With a bit more testing, it seems that I can get the release build to crash doing the same thing, but it takes a little longer for it to go. So I guess I'm instead switching this to "not particularly different"

crash.dmp
error.log
Player.log

[Frooxius]

Ah thanks for the update!

[Foxxboxx]

The Grand Oasis had a meet up today using the pre-release. During testing we ran into an issue with a user (Pyroboy) where switching to different controllers resulted in a crash.

Other observed behavior, but ones we can't determine are controllable in Resonite, is when tracking is lost the hands move to 0,0,0 of the userslot. Which can be disorienting or just annoying. If there is a way to "predict" or just remember the last known location, that would help as losing hand tracking happens quiet often.

We also were unable to find a gesture that could open the main dash. However, thumb to pinky would open the Context Menu.

Nothing else during the limited testing we did (not many users, few testing options) seemed to have issues.

Thinks tested:

Starting Resonite with Controllers disconnected
Starting Resonite in hand tracking mode
Tested with Steam Link Beta
Switched between hand tracking and controller mode repeatedly (done by Myself Mera)
Played around with hand tracking (grabbed objects, interacted with menus, buttons, and other UI elements, and used tools)
Changed between VR and Desktop mode
Pyroboy tested changing between 2 sets of Index controllers, which resulted in a crash.

[RockstarRaccoon]

Testing with Windows mixed reality, HP 1440 Spatial Computing headset.

Turn controllers off and then back on
Switch from VR to desktop and back
Do a few basic things - make sure the buttons work, you can move around, click things and so on

Going to add hitting the recenter button to this list.

Start Resonite with the controllers already on

Starting up worked fine, though my head position tracking had already died by the time I put my headset on.
Turning controllers off and on again worked fine.
Recentering and turning the controllers off and on again worked fine.
Forgot to check everything on the list, but I didn't see a problem with Resonite itself.

MSI - 2024.10.10.1348 - 2024-10-12 19_58_47.log

Start Resonite with the controllers off, then turn them on

When I turned on the controllers, there is a period where I was trying to do things, and even though my hands were moving in the world, I was unable to actually interact with anything or even open the dash menu.

After a minute though, this went away, and even though steam VR itself was malfunctioning and all sorts of weird ways, turning the controller on and off, re-centering, hitting all sorts of buttons, I wasn't able to trigger any more problems.

Something I noticed at this point is that I haven't been able to trigger the thing where the controllers are offset from where mixed reality is tracking them, so that's good.

MSI - 2024.10.10.1348 - 2024-10-12 20_22_22.log

Start Resonite with one controller on and then turn the other one on

I restarted steam VR itself for this one, to clear out the weird behavior it was doing.

When I turned on the left controller, it was not only slightly above where it was supposed to be, but tilted 90° on the y-axis and 45 on the x-axis, making it nearly unusable. I have seen worse offsets on the position, but never the rotation. Everything else seem to be working fine, but I stopped trying things there because I found a problem.

MSI - 2024.10.10.1348 - 2024-10-12 20_44_31.log

This seems to have been related to Resonite and not Steam VR, as going back into the Steam VR home the controller was showing up fine, though while I was restarting Resonite it started showing the offset again, and when Resonite came up, it still had that offset in my localhome.

MSI - 2024.10.10.1348 - 2024-10-12 20_52_17.log

Restarting the controllers did not change this offset, so the left one was still moved forward and the right one was still in the right place. When I got back to the steam homeworld, it still had that offset, so I'm wondering if maybe I just didn't notice it the first time (it was only like 5 cm).

The offset was not there in Windows MR, so I'm going to assume it was Steam as usual.

[troyBORG]

I tested it yesterday with my Quest Pro. I started the game with the controllers on.

I went into the Oculus Dashboard and turned on the hand tracking, and then went and told it to resume Steam Link.
Hand tracking was working. I grabbed my controllers and pulled the trigger and it switched back to controller tracking with no issues.
Turned off one controller and tried to turn on hand tracking. Looks like you can't do one hand tracking, one hand contorller (was hoping for something if battery died you could do a mixed, guess not).

My log file looks like it has a bunch of:

--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <9577ac7a62ef43179789031239ba8798>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0001a] in <9577ac7a62ef43179789031239ba8798>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <9577ac7a62ef43179789031239ba8798>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <9577ac7a62ef43179789031239ba8798>:0 
  at System.Runtime.CompilerServices.TaskAwaiter`1[TResult].GetResult () [0x00000] in <9577ac7a62ef43179789031239ba8798>:0 
  at ProtoFlux.Runtimes.Execution.Nodes.FrooxEngine.Network.WebRequestBase+<RunAsync>d__6.MoveNext () [0x00287] in <680c7251f78246f5906d1b1fd5b4524a>:0

DESKTOP-H0FLKOJ - 2024.10.10.1348 - 2024-10-12 19_54_24.zip

---

I did tried the same thing in current 10.8.1349 release, without closing the steamVR session. I was able to switch from controller and hand tracking without a crash. But when I went back to controllers, my avatar hands were straight up in the air, I could still see my FPSVR on my hand, but couldn't access my dash in game. But to quit Resonite from SteamVR overlay..

[Frooxius]

Does the Quest / Steam Link actually support mix of one hand controller one hand tracking? I've only ever seen it switch once both controllers are gone.

The code we have should technically support it, but I don't think the Steam Link ever activates just one hand or one controller, it always seemed to do both at once.

[Foxxboxx]

I do not believe the Quest itself does. Its in either hand tracking or its not.

[5H4D0W-X]

The Quest OS has supported hybrid "multimodal" tracking for a few months now, the SDK demo app even shows it off (but it doesn't seem to work on my Q2)

[Nytra]

Quest 3 supports multimodal hand tracking (one controller and one hand) but I'm not sure if Steam Link does

[Krzeszny]

I've tested everything on the list on a Reverb G2 with 2 pairs of controllers, WMR and Reverb ones, different configurations, no problems spotted.

[Frooxius]

Thank you for testing!

[Mhowser]

So I've got a pretty niche setup, I'm running Arch linux and launching Resonite with Proton.
My headset is a Pico 4.

I have tested this prerelease with Envision & ALVR

I am happy to report no issues with controllers or hand tracking!

[Frooxius]

Thank you for the report!

[BigRedWolfy]

Tried this now that I'm home with my index controllers...

On the regular build Resonite will not switch between controllers and handtracking at all.

On the current prerelease build, Resonite will attempt to launch and promptly stop, never going past a black screen.

Player.log
WOLFY-BOX - 2024.10.10.1348 - 2024-10-15 19_15_11.log

Logs are from the attempt on prerelease build only

[BigRedWolfy]

I'm doing a bit more testing, this will be elaborated on some more

[BigRedWolfy]

After a few attempts this does appear to be a little inconsistent
Additional details, I'm using the steamvr beta with steamlink on a quest pro mostly with index controllers and a tundra tracker on thge top of my headset for continuous space calibration. I'm also using only my normal cache and data paths that I would use with teh normal public release

With index controllers, the game starts normally with the controllers powered off and using hand tracking. It will switch to the controllers, but not back. While attempting to switch, I also manage to have the game crash.
WOLFY-BOX - 2024.10.10.1348 - 2024-10-15 19_37_41.log
Player.log

In another attempt I managed to switch one hand by accident and was unable to swap hand or controller tracking.

This log attempt was similar to the rest, powering off the controllers and then turning them on after Resonite started and it switched to the index controllers, but not back.
WOLFY-BOX - 2024.10.10.1348 - 2024-10-15 19_34_40.log

During this attempt I tried my quest pro controllers after powering off my Index controllers and exiting Resonite before using the quest pro controllers, starting Resonite, seeing it behave as normal and crashing after a few mins
Player.log
WOLFY-BOX - 2024.10.10.1348 - 2024-10-15 19_37_41.log

[BigRedWolfy]

Note: the last test I did, changing controllers, I did not restart steamvr or steamlink

[zahndy]

Since your setup is almost the same as mine, maybe you can add your problems to this issue #3069

[Frooxius]

I've updated the build to Beta 2024.10.21.1213, which has fixes for Vive controllers. Is anyone able to test?

[Frooxius]

Oops sorry: 2024.10.22.228

This should fix issues with the Vive controllers (hopefully)

[FoxoBread]

I'm running Monado with Vive controllers, would my data be of use? If so, I'm able to test in a few hours. :3

[BigRedWolfy]

Out of interest, would it be possible to have an option to switch the hand of a vive controller inside resonite if steam doesn't assign the correct hand?

[Frooxius]

@BigRedWolfy No. The left/right assignment is controlled by SteamVR, not us. We just have to map things properly.

Problem is that SteamVR is suddenly "Oh, the left controller is now suddenly the right one! And vice versa", which causes all kinds of complications with the system.

[Frooxius]

I pushed another update: 2024.10.22.1133

I rewrote the handling for the Vive controller left/right swapping, so hopefully this one works better.

[wisemagick1]

Tested with most recent prerelease update at the time of this post. I use a HTC Vive headset with Vive Controllers running with SteamVR . All the buttons seem to have worked however I did not have time to do a long term test. I did test powering the controllers on and off. When powered off one at a time the right controller would not regain tracking in game unless it was the first one to connect. There is a moment of hand switching when leaving the left controller on and switching the right on and off where the left hand becomes the right for a brief moment. When both are powered off and I connected the right controller first and then the left both regained tracking. The left hand would always regain tracking regardless of order.

[shiftyscales]

@wisemagick1 - Can you try again on build 2024.10.22.1133 and see if this is resolved? As Frooxius noted:

I rewrote the handling for the Vive controller left/right swapping, so hopefully this one works better.

[FoxoBread]

Works alright with my HTC Vive.
Ran with EnvisionVR as the XR runtime on Arch Linux.

No issues to report, other than found some odd skeletal alignment likely caused by Envision, as this did not happen on SteamVR.

[Frooxius]

Did you use the Vive wands or Index controllers?

Which version did you test? I released a new one a short while ago.

[FoxoBread]

Vive wands, on latest pre-release version Beta 2024.10.22.1133
Most notable is the pinky being rather deformed.

This is the same behavior I experienced before updating.

[Frooxius]

Ah sweet, thank you for the info!

[Frooxius]

Just heads up, since the last issues that was blocking this (the Vive wands) is resolved and I don't see any more GitHub issues that would be regressions, I'll be pushing out a build with this update soon!

[SiderealScout]

I was experiencing some crashes when swapping from controllers to hand tracking and back, using Quest3 and wireless streaming via VirtualDesktop and SteamLink both using the SteamVR runtime, I was planning to run some tests and posts some logs here, or should I wait, or post them in separate threads?

[Frooxius]

We already have issue for that here: #3066

Main question is, does this NOT occur at all on the main build? Unless it's a regression, we won't hold the release for it. So far every report of crashes seems to be about the same on main build and on this.

[shiftyscales]

@SiderealScout - That particular issue is not a regression (it occurs on the main build as well)- it has been reported a few times in #2041, #2945, #3066, and #3093.

[Frooxius]

Thanks everyone for helping to test! I'm closing this now.

Any other issues that you discover, please just make regular issues for it!