Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add UI button actions to send arbitrary OSC values. #140

Merged
merged 9 commits into from
Jan 26, 2025

Conversation

cubee-cb
Copy link
Contributor

Just adds an action to buttons that sends a Float to an address over OSC. (e.g. to control a VRChat avatar, or some other OSC application)

I've just implemented it with OscType::Float for now since it can represent Float, Int, and Bool. (at least for VRChat)
Unsure if making separate actions for each type or trying to handle the type internally somehow (type: float or float_value: 3.5) is better. (in draft for feedback)

# button press example
click_down:
  - type: SendOSCFloat
    parameter: "/avatar/parameters/button1"
    value: 1
click_up:
  - type: SendOSCFloat
    parameter: "/avatar/parameters/button1"
    value: 0

It also merges the OSC client creation from openxr.rs and openvr.rs into state.rs state.app, and makes osc.rs send_params() take in a list of devices instead of app_state itself, since I was having borrow issues calling app.sender.send_params(overlays, app) (approximately).

Small note, setting a bool to 0 works as False, and setting it to 1 shows up as True in VRChat's OSC debug panel, but my avatar doesn't react to the True state. Not sure if I broke something on my avatar, or what. Needs some more testing.

Would also need to update documentation here:
https://github.com/galister/wlx-overlay-s/wiki/Customize-UI#button-actions

a button action that sends an osc parameter.
struggling with borrows in openxr.rs and openvr.rs when getting the osc sender.
by passing a ref to the device list to send_params instead of the entire app state.
@galister
Copy link
Owner

i love the idea!

i just want to use OscType directly, but since it doesn't derive serde::Deserialize, i can't just have one OscType action or list of actions...
@cubee-cb
Copy link
Contributor Author

cubee-cb commented Jan 24, 2025

i'm not 100% happy with having a separate action per-type, and it's currently limited to one argument per parameter (so no vrchat chatbox messages, for example), but i'm not sure if i can do that without modifying something somewhere to make OscType serialisable.

ideally i want to use something like the following format:

    click_up:
      - type: SendOSC
        parameter: "/chatbox/input"
        arguments:
          - type: String # text to send
            value: "Test osc button"
          - type: Bool # send immediately
            value: true
          - type: Bool # play chatbox noise
            value: true

but, i'll mark this as ready if you're happy to merge this as-is.

edit: i came back and tried again with a "stepping-stone" enum for deserialisation that gets converted to the appropriate OscType when sending. much happier, but needs testing.

i added a new struct OscValue that has Deserialize, and now the action just converts that to the corresponding OscType when sending the parameters.
perhaps not the most elegant solution, but it's the best i can think of without modifying the rosc crate.
@cubee-cb
Copy link
Contributor Author

cubee-cb commented Jan 24, 2025

ok, after a brief test it seems to be working, and my avatar's issue of not detecting the True value is resolved now that it's actually a Bool. (perhaps was a vrchat bug...)
multiple values can be sent now, so now we can make quick-chat panels for vrchat's chatbox and such.

here's the testing buttons i used:
(i use a custom layout, these replaced my volume buttons)

  # OSC buttons
  - type: Button
    rect: [346, 50, 38, 38]
    corner_radius: 20
    font_size: 15
    fg_color: "#cad3f5"
    bg_color: "#006000"
    text: "Y"
    click_down:
      - type: SendOscValue
        parameter: "/avatar/parameters/hardware/headsetConnected"
        values:
          - type: Bool
            value: True
    right_down:
      - type: SendOscValue
        parameter: "/chatbox/input"
        values:
          - type: String # text to send
            value: "Test osc button"
          - type: Bool # send immediately
            value: True
          - type: Bool # play chatbox noise
            value: True
  - type: Button
    rect: [304, 50, 38, 38]
    corner_radius: 20
    font_size: 15
    fg_color: "#cad3f5"
    bg_color: "#600000"
    text: "N"
    click_down:
      - type: SendOscValue
        parameter: "/avatar/parameters/hardware/headsetConnected"
        values:
          - type: Bool
            value: False
osc.mp4

@cubee-cb cubee-cb marked this pull request as ready for review January 24, 2025 06:39
@cubee-cb
Copy link
Contributor Author

wrote a section for Customize UI / Button actions on the wiki:

Customize-UI.md

@galister
Copy link
Owner

did you do a cargo format on this? if not, please do

@galister
Copy link
Owner

looks good otherwise

@cubee-cb
Copy link
Contributor Author

cargo fmt?

@galister galister merged commit da129b6 into galister:main Jan 26, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants