feat: ✨ Add chroma key support; make passthrough settings real time

[zmerp]

No description provided.

[shinyquagsire23]

So, looking at the settings I've already got a few critiques:

• Default HSV should be #107C10 imo (120deg, 0.871 saturation, 0.486 value), having the green be more dull gives the thresholds more head/foot room to measure distances from, whereas neon green is at the top of the HSV cylinder and can only go downward (and that's also what I use with this SteamVR environment, which I'd highly recommend testing against bc it also gives you a nice way to test semiopacity on the dash).
  • Yes I know most flight simulators and VRChat environments use neon keyed stuff, and they're wrong and they should feel bad about it
• For visionOS what I found works well is having a plain distance min/max threshold instead of per-component distances, especially for semitransparency, with defaults set at [0.35, 0.7] min/max for good-ish semitransparency on that environment. That way the desaturated grays on semitransparency get as much treatment as AA edges and other greens. Though I think technically my hue distances might be borked since 0.0 hue should be 0.0 distance from 1.0 hue, but idk.
  • Ex: If the distance between a pixel and the chroma color is 0.35 or lower, the alpha should be 0.0, if the distance is 0.525 the alpha should be 0.5, if the distance is 0.7 or greater the alpha should be 1.0
• For some reason I was only able to key out the green on that environment by setting the hue to like 200-something degrees and messing with the feathering? It's totally nonsensical for some reason.
• IMO the chroma key color should just be inputted as an sRGB hex code or at least dual-inputtable with one (ideally with something in the UI that shows the color selected).

Edit: Just realized the feathering is basically just my distance thing, but the distances being centered around 1.0 doesn't make a ton of sense, esp considering the visionOS defaults are [0.35, 0.7], so both very low

[shinyquagsire23]

tbh this being in degrees doesn't make a ton of sense? I can see the logic of weighting each of the components in a distance function mayyybe, but the default should be 1.0 weight and should be conveyed as a 0~2 weighting, everything made way more sense when I set this value to 180deg

[daniel5gh]

Quickly glanced over your implementation out of interest, since I was looking to do something similar.

Found a typo.

[shinyquagsire23]

I think the current iteration is extremely servicsble for sims/VRChat where the expectation is like, 'key out my neon #FF0000 and semitransparency isn't a concern'. Good defaults as well. But I will say that the green semitransparency and outlines when I change the saturation ranges makes me a bit sad lol, I think maybe the distance-based keying with the subtract will have to be its own configuration (which is probably fine, it doesn't have to be one size fits all). The distance based keying always struggled with being overly broad to its own detriment, so to reiterate I think this current iteration has an actual strength in that regard.

Other downside is I feel like the min/max stuff is still a bit hostile to the possibility of color pickers (I don't know anyone that can do RGB->HSV conversions off the top of their head), but again maybe a super basic keying option separate from this one would be the better way to handle that.

[zmerp]

I agree that a simpler chroma keying option would work for many people that closely control the key color in their video content, so we offer a single RGB color as parameter. But then we would need to fix the color bleeding. I've come to terms that the color fix on the edges of the mask cannot be purely based on color but also on location of the mask pixels, the neared full 1.0 mask pixel color should bleed into the transition area. I can see that this might be not feasible unless we impose strong restrictions on the size of the transition area.
I'd say to defer this problem to a different PR.

[shinyquagsire23]

tbh distance with min/max feathering and the key subtract does a good job on just edges, but it's just fundamentally reliant on ~some linear relationship between the mask and the key color. But yeah, different PR. This method is good for nuking entire color ranges, which min/max feathering can't really do.

[alexferreira1]

Can you please explain to me the start-end-min-max for the HSV variables? I'd like to passthrough a magenta colour but am struggling to fine tune it