Mouse not working via VNC / VMware ESXi

[deeplow]

The mouse is not properly working, which is a key requirement for interactive debugging of issues in CI.

Current situation

• Mouse clicks / scroll are detected
• mouse movements are not detected
• mouse is presented as a virtual USB device (available in sys-usb), which needs to have an allow policy to even get to dom0

[deeplow]

So in reality this seems to have nothing to do with VNC but rather just the fact that dom0 does not properly interpret the mouse as a proper device. Here are some preliminary findings:

• after enabling qubes.InputMouse RPC policy sys-usb correctly prompts dom0 to have it connected
• after that a new input device shows as /dev/input/mouse2

On a typical system cat /dev/input/mouse2 prints output when the mouse is moved. However, on our setup it does nothing. So my hypothesis at the moment is that the the main issue is that this mouse virtual device is somehow not properly sending inputs when we hover over the screen.

I'll have to follow up on some linux system to see how its behavior differs from the Qubes setup.

[mig5]

Wanted to note an adjacent matter: I recall the last time I 'persisted' the qubes.InputMouse RPC policy to allow the sys-usb to connect with dom0, this actually broke SDW pytests - there seemed to be a test that examined the RPC policies and failed when it found it didn't match the default one for the mouse. Not sure if that test is still in place.

It was probably this one (at a glance/guess): freedomofpress/securedrop-workstation@bb637dd

Might no longer be applicable (at least on 4.1) due to freedomofpress/securedrop-workstation@20ea5c6#diff-834091304e86b55f7c54d58c8dea60e83802184bf7888bde40f183532ccf9abf

[deeplow]

Thanks for the background / warning.

[deeplow]

Attempt at using treating

Marek suggested that this could be the same case as PiKVM which is detected as a mouse but sends instead absolute coordinates like a tablet. The original goal was to add some udev rules to lead to proper detection. However, I was not getting lucky with those and was finding it difficult to detect which rules were being applied an when. So I opted for another strategy instead.

I modified which qubes-input-trigger in fedora-39-xfce so that next to the PiKVM exception it had this:

elif 'ID_INPUT_MOUSE' in udevreturn and 'ID_USB_MODEL=VMware_Virtual_USB_Mouse' in udevreturn:
        service = 'qubes-input-sender-tablet'

and edited the dom0 policy for qubes.InputTablet to be allowed to dom0.

After restarting sys-usb (or re-running qubes-input-trigger on sys-usb) I get a dom0 prompt for qubes.InputTablet, which means that the change worked. However, nothing happened regarding mouse movements. This could be because it does not work as a tablet or because something got broken.

I think the "something got broken" hypothesis has a case here. Previously, with qubes.InputMouse another device would show on dom0 under /dev/input/mouse2. I reverted the machine to a state where I could test if that was working just to see the kind of data I got, but I could no longer get any data out of it. I tried sudo cat /dev/input/mouse2` on sys-usb where it should be properly be detected and it printed nothing as I moved the mouse. So my best guess at the moment is that somehow the mouse device stopped working completely in my browser setup...

Removing mouse USB controller from sys-usb and not hidding usb controllers from dom0

I was trying to get the USB mouse to directly mount to dom0.

1. On Dom0: qvm-kill sys-usb && qubes-vm-settings sys-usb (and remove the first USB controller -- can also be done via qvm-pci)
2. Edit /etc/default/grub and remove rd.qubes.hide_all_usb
3. Regenerate grub config for BIOS with sudo grub2-mkconfig -i /boot/grub2/grub.cfg

But in the end this did not work. I could see the mouse in dom0 via lsusb, but catting data out of the mouse device yielded no data. This kind of feels like the issue with getting qubes.InputTablet to work.

Next steps

• reinstall OS and see if mouse data reaches dom0 (via qubes.InputMouse)
• reinstall OS with USB mice enabled and see if that works out of the box
• install a linux dristro to see what the mouse behavior is like when catting the device directly and see what differs from the Qubes setup

[deeplow]

• Tried getting USB mouse output via sudo evtest both in sys-usb and in dom0. No event showed, which means that it is not getting detected properly even in sys-usb. This could be due to the drivers
• installed open-vm-tools (VMWare "guest" tools) both in dom0 and sys-usb without much progress

Comparison of evtest listed input devices

In the following image we see in order: the dom0 terminal, sys-usb terminal, and a debian (non-Qubes) VM:

Current hypothesis

My current hypothesis is that something is up with how VMWare provisions mice. On Debian, it is able to detect the PS/2 devices, but on Qubes's dom0 it doesn't show up in dom0.

[deeplow]

I have continued this a bit today, uploading a fedora 40 ISO and checking its behavior with open-vm-tools. I noticed that it was running the same version of open-vm-tools that was installed on Qubes. So I copied over it's settings onto the Qubes VM like so:

Then I also noticed that open-vm-tools even though installed it was not running. So I had to enable the vmtoolsd systemd service. After that I could get window scaling working. Hurray! Progress!

However, the mouse issue was still a problem. So I filed an upstream bug report with open-vm-tools.

[deeplow]

Follow up from the open-vm-tools developers:

To work with the ESXi web console, they either need the vmmouse Linux driver (which is not part of open-vm-tools, but a separate X11 driver package on their distro), or they need to attach a USB controller so that we can use a virtual USB mouse.
The web console only gets absolute mouse positions, and we can't inject that into the Guest without the proper mouse device.
Alternatively, if they use VMRC to connect to the VM, it does support relative mice, and might work if we're guessing the problem correctly.

I have not tried this solution yet, but it seems like this could be it. When searching for vmmouse I also came across this article: https://knowledge.broadcom.com/external/article/341977/mouse-does-not-function-properly-in-linu.html. This could be it.

[deeplow]

Follow up a bit more on this. I found an issue that looks exactly like the same year. But it was 15 years ago or so: https://bugzilla.redhat.com/show_bug.cgi?id=523914