Skip to content

WIP: Fake cursor position with subsurfaces

Roman Gilg requested to merge romangg/xserver:fake-cursor into master

Motivation

The motivation for this work is to support the Steam Controller in a Wayland session when the Steam client is running.

The Steam Controller basically runs in two different modes with Wayland compositors:

  1. When Steam client is not running the controller works via libinput like a standard USB mouse + keyboard.
  2. When Steam client is running the controller communicates directly with the client and that client then uses XTest extension for cursor movement.

In Xwayland for the second mode we need some kind of direct control of the cursor image position. This is not trivial in Wayland since clients are not supposed to have direct control over the cursor position.

For reference: gamescope issue.

Solution idea

The strategy considered here is to hide the Wayland cursor image and fake it with a subsurface whenever the X-internal cursor position and the one of the Wayland compositor diverge.

For moving the real pointer to the last position of the fake one the real pointer is pointer-locked when the fake cursor is created and a position hint is set.

When the X focus window changes the subsurface is recreated with the new focus window as parent.

Pros

  • Uses already established patterns with subsurfaces and pointer lock.
  • Mostly encapsulated code.
  • Falls back to default behavior when not available or normal pointer is used again.

Cons

  • The recreation of subsurfaces on focus change is complicated.
  • The fake cursor can only interact with Xwayland windows.
  • Subsurface support is underdeveloped in Wayland compositors. I noticed issues with sway which I didn't hit with KWinFT and the other way around.

Current issues

I tested this patch with sway and KWinFT.

  • The locked pointer position hint did always work over the Steam client and only very few times over an Xwayland Konsole window.
  • With KWinFT the cursor image sometimes was not correctly loaded into the subsurface when changing the focus window.
  • With KWinFT the subsurface was not rendered when outside the bounds of the parent window.
  • I haven't done much testing with games yet. With Portal I had sometimes crashes.
  • The keyboard focus was not changed on sway when the Xwayland fake cursor focus changed.

Alternative

Instead of faking the cursor with a subsurface a new Wayland protocol could be introduced giving clients direct control over the pointer position. Such protocols already exist: KDE's fake input protocol and wlroot's virtual pointer protocol (there is wayland/wayland-protocols!12 (closed) for upstream inclusion).

I believe using such a protocol would be more robust in comparison to the approach here with subsurfaces. And it has the big advantage that the pointer could also interact with Wayland native surfaces.

But there is not yet such a protocol in upstream Wayland protocols and while in general useful (for example clients like KDE Connect need it for remote input) there is the open question of security of such an input-injecting protocol that has not yet been fully addressed to my knowledge by any compositor. In particular do we want Xwayland clients to give access to such a protocol?

Merge request reports