Support all generic tablets and not only OpenTabletDriver

[TailsFanLOL]

I have a literal standalone tablet that I grab stylus input from via intercept and then transfer over the network and put on my computer via uinput (both from intercept-tools, Linux magic). It is recognized as a tablet in the OS and in art programs but not in Osu!, so I can't change the playing area. This gets very annoying very quickly.

I suggest allowing to change the area for all generic tablets. I have no idea how the tablet settings menu works and why it tells me to install OpenTabletDriver, but SDL provides a really nice abstraction for all tablets (libevdev, mobile, windows...).

From SDL3 docs:

typedef struct SDL_PenMotionEvent
{
    SDL_EventType type; /**< SDL_EVENT_PEN_MOTION */
    Uint32 reserved;
    Uint64 timestamp;   /**< In nanoseconds, populated using SDL_GetTicksNS() */
    SDL_WindowID windowID; /**< The window with pen focus, if any */
    SDL_PenID which;        /**< The pen instance id */
    SDL_PenInputFlags pen_state;   /**< Complete pen input state at time of event */
    float x;                /**< X coordinate, relative to window */
    float y;                /**< Y coordinate, relative to window */
} SDL_PenMotionEvent;

Just take the X and Y coordinates and then allow the user to map a range of that to the play field, effectively making a rectangle on the tablet correspond to the entire game window. Obviously, ignore mouse input when the pen is in proximity.

Also, I think the game lacks a little indication that a pen is even acting as one, I got an idea for that too. You know how pressing in on the pen/mouse makes the cursor rotate towards the center when restarting a match? Do the same but with actual tilt angles when in any of the menus.

From SDL3 docs (again):

typedef enum SDL_PenAxis
{
    SDL_PEN_AXIS_PRESSURE,  /**< Pen pressure.  Unidirectional: 0 to 1.0 */
    SDL_PEN_AXIS_XTILT,     /**< Pen horizontal tilt angle.  Bidirectional: -90.0 to 90.0 (left-to-right). */
    SDL_PEN_AXIS_YTILT,     /**< Pen vertical tilt angle.  Bidirectional: -90.0 to 90.0 (top-to-down). */
    SDL_PEN_AXIS_DISTANCE,  /**< Pen distance to drawing surface.  Unidirectional: 0.0 to 1.0 */
    SDL_PEN_AXIS_ROTATION,  /**< Pen barrel rotation.  Bidirectional: -180 to 179.9 (clockwise, 0 is facing up, -180.0 is facing down). */
    SDL_PEN_AXIS_SLIDER,    /**< Pen finger wheel or slider (e.g., Airbrush Pen).  Unidirectional: 0 to 1.0 */
    SDL_PEN_AXIS_TANGENTIAL_PRESSURE,    /**< Pressure from squeezing the pen ("barrel pressure"). */
    SDL_PEN_AXIS_COUNT       /**< Total known pen axis types in this version of SDL. This number may grow in future releases! */
} SDL_PenAxis;
typedef struct SDL_PenAxisEvent
{
    SDL_EventType type;     /**< SDL_EVENT_PEN_AXIS */
    Uint32 reserved;
    Uint64 timestamp;       /**< In nanoseconds, populated using SDL_GetTicksNS() */
    SDL_WindowID windowID;  /**< The window with pen focus, if any */
    SDL_PenID which;        /**< The pen instance id */
    SDL_PenInputFlags pen_state;   /**< Complete pen input state at time of event */
    float x;                /**< X coordinate, relative to window */
    float y;                /**< Y coordinate, relative to window */
    SDL_PenAxis axis;       /**< Axis that has changed */
    float value;            /**< New value of axis */
} SDL_PenAxisEvent;

Well, use SDL_PEN_AXIS_ROTATION to rotate the cursor and "stretch" it with the tilt values by adding them to the size of the cursor, or subtracting them from the starting position when negative. Perfect, now we know that it's actually recognized! Might also use pressure to make the cursor bigger/smaller but I think that would make it an accessibility issue because some tablets lie about the pressure and it would make the cursor too small.

I assume most of this should be implemented in Osu!framework. Should this also be an issue or..?

[peppy]

ppy/osu-framework#6511
#31704
ppy/osu-framework#6523
ppy/osu-framework#6509
#31680

[TailsFanLOL]

The first one is already merged? I tested on the stable branch, gonna try master, I assume it's there by now. EDIT: no, and now the OS "aim" cursor is on top of the game

The other two are about making pen inputs use touch instead?

Also, what about my second suggestion?

[hwsmm]

First of all, SDL does not receive events from evdev, but from X11/Wayland. Receiving events from evdev requires an appropriate permission on most distros, too, as far as I know.

Implementing this would require very hacky code. SDL just sends the pen events from the display server to the game, and osu! already handles them in their own way (but it is being reworked on as said in peppy's comment).

In the other hand, OpenTabletDriver gets position data directly from the tablet itself, and translates it to in-game cursor position.

SDL_PenMotionEvent has X/Y coordinate, relative to window. It is not relative to the actual tablet. Which means that, if the pen goes out of the window, we cannot get pen events from SDL. It makes an area larger than the window impossible.

By the way, osu!lazer has full OpenTabletDriver implementation, and doesn't require installing desktop OpenTabletDriver. If it doesn't work for you, maybe try reading https://opentabletdriver.net/Wiki/FAQ/Linux.

[hwsmm]

X11/Wayland gets events from evdev, but from the perspective of an app, it can only see what the display server exposes, so the game cannot see the cursor outside of the window.

You say that there is no one playing in windowed mode, but that's just your assumption... Regardless of how useful it would be, it would be very hard (or impossible) to make it work properly.

[TailsFanLOL]

but that's just your assumption

Which I never claimed it wasn't

it would be very hard (or impossible) to make it work properly

Suggestion:
Relax. Make it work "not properly".
If someone somehow:

1. Has a tablet not supported by OTD
2. Plays in windowed mode
3. Doesn't like having the window be only in the play area
4. Cares enough to raise an issue

Just stash it behind the 300 other issues (or 1.2k if it's on this repo).

I have a strong feeling that even if it's implemented "properly", the game is gonna loose focus when touching the pen outside of the window.

[hwsmm]

Let's say that the game is in fullscreen. You cannot still use a bigger area in the game than your actual area in desktop, because your tablet driver will not send input outside of the area to the display server. This is way too tricky to implement.

[TailsFanLOL]

You cannot still use a bigger area in the game than your actual area in desktop

Well, I don't know about Windows, but KDE Plasma allows one to stretch the tablet area to the desktop. In fact, it's the default. It's also the default on sway last time I used that. And I would imagine most tablets being closer to a square than most monitors by width, not by height which would be critical to gameplay. And most people would want an area smaller. Again, just an assumption.

[TailsFanLOL]

Why do you even keep insisting that having this limitation makes the game unplayable?

[TailsFanLOL]

So, are those two gonna be implemented? (y/n)

This is mostly gonna benefit people owning generic HID tablets. This will also benefit people using styluses built into device screens when they are using an external monitor (mostly Chromebooks, Lenovo Yoga and Android tablets, hid-over-i2c). And of course me: the only smartass that decided to use a Chromebook as an input device for my other computer.