Multi-pen support and persistent pointerId

[scottlow]

After digging around in issues for a while, I don't believe this has been raised before, but please feel free to dupe if so.

Lately, we at Microsoft have been thinking a bit about multi-pen scenarios for devices like Surface Hub. In cases like these, it is sometimes desirable to assign a pen a unique pointerId per top-level browsing context that remains static for the duration of the session regardless of whether the pen moves beyond the range of the digitizer. This would enable various scenarios:

• "Remembering" the selected attributes of a specific physical pen within a web-based drawing application, allowing users to switch between physical pens of multiple colors/types/stroke widths/etc. just as they can do on a physical whiteboard
• Allowing multiple people to collaborate at once using "different" pens. Today, I believe the best possible experience we can offer on the web is to let multiple people collaborate using a single pen style

Given various language in the PointerEvents spec such as

If a touch contact or pen stylus is lifted beyond the range of the digitizer, then it is no longer considered active

and

A user agent MAY recycle previously retired values for pointerId from previous active pointers, if necessary.

It may be that to maintain backwards-compatibility, such an idea would be best implemented as part of a new specification specifically designed for this use case, but I thought I'd float the idea here first to get folks' thoughts.

[patrickhlauke]

Personally, I think it would make sense to have this as part of the PE spec, rather than a separate one. My very naive take would be that this is already possible/allowed by that "MAY" sentence you mention, and that it can be expanded to mention this particular scenario as a rationale for the "MAY".

I do wonder if there's privacy concerns though, and if this may need some opt-in on the part of the users (also, how long does it persist, what's a "session" in this context, etc).

But generally, I'd personally be in favour of this.

[bmathwig]

Hey @patrickhlauke

I think there is already some context around the lifetime of the pointerId in the NOTE section under pointerId. Our current line of thinking is that each pen would be mapped to an index, starting with 1, and each time that pen generated a PE the pointerId would be the same. If the document is closed or refreshed, the pointerId can be different for each pen. Behind the scenes, the implementation would map the Pen hardware ID to this index and keep track of it, limiting exposure of the unique Pen hardware ID from fingerprinting.

Do you think there is a backwards compatibility risk by doing it this way? Is there a case where someone would take a dependency on a "new-ID-per-event" system for creating unique SVG strokes? Is this use case exempt because of the verbiage:

Therefore authors cannot assume values convey any particular meaning other than an identifier for the pointer that is unique from all other active pointers

[bmathwig]

Alternative proposal to add new deviceId to PointerEvent:

partial interface PointerEvent {
    [SameObject] readonly attribute deviceId;
}

@patrickhlauke Can this thread be brought up at the next PointerEvent WG meeting to start a discussion about the best way to achieve this?

[patrickhlauke]

@bmathwig tagged this as a future idea. currently we're in the process of trying to stabilise Pointer Events Level 3 for a possible publication this year.

[RByers]

Note that there's a proposal to add this to Chromium. It doesn't really matter to Chromium process if that's done by (as planned) an independent WICG spec, some future-branch PE L4 spec, or a separate PE extensions spec as we've done in the past for new additions while we're stabilizing a version of the spec. @patrickhlauke / @mustaqahmed any preference?

[patrickhlauke]

@RByers hey stranger. I'd love to see this as part of L4 (or living standard, or however we want to set this up for future), so whichever approach would lend itself the best for that is ok with me. Maybe a WICG extension?

[RByers]

Hey man, good to have an excuse to post on PEWG again 😄. Sounds good, sounds like you are both thinking a tiny little WICG extension is simplest, so that's fine - we can move it into PE once L3 ships. FWIW I would love to see y'all move to a living standard model.

[mustaqahmed]

Great to see the revived interest here! I agree with @RByers and @patrickhlauke re the WICG path until we are done with L3.

Let me bring back an old question from @bmathwig above, which I didn't get a chance to comment before:

Do you think there is a backwards compatibility risk by doing it this way? Is there a case where someone would take a dependency on a "new-ID-per-event" system for creating unique SVG strokes?

We don't know any use-case like this, and therefore re-purposing pointerId (as per the original proposal here) sounds like an equally acceptable solution to me. As @patrickhlauke mentioned, the "MAY" in the current spec would allow this already. Moreover, this could be interpreted as a generalization of the (existing) constant pointerId of mouse. Let's discuss this in the WICG repo.

[flackr]

therefore re-purposing pointerId (as per the original proposal here) sounds like an equally acceptable solution to me

I don't think re-purposing pointerId works because pointerId is expected to remain fixed for a given pointer but the hardware limitations suggest that for many devices the hardware id is not known until after it starts moving on the surface, and as such the device id would be -1 on the event stream until the point at which the hardware id is known.

I think we should discuss this proposal again at the PEWG. It has been implemented with a draft spec PR, and a proposal to ship in chromium.

[RByers]

Yeah that property of the hardware ID not necessarily being known at pointerdown time was, IIRC, the key aspect necessitating this design. If PEWG is now able to take this up, that's great! But we also shouldn't be afraid to incubate here - I don't think the stakes are very high (see my comment on the I2S).

[flackr]

This was discussed yesterday (notes) where the consensus was that we would like to see this identifier added to a common structure where other device specific customizations could go. The rationale is to avoid having a bunch of new optional members of pointer events at the base level.

[flackr]

Another question that came up is when (i.e. for what devices / OS's) this would apply. My understanding was that we should have a deviceId for any device in which the same pointer can be known from one contact to the next. I suspect this means we may not expect a deviceId for a touch coming from a particular device since a subsequent touch may or may not be from the same "pointer" (i.e. finger), in which case it may be that we want the name to suggest something about the contact, rather than "device" which suggests something about the device which was contacted with.

[sahirv]

Following up on the conversation from the deviceId I2S.

I think we should only have a deviceId when we can uniquely identify a particular pointer.

"pointer" (i.e. finger), in which case it may be that we want the name to suggest something about the contact, rather than "device" which suggests something about the device which was contacted with.

I think excluding touch from deviceId seems reasonable, at least to me, since I would not expect my individual fingers to be identified, but I can understand the rationale behind this question. I think my vocabulary isn't big enough or an alternative name :P ... it would need to emphasize that the pointer event has the same source device though.

[flackr]

One more question, is deviceProperties.deviceId a bit redundant? Should it just be something like deviceProperties.identifier, deviceProperties.id or maybe deviceProperties.uniqueId (hinting at the purpose) or would this be too confusing?

[sahirv]

I don't mind deviceProperties.id or deviceProperties.uniqueId. id works since we already know it's a property of the device. Likewise uniqueId is great as well, and is my main preference.

[patrickhlauke]

x-ref: #495

[sahirv]

After some new concerns raised in the spec pr, it seems like this is taking the original direction of an id on the main PointerEvent interface. The id will be called persistentDeviceId.

Wondering if the PEWG is in agreement with the naming and overall shape of the API. @flackr @patrickhlauke

[patrickhlauke]

Closed via #495