Captured Surface Control

[guidou]

Request for Mozilla Position on an Emerging Web Specification

• Specification title: Captured Surface Control
• Specification or proposal URL (if available): https://screen-share.github.io/captured-surface-control/
• Explainer URL (if available): https://github.com/screen-share/captured-surface-control/blob/main/README.md
• Proposal author(s) (@-mention GitHub accounts): @eladalon1983 @guidou
• Caniuse.com URL (optional):
• Bugzilla URL (optional):
• Mozillians who can provide input (optional): @jan-ivar
• WebKit standards-position: Captured Surface Control WebKit/standards-positions#388

Other information

This is still under discussion in the WebRTC WG

[jan-ivar]

Looks like this is on the WebRTC WG agenda for Tuesday. Looking forward to learning more then. cc @martinthomson

[jan-ivar]

The local problem of how a presenter interacts with their capture choice seems a convincing use case worth solving.

But if we can solve it without granting permission or risk of remote (team viewer) control, then users would have a smoother experience, and avoid a class of risks that seem unnecessary.

So I like the direction presented Tuesday for "captureWheel" where the VC app passes in an HTMLMediaElement whose user interactions (wheel/trackpad/scrollbar scrolling, pinch-zoom) translate to the captured surface — a better name maybe?

I'd explore solving page zoom the same way.

The different approach for setZoomLevel seems problematic. Transient activation-consumption (up to 4 seconds) does not fully mitigate remote control, especially when discrete zoom levels can be set. Zooming out can cause more information to be captured than the user intended; max zoom-in can confuse the user. The mere idea of remote control can scare users.

I'd consider other approaches:

1. pinch zoom
2. UX in the HTML element (or PiP), when pinch zoom isn't available
3. Inherit the VC app's Page Zoom level
4. all of the above

The remaining justification for setZoomLevel seems to be app control over button positioning, which IMHO does not justify the remote control risk, however small. I'd be open to exploring app control over button positioning and opt-in/out.

[eladalon1983]

Thanks for the feedback, @jan-ivar. Would w3c/mediacapture-surface-control#14 address these concerns?