Initial camera rotation incorrect (Chrome/Android)

[homerjam]

Description:

The initial/default camera rotation seems to be wrong since an update to Chrome/Android. As seen in the example, the camera should face the logo at the start (which it does on desktop). However on the device specified below the camera faces 180 degrees.

Side note - is there a 'look at' type method for the camera that would correct this for me in the short term?

• A-Frame Version: 0.8.2
• Platform / Device: Chrome 67.0.3396.87 / Android 8.0.0 / OnePlus 3T
• Reproducible Code Snippet or URL: https://www.homerandfarley.com/

[Xrispy7]

@homerjam I just checked your website on my Honor 6x and it seems to be working fine, unless i don't understand your issue. Can you confirm it is still a problem?

Honor 6x | Android 7.0 EMUI 5.0.2 / Google Chrome 67.0.3396.87 / aframe 0.8.2

[homerjam]

Hi @Xrispy7 thanks for testing. Yep still a problem, just to help clarify the first image is what you should see on entry to the site, the second is what you actually see...

1. Correct view:

2. Actual view (on my device):

[Xrispy7]

That's odd, I'm new to A-frame but it's possibly a device or chrome browser issue because your site loads correctly on my device.

Honor 6x | Android 7.0 EMUI 5.0.2 / Google Chrome 67.0.3396.87

[homerjam]

I suspect you are correct. I've attempted to raise an issue for OnePlus as that seems more likely. Leaving open for now until I get clarity.

[dmarcos]

Can you also try with Chrome Canary (m69) to see if it still reproduces?

[homerjam]

Same issue in Canary. I suppose this would indicate a device/sensor problem?

[homerjam]

FWIW this also happens on the A-Frame examples. I have now filed a report with OnePlus.

In the meantime if anyone has any ideas for a workaround (I can detect a OnePlus device using user agent) that'd be appreciated. I'm wondering if I can somehow offset the devices accelerometer Y rotation - not sure at what stage I need to do this though - Browser/Three.JS/WebVR Polyfill/A-Frame??

[homerjam]

I think I have tracked down where the offset needs to go: https://github.com/immersive-web/cardboard-vr-display/blob/d97e0ec3e4144bd227b498bd7afa6f6270f712d2/src/sensor-fusion/fusion-pose-sensor.js#L183

[jsantell]

Due to the sensors being incorrect on specific devices, not sure how this is something the polyfill can detect and handle, and should most likely be handled after receiving an updated pose -- can probably be handled by nudging the A-Frame camera on every frame; more info: immersive-web/webvr-polyfill#310 (comment)

[weddingdj]

The angle is still wrong in Chrome M69 when using the WebXR Device API (For Chrome M69+) origin trial. Are there any updates on this?

[jsantell]

If WebXR is enabled, WebVR will not be used, falling back to the webvr polyfill.

In terms of where in the chain (browser, three, polyfill, aframe) this should live, modifying the pose anywhere would work. The issue that is unclear is how to detect if a device is using incompatible values for the devicemotion or GenericSensor APIs. If there's a way to determine that, it could be something added to the polyfill.

[dsinni]

Okay, some really strange things I'm noticing which may be related.

The following is affecting ALL Chrome-based Android browsers (Chrome, Canary, Chrome Dev, Chrome Beta, Edge) on my Samsung Galaxy S9+:

• When tested locally (via http-server), the camera faces forward as it should.
• When the same exact code is deployed (surge.sh), the camera faces backward! What??
• Android Firefox and Samsung Internet display as they should.

Here is an demo for my 404 Page (WIP), which is affected: https://404d.surge.sh

Using latest versions of all browsers.

[dsinni]

I'm also working on an Android APK wrapper solution which is experiencing the same issue, without being deployed.

[jsantell]

That's a really strange one. Hm. It works for me on Pixel devices in Chrome on the surge link FWIW.

Some ideas, but need to know what is determining the pose on S9+ on Chrome; some links of where in the polyfill you could check some state:

• For a little while now, Generic Sensors API is used for the pose in the polyfill. Could be related to the local testing vs deployed; is it possible to have that feature disabled by default outside of localhost? Is it wrong at all rotations (landscape, portrait, other landscape)?
• If no support for generic sensors found (although Chrome does support that now, should be same source as devicemotion), the fusion pose sensor branch is used, using the devicemotion event to create a pose. Within that, there are a handful of cases where the pose is transformed based off of platform. Is any different between deployed and localhost?

My guess is something with the Generic Sensor status and permissions, maybe referenceScreen.

[dsinni]

@jsantell Aha! Well, I just noticed that the orientation does matter; the initial position is only backward when you start in portrait mode, which is pretty common on mobile, so I do still see this as an issue. Thanks for pointing that out.

I'll have to look into the links/info you sent a bit more, but I still find it perplexing that the same, exact code is behaving differently depending on from where it's served. As I mentioned, one is served from surge.sh, and the other from my laptop using http-server, live-server, etc., accessed by my S9+ via LAN.

Why on earth would they behave differently if they're both being served from a web server? That is pretty odd to me. UNLESS... hmmm... Could it be that surge.sh is https and local is not? Even still, I'd think that https would have the desired result and not the other way around.

Will have to look into this further. Any thoughts appreciated.

Thanks.

[jsantell]

If orientation affects what you see, then maybe the referenceScreen parameter in the sensor is not being picked up. See the original issue (immersive-web/cardboard-vr-display#10, PR immersive-web/cardboard-vr-display#13) for landing generic sensors, but maybe more importantly the addition of referenceScreen issue (immersive-web/cardboard-vr-display#17, commit immersive-web/cardboard-vr-display@13e4fe6) which removes the need for converting based off of screen orientation. The sensor should throw if it doesn't recognize that option AFAIK, but the orientation working OK in landscape has me suspicious.

But confirming what path the code is being executed on would be helpful -- you can also disable Generic Sensors in chrome://flags which should trigger the devicemotion path

[jsantell]

Also, check out to see if you see the same results on the cardboard vr display demo (https://immersive-web.github.io/cardboard-vr-display) and the webvr-polyfill demo (https://github.com/immersive-web/webvr-polyfill), most likely this issue can be tracked in the cardboard vr display

[dsinni]

This is still an issue, and it seems to be affected by https + portrait orientation.

https is showing incorrect orientation (usually 180°) while in portrait orientation. while http is fine in either orientation. Any idea what would cause them to display differently?

As it is now, I'm experiencing this Android Chrome. I'm on a Samsung Galaxy S9+.

Firefox and Samsung Internet appear to show the correct orientation regardless of protocol and orientation.

This has been a big hurdle in embedding into a native app, because the camera is always facing in the wrong direction.

[jsantell]

@dsinni can you repro in the cardboard-vr-display demo? Are the demos here consistent with expectations? https://intel.github.io/generic-sensor-demos/ Not sure if those demos use referenceScreen but that's something else to test as well

[dsinni]

@jsantell Yes, the cube is appearing approximately 90 degrees to the right in portrait, but fine in landscape.

There appears to be an https redirect, so I couldn't try the http version.

[dmarcos]

@dsinni @jsantell Any API that is only enabled over https is messing things up? HTTP behaves as expected if I understood from the comments

[jsantell]

@dmarcos generic sensors are only https and localhost, sounds like an issue with that (falls back to devicemotion) -- if having issues only in one orientation my guess is referenceScreen in generic sensors isn't working (but was added in chrome 65-67ish) and I'm not aware of another implementation

[dsinni]

Any API that is only enabled over https is messing things up? HTTP behaves as expected if I understood from the comments

@dmarcos Yes, from my experience, http behaves as expected. Is there any way to rectify this here, or is this a bug in a separate API?

Beside the issue presenting itself on mobile Web, this bug is hindering the possibility of effectively encapsulating A-Frame projects in native apps via Cordova, as the bug persists there. Otherwise, it seems like a pretty decent solution for distributing A-Frame apps as native, at least for Android, so it would be great if something could be done.

Appreciate all of the great work and congrats on 0.9.0! It's been much anticipated.

[DougReeder]

FWIW, on a Galaxy S6 in Chrome 72.0.3626.96, when I start in portrait orientation the first frame is correctly oriented (in my project, https://elfland-glider.surge.sh/), then the rest are 90 degrees to the left. If I start in landscape orientation, the camera is rolled 90 degrees, but recovers over a fraction of a second. This doesn't happen over HTTP, nor in Firefox, nor in Oculus Browser, Samsung Internet VR, nor on desktop.

It does look like mobile Chrome over HTTPS is using a different API to get the orientation.

[Kershawj]

After lots of testing on loads of different devices the results are so mixed, some start 180 degrees out, some -45, some 45, some appear quite random.
For example the Galaxy S7 starts with the scene pinned to it's boot gyo rotation, not current or anything useful.
I made a wrapper entity around the camera and registered a component on this which tests for 3 3D points being in the current calculated camera frustum using 'containsPoint'. If they are not in view then rotate the wrapper, keep going until those points have been in view for a few ticks and this ensures the scene always appears center on any device.