Skip to content

Quicklook ar placement #4030

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Quicklook ar placement #4030

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 23 additions & 2 deletions packages/model-viewer/src/features/ar.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -434,13 +434,34 @@ configuration or device capabilities');

updateSourceProgress(0.2);

const exporter = new USDZExporter();
const arraybuffer = await exporter.parse(model);
const usdzOptions = {
ar: {
anchoring: { type: 'plane' },
planeAnchoring: {
alignment: this.arPlacement === 'wall' ? 'vertical' : 'horizontal',
},
},
};

// necessary because quicklook internally rotates model when placing vertical.
// See https://github.com/google/model-viewer/issues/3989
if (usdzOptions.ar.planeAnchoring.alignment === 'vertical') {
model.rotateX((-90 * Math.PI) / 180);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not update the GLB; it's a good test precisely because many models are not aligned to the origin. I believe you'll need to also apply the scene.target translation to make it work.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It sounds like USDZ may have particular assumptions about where the origin should be relative to the placement point, so you may also need to make use of $scene.boundingBox to find the translation you need.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of (-90 * Math.PI) / 180 we can use -Math.PI / 2 here.

Also we have the same auto rotation issue with alignment "any" (https://developer.apple.com/documentation/realitykit/uniform-token-preliminary-planeanchoring-alignment), so maybe doing usdzOptions.ar.planeAnchoring.alignment !== 'horizontal' would be better.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you like to take over this PR or make your own version?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't understand what you were suggesting in the comments above with the translation and bounding box, and also every time I tried to do threejs operation like that translation followed by rotation I always do things wrong and lose my hairs, so I probably can't do any change related to that.
But not sure if we should go this way if removing the lines https://github.com/mrdoob/three.js/blob/aaafb4ffe8830932e31aa44003376457265d126a/examples/jsm/exporters/USDZExporter.js#L209-L210 also fixes the issue.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

related comment #3989 (comment)

model.updateMatrixWorld();
}

const exporter = new USDZExporter() as any;
const arraybuffer = await exporter.parse(model, usdzOptions);
const blob = new Blob([arraybuffer], {
type: 'model/vnd.usdz+zip',
});

const url = URL.createObjectURL(blob);

if (usdzOptions.ar.planeAnchoring.alignment === 'vertical') {
model.rotateX((90 * Math.PI) / 180);
model.updateMatrixWorld();
}

updateSourceProgress(1);

Expand Down
2 changes: 2 additions & 0 deletions packages/modelviewer.dev/examples/augmentedreality/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,8 @@ <h4>Here the Scene Viewer app is given priority, to make it easier to compare wi
<div class="heading">
<h2 class="demo-title">Placing on a Wall</h2>
<h4>This demonstrates the <code>ar-placement</code> attribute, which defaults to "floor", but using "wall" gives a different AR placement experience.</h4>
<p>QuickLook will ignore this attribute if a .usdz or .reality model is supplied through <code>ios-src</code> and instead look at the <code>planeAnchoring:alignment</code> option
inside the USDZ file.</p>
</div>
<example-snippet stamp-to="wall" highlight-as="html">
<template>
Expand Down