Can't zoom after switching between cameras

[vinkovsky]

Describe the bug

After dynamically updating the camera from perspective to orthographic, zooming in orthographic mode will not work.

To Reproduce

Steps to reproduce the behavior:

1. Go to this demo
2. Click on the top edge of the view cube or rotate the camera to position it above the scene cubes.
3. Your camera now switches to ortho mode.
4. Try zooming in. You will not make it
5. Go to camera-view.tsx to see the code

Code

class Controls {
  perspectiveCamera: PerspectiveCameraImpl;
  orthographicCamera: OrthographicCameraImpl;
  currentCamera: PerspectiveCameraImpl | OrthographicCameraImpl;
  object: CameraControlsImpl;

  constructor(private container: HTMLElement, private scene: Scene) {
    this.perspectiveCamera = new PerspectiveCameraImpl(
      70,
      this.container.clientWidth / this.container.clientHeight,
      0.1,
      4000
    );
    this.orthographicCamera = new OrthographicCameraImpl(
      this.container.clientWidth / -2,
      this.container.clientWidth / 2,
      this.container.clientHeight / 2,
      this.container.clientHeight / -2,
      0.1,
      4000
    );
    this.currentCamera = this.perspectiveCamera;

    const subsetOfTHREE = {
      Vector2: Vector2,
      Vector3: Vector3,
      Vector4: Vector4,
      Quaternion: Quaternion,
      Matrix4: Matrix4,
      Spherical: Spherical,
      Box3: Box3,
      Sphere: Sphere,
      Raycaster: Raycaster,
    };

    CameraControlsImpl.install({ THREE: subsetOfTHREE });

    this.object = new CameraControlsImpl(this.currentCamera, this.container);
    this.object.setPosition(1, 1, 1);
    this.object.setTarget(0, 0, 0);
    this.object.restThreshold = 0.1;
    this.object.dollyToCursor = true;
  }

  dispose() {
    this.object.disconnect();
    this.object.dispose();
  }

  update(delta: number) {
    if (this.object.polarAngle <= 0.01) {
      if (this.currentCamera.type === "PerspectiveCamera") {
        this.setOrthographicCamera();
      }
    } else if (this.currentCamera.type === "OrthographicCamera") {
      this.setPerspectiveCamera();
    }

    this.object.update(delta);
  }

  private updateOrthographicCameraFrustum() {
    const { distance } = this.object;

    const halfWidth =
      frustumWidthAtDistance(this.perspectiveCamera, distance) / 2;
    const halfHeight =
      frustumHeightAtDistance(this.perspectiveCamera, distance) / 2;
    const halfSize = { x: halfWidth, y: halfHeight };
    this.orthographicCamera.top = halfSize.y;
    this.orthographicCamera.bottom = -halfSize.y;
    this.orthographicCamera.left = -halfSize.x;
    this.orthographicCamera.right = halfSize.x;
  }

  updateFrustum() {
    this.perspectiveCamera.aspect =
      this.container.clientWidth / this.container.clientHeight;
    this.updateOrthographicCameraFrustum();
    this.currentCamera.updateProjectionMatrix();
  }

  setOrthographicCamera() {
    this.orthographicCamera.position.copy(this.perspectiveCamera.position);
    this.updateOrthographicCameraFrustum();
    this.currentCamera = this.orthographicCamera;
    this.object.camera = this.orthographicCamera;
    this.orthographicCamera.updateProjectionMatrix();
  }

  setPerspectiveCamera() {
    const oldY = this.perspectiveCamera.position.y;
    this.perspectiveCamera.position.copy(this.orthographicCamera.position);
    this.perspectiveCamera.position.y = oldY / this.orthographicCamera.zoom;
    this.currentCamera = this.perspectiveCamera;
    this.object.camera = this.perspectiveCamera;
    this.perspectiveCamera.updateProjectionMatrix();
  }
}

function frustumHeightAtDistance(
  camera: PerspectiveCameraImpl,
  distance: number
) {
  const vFov = (camera.fov * Math.PI) / 180;
  return Math.tan(vFov / 2) * distance * 2;
}

function frustumWidthAtDistance(
  camera: PerspectiveCameraImpl,
  distance: number
) {
  return frustumHeightAtDistance(camera, distance) * camera.aspect;
}

Live example

demo

codesandbox

repo

Expected behavior

Zoom must work in ortho mode

Screenshots or Video

Screen.Recording.2024-04-22.at.00.31.43.mov

Device

Desktop

OS

MacOS

Browser

Chrome

[Muralitob]

same bug

[seanhouli]

I needed this functionality as well, I took your demo and got it working: https://codesandbox.io/p/github/vinkovsky/switching-cameras-demo/csb-jzt6zp/draft/gallant-cannon?file=%2Fapp%2Fcamera-view.tsx. I commented out the automatic switching and just bound to 't' to toggle between to make debugging easier. its pretty satisfying to use 😄 . I also am using the getPosition() from camera-controls rather than .position as it's my understanding that's more correct.

[vinkovsky]

@seanhouli Hello! Thanks for your reply. Unfortunately couldn't open your solution :(

[seanhouli]

@seanhouli Hello! Thanks for your reply. Unfortunately couldn't open your solution :(

Can you try again? I have not used codesandbox a ton, but under sharing it does say it's publicly accessible.

Also, upon playing with it more, there does seem to be issues with the swapping code itself giving strange results when you stress test it, probably because camera-controls has its own quirks

edit: sent a friend the same link and could open it just fine

[vinkovsky]

@seanhouli still not working

[seanhouli]

All I can do is share the link, which I have done 🤷

[seanhouli]

@vinkovsky Here are the contents of the camera-view.tsx file if you still cannot use the link.

import { useEffect, useMemo } from "react";
import { useFrame, useThree } from "@react-three/fiber";
import {
  PerspectiveCamera as PerspectiveCameraImpl,
  OrthographicCamera as OrthographicCameraImpl,
  Vector3,
  Vector2,
  Vector4,
  Quaternion,
  Matrix4,
  Spherical,
  Box3,
  Raycaster,
  Sphere,
  EventDispatcher,
  Scene,
} from "three";
import CameraControlsImpl from "camera-controls";

export function Camera() {
  const gl = useThree((state) => state.gl);

  const set = useThree((state) => state.set);
  const get = useThree((state) => state.get);
  const scene = useThree((state) => state.scene);

  const controls = useMemo(
    () => new Controls(gl.domElement, scene),
    [gl.domElement, scene],
  );

  useEffect(() => {
    const onkeydown = (e: KeyboardEvent) => {
      console.log(e.key);
      if (e.key.toLowerCase() === "t") {
        controls.currentCamera === controls.perspectiveCamera
          ? controls.setOrthographicCamera()
          : controls.setPerspectiveCamera();
      }
    };

    window.addEventListener("keydown", onkeydown);

    return () => {
      window.removeEventListener("keydown", onkeydown);
    };
  }, [controls]);

  useEffect(() => {
    if (scene.children.length) {
      fitCameraToSceneBoundingSphere(controls.object, scene);
    }

    console.log("scene.children.length", scene.children.length);
  }, [controls.object, scene.children]);

  useFrame((_, delta) => {
    controls.update(delta);
    gl.render(scene, controls.currentCamera);
  }, -1);

  useEffect(() => {
    const oldControls = get().controls;
    const oldCamera = get().camera;
    set({
      controls: controls.object as unknown as EventDispatcher,
      camera: controls.currentCamera,
    });
    return () => set({ controls: oldControls, camera: oldCamera });
  }, [controls, get, set]);

  return (
    <>
      <primitive object={controls.object} />
      <primitive object={controls.currentCamera} />
    </>
  );
}

// https://gist.github.com/nickyvanurk/9ac33a6aff7dd7bd5cd5b8a20d4db0dc
class Controls {
  perspectiveCamera: PerspectiveCameraImpl;
  orthographicCamera: OrthographicCameraImpl;
  currentCamera: PerspectiveCameraImpl | OrthographicCameraImpl;
  object: CameraControlsImpl;

  constructor(
    private container: HTMLElement,
    private scene: Scene,
  ) {
    this.perspectiveCamera = new PerspectiveCameraImpl(
      70,
      this.container.clientWidth / this.container.clientHeight,
      0.1,
      4000,
    );
    this.orthographicCamera = new OrthographicCameraImpl(
      this.container.clientWidth / -2,
      this.container.clientWidth / 2,
      this.container.clientHeight / 2,
      this.container.clientHeight / -2,
      0.1,
      4000,
    );
    this.currentCamera = this.perspectiveCamera;

    const subsetOfTHREE = {
      Vector2: Vector2,
      Vector3: Vector3,
      Vector4: Vector4,
      Quaternion: Quaternion,
      Matrix4: Matrix4,
      Spherical: Spherical,
      Box3: Box3,
      Sphere: Sphere,
      Raycaster: Raycaster,
    };

    CameraControlsImpl.install({ THREE: subsetOfTHREE });

    this.object = new CameraControlsImpl(this.currentCamera, this.container);
    this.object.setPosition(1, 1, 1);
    this.object.setTarget(0, 0, 0);
    this.object.restThreshold = 0.1;
    this.object.dollyToCursor = true;
  }

  dispose() {
    this.object.disconnect();
    this.object.dispose();
  }

  update(delta: number) {
    // if (this.object.polarAngle <= 0.01) {
    //   if (this.currentCamera.type === "PerspectiveCamera") {
    //     this.setOrthographicCamera();
    //   }
    // } else if (this.currentCamera.type === "OrthographicCamera") {
    //   this.setPerspectiveCamera();
    // }

    this.object.update(delta);
  }

  private updateOrthographicCameraFrustum() {
    const { distance } = this.object;

    const halfWidth =
      frustumWidthAtDistance(this.perspectiveCamera, distance) / 2;
    const halfHeight =
      frustumHeightAtDistance(this.perspectiveCamera, distance) / 2;
    const halfSize = { x: halfWidth, y: halfHeight };
    this.orthographicCamera.top = halfSize.y;
    this.orthographicCamera.bottom = -halfSize.y;
    this.orthographicCamera.left = -halfSize.x;
    this.orthographicCamera.right = halfSize.x;
  }

  updateFrustum() {
    this.perspectiveCamera.aspect =
      this.container.clientWidth / this.container.clientHeight;
    this.updateOrthographicCameraFrustum();
    this.currentCamera.updateProjectionMatrix();
  }

  private lastPerspectivePosition: Vector3 = new Vector3();

  setOrthographicCamera() {
    const perspectivePos = this.object.getPosition(new Vector3());
    this.updateOrthographicCameraFrustum();
    this.currentCamera = this.orthographicCamera;
    this.object.camera = this.currentCamera;
    this.object.setPosition(
      perspectivePos.x,
      perspectivePos.y,
      perspectivePos.z,
    );
    this.orthographicCamera.updateProjectionMatrix();

    this.lastPerspectivePosition.copy(perspectivePos);

    this.object.mouseButtons.wheel = CameraControlsImpl.ACTION.ZOOM;
  }

  setPerspectiveCamera() {
    const orthoPos = this.object.getPosition(new Vector3());

    const oldY = this.lastPerspectivePosition.y;
    this.currentCamera = this.perspectiveCamera;
    this.perspectiveCamera.updateProjectionMatrix();
    this.object.camera = this.currentCamera;
    this.object.setPosition(
      orthoPos.x,
      oldY / this.orthographicCamera.zoom,
      orthoPos.z,
    );

    this.object.mouseButtons.wheel = CameraControlsImpl.ACTION.DOLLY;
  }
}

function frustumHeightAtDistance(
  camera: PerspectiveCameraImpl,
  distance: number,
) {
  const vFov = (camera.fov * Math.PI) / 180;
  return Math.tan(vFov / 2) * distance * 2;
}

function frustumWidthAtDistance(
  camera: PerspectiveCameraImpl,
  distance: number,
) {
  return frustumHeightAtDistance(camera, distance) * camera.aspect;
}

export const fitCameraToSceneBoundingSphere = (
  controls: CameraControlsImpl,
  scene: Scene,
) => {
  const boundingBox = new Box3();
  const center = new Vector3();
  const sphere = new Sphere();

  const defaultRadius = 1;
  const bbox = boundingBox.setFromObject(scene);

  if (controls.fitToSphere) {
    // https://stackoverflow.com/a/63243915/11416728
    controls.fitToSphere(
      bbox.getBoundingSphere(sphere.set(bbox.getCenter(center), defaultRadius)),
      true,
    );
  }
};

[seanhouli]

@vinkovsky it seems that reverting back to your original version that uses .position is actually working better for me, no idea why. Here are the changes I have made

  setOrthographicCamera() {
    this.object.cancel()
    // .... same as yours
    this.object.mouseButtons.wheel = CameraControlsImpl.ACTION.ZOOM
  }

  setPerspectiveCamera() {
    this.object.cancel()
    // .... same as yours
    this.object.mouseButtons.wheel = CameraControlsImpl.ACTION.DOLLY
  }

still seeing weirdness when stress-testing hard and swapping back and forth, panning, orbiting aggressively

[vinkovsky]

@seanhouli Thanks for the code snippet! I appreciate it. Yes, I also noticed differences in behavior between .position and .getPosition(). Thats why i used the first one

[Muralitob]

@seanhouli genius！ it work！