The goal of the project is to create a program that can detect a target and place virtual objects in the scene relative to the target, accounting for camera and target movement. The process involves calibrating the camera using a chessboard image to determine distance and then generating virtual objects on the chessboard. The virtual objects, including pyramids and a torus (donut), are projected onto the image plane in real time based on the movement of the camera or target. In addition to the chessboard, I have also used ArUco markers and overlayed images/videos on top of it.
I created three slanted pyramids with the top connected with a green-colored roof.
I overlaid images onto those ArUco marks:
I can even rotate the ArUco marks; the images will be rotated accordingly!
Please find a screen recording here: https://drive.google.com/file/d/1ZGoQKcxPYRRTgvMnKS7Re-LlHhK_q9xV/view?usp=drive_link
Run augmentedReality.cpp
You should start by calibrating your camera with a chessboard. This can be done by pressing "d", and then "s" at least 5 times (each from diff angle of the chessboard).
Some useful hotkeys:
- n = Do nothing/ reset the camera frames
- s = Save the current frame for camera calibration. Need to press 'd' before pressing 's'. Need to save at least 5 images, and then calibration will happen by itself.
- q = Quit program
- d = Detect chessboard corners and display it
- p = Calculate the Camera's current position and print out the rotation and translation vectors.
- a = Overlay images onto ArUco markers found. Do nothing if no markers are found.
- v = Overlay videos onto ArUco markers found. Do nothing if no markers are found.
- o = Draw virtual objects (Three pyramids with the top connected with a "roof") onto the chessboard
- f = Find and display FAST features onto the frame
- h = Find and display Harris corners features onto the frame
- x = Display the 3D axes on the chessboard attached to the origin
- t = Draw a Torus, aka donut, onto the chessboard.
OS: MacOS Ventura 13.0.1 (22A400)
IDE: XCode
- Calibration: https://docs.opencv.org/4.x/d4/d94/tutorial_camera_calibration.html
- Torus : https://www.vedantu.com/maths/torus
- Harris Corner: https://docs.opencv.org/3.4/d4/d7d/tutorial_harris_detector.html
- FAST Algorithm: https://blog.francium.tech/feature-detection-and-matching-with-opencv-5fd2394a590
- ArUco:
- https://stackoverflow.com/questions/67423382/overlay-filled-semi-transparent-contours/67426795#67426795