Projects for the course CMSC733 - Classical and Deep Learning Approaches for Geometric Computer Vision
This was an introductory project for the course and was divided into two phases:
Boundary detection is an important, well-studied computer vision problem. Clearly it would be nice to have algorithms which know where one object transitions to another. But boundary detection from a single image is fundamentally diffcult. Determining boundaries could require object-specific reasoning, arguably making the task hard. A simple method to find boundaries is to look for intensity discontinuities in the image, also known of edges.
In this homework, a simplified version of probability of boundary detection algorithm was developed, which finds boundaries by examining brightness, color, and texture information across multiple scales (different sizes of objects/image). The output of the algorithm is a per-pixel probability of the boundary detected. The simplified algorithm performs much better when compared to classical edge detection algorithms like Canny and Sobel. The original image and the output of the implemented pipeline is shown below:
The implementation detials and results of the homework can be found here and instructions on how to run the code are given here.
For phase 2 of this homework, multiple neural network architectures were implemented. Various criterion like number of parameters, train and test set accuracies were compared for each network architecture and detailed analysis of why one architecture works better than another one was provided. The details about implementation and the results are provided in the submitted report.
Estimating parameters of the camera like the focal length, distortion coefficients and principle point is called Camera Calibration. It is one of the most time consuming and important part of any computer vision research involving 3D geometry. An automatic way to perform efficient and robust camera calibration was presented by Zhengyou Zhang of Microsoft in this paper and is regarded as one of the hallmark papers in camera calibration. A pipeline based on Zhang's algorithm, which minimizes reprojection error to estimate the intrinsic camera matrix and the distortion coefficients, was implemented as a part of the project. The implementation details and results can be found here and the code is available here.
Homography between 2 image frames is defined as the projective transformation between these frames and is one of the very key concepts in Computer Vision. In this project we used Homography to warp multiple images and stitch a panorama using three different techniques - Traditional approach using feature matching and RANSAC, Supervised approach to predict a 4 point parametrization of Homography between two images and an Unsupervised approach to predict Homography without the presence of a ground truth. Some of the output panoramas generated with the traditional approach are shown below:
The implementation details and results can be found here and the code is available here
The aim of this project was to implement an end-to-end pipeline to swap faces in a video just like Snapchat’s face swap filter or this face swap website. It’s a fairly complicated procedure and variants of the approach we've implement have been used in many movies.
Note that, you need to be aware of ethical issues while replacing faces. Similar methods have been used by people for the creation of fake videos of celibrities called Deep Fakes. Three pipelines were implemented:
- Using Delaunay Triangulation
- Using Thin Plate Splines
- Using a deep learning approach: Joint 3D face reconstruction and dense alignment with position map regression network (link)
The details of the implementation and results are available here and the code is available here.
The comparison videos for the three pipelines on one of the data sets can be seen here:
