This repository explores the task of selecting an optimal autoencoder architecture using Bayesian optimization. An autoencoder is a differentiable model composed of two main components:
- Encoder: Transforms the input data into a latent vector representation.
- Decoder: Reconstructs the original input from the latent representation.
The structure of the autoencoder is defined by a set of hyperparameters, which are optimized using a novel two-stage Bayesian optimization approach.
We introduce a modified Bayesian optimization technique with the following key steps:
- Candidate Selection: At each iteration, a set of points with the best model quality estimates is chosen.
- Dynamic Learning Analysis: The best candidate is selected based on training dynamics and performance.
The approach is theoretically justified and experimentally validated on the CIFAR and Fashion-MNIST datasets, demonstrating improved efficiency in architecture selection.
- Bayesian Optimization: Efficient hyperparameter tuning for autoencoder architectures.
- Two-Stage Selection: Combines quality estimation with training dynamics for better convergence.
- Theoretical & Empirical Validation: Supported by both theoretical analysis and experiments.
- CIFAR
- Fashion-MNIST