I am interested in Deep Reinforcement Learning and its application in continuous-control tasks. My research focused on improving the optimization stability of off-policy gradient-based $Q$-learning algorithms over a range of tasks and hyperparameters.

I've written two works along this research direction:

1. Stabilizing Q-Learning for Continuous Control
   David Yu-Tung Hui
   MSc Thesis, University of Montreal, 2022
   There are two topics in this thesis. First, I investigated the duality between maximizing entropy and maximizing likelihood in RL. Then, I showed that LayerNorm reduced divergence in $Q$-learning for high-dimensional continuous control.
   [.pdf] [Errata]

2. Double Gumbel Q-Learning
   David Yu-Tung Hui, Aaron Courville, Pierre-Luc Bacon
   Spotlight at NeurIPS 2023
   We showed that using function approximation in $Q$-learning introduced two heteroscedastic Gumbel noise sources. Modeling these noise sources improved by up to $2\times$ the aggregate performance of a $Q$-learning algorithm at 1M timesteps over 33 continuous control tasks.
   [.pdf] [Reviews] [Poster (.png)] [5-min talk] [1-hour seminar] [Code (GitHub)] [Errata]

See my Google Scholar for more of my research.