[Proposal] Discrete observations

[sparisi]

Proposal

Have a wrapper to make environments fully discrete.

Motivation

Currently, obs spaces are Box. It would be great to have versions of the env with fixed grids and Discrete spaces to test provably efficient algorithms in discrete MDPs.

Pitch

MiniGrids are amazing to test exploration algorithms, but they cannot be used to tabular algorithms. With wrappers or versions of the env that are fully discrete, early-stage of algorithms can be tested. Say I develop a novel version of Q-Learning for exploration. I could first test it with tabular Q-Learning, and then move to DQN.

Checklist

• I have checked that there is no similar issue in the repo (required)