vcmi-gym is a project which aims to create a gym-compatible reinforcement
learning environment for VCMI (the open-source recreation of
Heroes of Might & Magic III game) along with implementations of RL algorithms
and other supplementary code (orchestration, hyperparameter tuning, observability)
needed to produce VCMI combat AI models.
The project is in a late stage of active development. The gym environment is stable and AIs can be successfully trained, although the optimal NN architecture and RL algorithm for the task are yet to be discovered (a process which involves a lot of experimentation).
An RL model is currently in training, but the progress is slow due to the resource-constrained harware environment it is trained on.
And as soon the RL model is able to surpass VCMI's default "BattleAI" bot performance (even by a small fraction), the project will advance to its final stage, involving collaboration with the VCMI devs to come up with a strategy for rolling out pre-trained AI models (e.g. as a VCMI mod).
A high-level overview of the vcmi-gym project is given in the below diagram:
The W&B external component is optional. VCMI (the game itself) is required --
it is a fork of VCMI with some modifications for RL training purposes and is
managed in a separate repo, tracked as a
git submodule located
at ./vcmi.
A step-by-step setup guide can be found below:
Please refer to this document for more information about the RL environment.
Please refer to this document for information about the RL training setup and tools used in this project.
Please refer to this document for information about the Connector component.
Fellow HOMM3 AI enthusiasts are more than welcome to help with this project. There is a lot of headroom for improvement, be it in the form of NN architectures, RL algorithm implementations, hyperparameter search, reward shaping, etc. The preferred approach is to submit a Pull request, but if you have stumbled upon a bug which you can't fix yourself, submitting an issue can help me (and others) fix it. You can also help with the ongoing AI training process by "plugging in" your own piece of hardware into the mix.
Please check for existing issues and verify that your issue is not already submitted. If it is, it's highly recommended to add to that issue with your reports.
When submitting a new issue, please be as detailed as possible - OS and Python versions, what did you do, what did you expect to happen, and what actually happened.
- Find an existing issue to work on or follow "Submitting an issue" to first create one that you're also going to fix. Make sure to notify that you're working on a fix for the issue you picked.
- Branch out from latest
mainand organize your code changes there. - Commit, push to your branch and submit a PR to
main.
If you have spare hardware and would like to help with this project, please reach out to me - together, we can ponder on putting it to good use: e.g. for training new models (GPU-bound task), evaluating existing models (CPU-bound task), or creating and rebalancing new training maps (CPU and HDD-bound task).