Infrastructure

AI:MMO uses Kubernetes for development and deployment. We use one kubernetes cluster for all our games and game workers. Locally, we use minikube. We use one node (i.e. our virtual machine) for all the pods, services, replication controllers and ingress.

Game creator

There is one aimmo-game-creator replication controller which controls one aimmo-game-creator pod. This is created at the time the cluster is created.

When a user creates a new game:

1. KubernetesWorkerManager (inside worker_manager.py) creates the game service called game-#{game_id}
2. The worker manager then also creates a replication controller for that game.
3. That replication controller controls one game pod which runs the code inside of the aimmo-game module.

Games and Game Workers

Games are responsible for creating and removing game-workers (which run the player's code). Each game-worker is a pod whose name is in the format aimmo-#{game_id}-worker-#{player_id}.

Game <-> Game Worker Communication

A game and it's game-workers communicate to each other via http requests using the in-cluster DNS. For example to access game-1 inside of the cluster, we can use the URL http://game-1 which will resolve to the IP address of the game-1 service.

Exposing game services via Ingress

The browser uses WebSockets to connect to a game and listen for updates. We use ingress to expose our game services via paths. Below is an example of an Ingress specification for one game.

 "rules": [
      {
        "host": "dev.aimmo.codeforlife.education",
        "http": {
          "paths": [
            {
              "path": "/game-1",
              "backend": {
                "serviceName": "game-1",
                "servicePort": 80
              }
            }
          ]
        }
      }
 ]

With these set of rules, the browser can open a WebSocket connection with dev.aimmo.codeforlife.education/game-1 in order to connect to game-1.

Updating Ingress

When a game is created or removed, we update the ingress specification/rules which are then reloaded automatically. We use a HTTP PATCH request to the aimmo-ingress to do this. For example to expose a second in the specification above we can use this PATCH request:

[
  {
    "op":"replace", 
    "path":"/spec/rules", 
    "value": {
        "rules": [{
          "host": "dev.aimmo.codeforlife.education",
          "http": {
              "paths": [
                {
                  "path": "/game-1",
                  "backend": {
                      "service_name": "game-1",
                      "service_port": 80
                  }
                },
                {
                  "path": "/game-2",
                  "backend": {
                      "service_name": "game-2",
                      "service_port": 80
                  }
                }
              ]
            }
        }]
    }
  }
]