improved process for users to update wis2box to latest release

[maaikelimper]

Based on the feedback from users during the recent WIS2 Training workshop, we should review how to improve the process of updating wis2box to the latest release

[maaikelimper]

adding the label "security" as we need a process to allow users to easily update wis2box when vulnerabilities are found.

[tomkralidis]

2024-11-21: brainstorming:

• define a process to pull updates between releases
• add a "check for updates" capability to identify new releases

Delineate update concepts:

• software updates

• content updates (configurations, tables)

• major/minor/patch

• patch releases are for:

  • bugfixes
  • configuration updates
  • tables/schemas updates

WMO FT cycle:

• 2x/year

• WIS2

• WIGOS

• BUFR/GRIB

• wis2box updates would listen on bugfix releases only (based on WMO FT)

• wis2box image stable branches

What is a patch

• artifacts
  • anything resulting from WMO FT
  • wis2box configurations
• software
  • bug fixes

Decisions

• all wis2box repositories are managed with branches:
  • main branch (master/tip)
  • stable branches (patch releases)
• in each repository, changes to stable branches can feed into a patch release via backporting
• wis2box-ctl.py script updated to include the following:
  • for a given MINOR release, compare/detect available update
  • present the user with the list of images that can be updated
  • user accepts, docker pull, wis2box-ctl restart
• any updates of this type are bound to the same MINOR release version only

[tomkralidis]

• add release management documentation
• update wis2box-ctl.py

[tomkralidis]

@maaikelimper to implement:

• process docker-compose.yml (create backup)
• for all services whose image is in the wmo-im namespace
  • compare current version against the latest minor version (git tags)
  • if latest minor version > current version, update to latest minor version and delete current version of image on host
  • save
• add clean subcommand to prune images

[maaikelimper]

feedback from my investigation on how implement this:

• ghcr.io does not support an API to request tags that does works without a personal github-token
• docker-hub does have a public API to support this

I wrote a POC on how the update command would work if we reference images from docker-hub, this is what running the 'update' command would look like:

PS C:\Users\maaik\Documents\GitHub\wis2box> python wis2box-ctl.py update
Get latest image tag for wis2box-ui
Found 1 tags for image 'wis2box-ui: ['0.5.3']'
Set wis2box-ui to 0.5.3
Get latest image tag for wis2box-webapp
Found 1 tags for image 'wis2box-webapp: ['wis2box-1.0b5']'
Set wis2box-webapp to wis2box-1.0b5
Get latest image tag for wis2box-api
Found 4 tags for image 'wis2box-api: ['1.0b8-dev', 'wis2box-1.0b5', '0.5.1-training', '0.5.1']'
Set wis2box-api to wis2box-1.0b5
Get latest image tag for wis2box-broker
Found 11 tags for image 'wis2box-broker: ['1.0b8', '1.0b8-dev', '1.0b7', '1.0b7.dev0', '1.0b6', '1.0b5', '1.0b4', '1.0b3', '1.0b2', 'test', '1.0.beta1']'
Set wis2box-broker to 1.0b8
Get latest image tag for wis2box-management
Found 13 tags for image 'wis2box-management: ['1.0b8', '1.0b8-dev', '1.0b7', '1.0b7.dev0', '1.0b6', 'bugfix', 'test', '1.0b5', '1.0b4', '1.0b3-training', '1.0b3', '1.0b2', '1.0.beta1']'  
Set wis2box-management to 1.0b8
Get latest image tag for wis2box-mqtt-metrics-collector
Found 9 tags for image 'wis2box-mqtt-metrics-collector: ['1.0b8', '1.0b8-dev', '1.0b7', '1.0b7.dev0', '1.0b6', '1.0b5', '1.0b4', '1.0b3', '1.0b2']'
Set wis2box-mqtt-metrics-collector to 1.0b8
Get latest image tag for wis2box-auth
Found 1 tags for image 'wis2box-auth: ['0.5.0']'
Set wis2box-auth to 0.5.0
Get latest image tag for wis2downloader
Found 1 tags for image 'wis2downloader: ['1.0b8-dev']'
Set wis2downloader to 1.0b8-dev

To move forward with this approach we need to ensure we have properly tagged images per release in https://hub.docker.com/u/wmoim

@tomkralidis what do you think ?