Use anaconda-installers preset from anaconda/renovate-config (#500)

anaconda · Apr 22, 2024 · ee04334 · ee04334
1 parent a39d8e7
commit ee04334
Show file tree

Hide file tree

Showing 7 changed files with 14 additions and 116 deletions.
diff --git a/README.md b/README.md
@@ -44,11 +44,9 @@ If an image with the same version needs to be republished, the version number sh
 
 ## Automatic updates using renovate
 
-### Preparing `Dockerfile`s for updates
-
 Docker images using the Miniconda or Anaconda Distribution installers can be updated using `renovate`.
 
-#### Updates with sha256 checks
+### Updates with sha256 checks
 
 To update Dockerfiles that contain sha256 checks, the full URL and sha256 sums need to be provided for each installer:
 
@@ -71,7 +69,7 @@ ARG INSTALLER_URL_AARCH64="<URL to linux-aarch64 installer file>"
 ARG SHA256SUM_AARCH64="<linux-aarch64 installer checksum>"
 ```
 
-#### Simple version number updates
+### Simple version number updates
 
 If not checksums are needed, the Dockerfile can be simplified:
 
@@ -81,76 +79,3 @@ ARG INSTALLER_VERSION=<installer version>
 ```
 
 `Linux-x86_64.sh` can be replaced by any installer suffix supported in the latest release.
-
-### Understanding the transform templates
-
-Since the installers are updated in a format that is not covered using pre-existing `renovate` managers, a [custom datasource](https://docs.renovatebot.com/modules/datasource/custom/) had to be created.
-It uses a transform template to convert the `.files.json` file into a format `renovate` can process.
-Unfortunately, `renovate` expects a an array of objects whereas the `.files.json` file is a nested object.
-The file needs to be processed using [JSONata](https://jsonata.org/) and stored as `transformTemplates`.
-
-Pretty-printed, the template for is the following for Miniconda:
-
-```
-{
-    "releases": [
-        $map($sift($, function($v, $k) {
-            $not($k.$contains("latest"))
-            and $v.sha256 = $lookup($, "Miniconda3-latest-{{ packageName }}").sha256
-        }).$keys(), function($v) {
-            {
-                "digest": $lookup($, $v).sha256, "version": $contains($v.$split("-")[2], /^[0-9]+$/) ? $v.$split("-")[1] & "-" & $v.$split("-")[2] : $v.$split("-")[1],
-                "changeLogUrl": "https://docs.anaconda.com/free/miniconda/miniconda-release-notes/#miniconda-" & ($contains( $v.$split("-")[2], /^[0-9]+$/) ? $v.$split("-")[1].$split("_")[1] & "-" & $v.$split("-")[2] : $v.$split("-")[1].$split("_")[1]).$replace(".", "-"),
-
-                "sourceUrl": "https://repo.anaconda.com/miniconda/" & $v, releaseTimestamp": $fromMillis($lookup($, $v).mtime * 1000, "[Y0001]-[M01]-[D01]T[H01]:[m01]", "-6")
-            }
-        }
-    )],
-    "sourceUrl": "https://repo.anaconda.com/miniconda/",
-    "homepage": "https://anaconda.com"
-}
-```
-
-and the following for Anaconda Distribution:
-
-```
-{
-    "releases": $map(
-        $sift($, function($v, $k){
-            $k.$contains("{{ packageName }}")
-        }).$keys(), function($v) {
-            {
-                "digest": $lookup($, $v).sha256,
-                "version": $contains($v.$split("-")[2], /^[0-9]+$/) ? $v.$split("-")[1] & "-" & $v.$split("-")[2] : $v.$split("-")[1],
-                "changeLogUrl": "https://docs.anaconda.com/free/anaconda/release-notes/#anaconda-" & ($contains( $v.$split("-")[2], /^[0-9]+$/) ? $v.$split("-")[1] & "-" & $v.$split("-")[2] : $v.$split("-")[1]).$replace(".", "-"),
-                "sourceUrl": "https://repo.anaconda.com/archive/" & $v,
-                "releaseTimestamp": $fromMillis($lookup($, $v).mtime * 1000, "[Y0001]-[M01]-[D01]T[H01]:[m01]", "-6")
-            }
-        }
-    ),
-    "sourceUrl": "https://repo.anaconda.com/archive/",
-    "homepage": "https://anaconda.com"
-}
-```
-
-The `$map` function creates an array of releases containing data such as the version, the digest (the sha256 sum), changelog URL, etc.
-The `$sift` function is used to filter the appropriate installer file names that are passed into `$map`.
-
-All `$sift` functions make use of the `{{ packageName}}` template value.
-`packageName` corresponds to `depName` in the regular expressions of the `matchStrings` (`depName` cannot be used in the JSONata because renovate only replaces [a few variables](https://docs.renovatebot.com/modules/datasource/custom/#usage)).
-The value of `depName` and `packageName` is the installler suffix (e.g., `Linux-x86_64.sh`)
-to capture and group installer types.
-Overloading `depName` and `packageName` this way has the drawback that by default `renovate` would create a pull request for each architecture.
-To override this behavior, the `packageRules` bundle all Miniconda/Anaconda Distribution updates into a single PR each.
-
-Since Miniconda has multiple installer variants and also a set of "latest" installers, the logic is more complex.
-The `$sift` condition only grabs the installer that is identical to the "latest" installer, i.e., `releases` will only contain the latest installer.
-JSONata flattens arrays with only one element, which is why `[]` is wrapped around the `$map` function for Miniconda, but not for Anaconda Distribution.
-
-The version string uses a ternary operator to account for the build number, which not all installers have.
-The same pattern is used in `changeLogUrl`, but `.` needs to be replaced with `-` to get the appropriate anchor. For Miniconda, the extra `$split` filters out the `py*_` prefix in the version.
-
-The timestamp is given in Central Standard Time, which is used for release dates in documentation.
-The format specifier follows the [XPATH F&O 3.1](https://www.w3.org/TR/xpath-functions-31/#date-picture-string) specification, as required by JSONata.
-
-The transform template currently produces more data than needed because it is expected to become global once it is fully tested.
diff --git a/anaconda-pkg-build/linux/Dockerfile b/anaconda-pkg-build/linux/Dockerfile
@@ -84,7 +84,7 @@ ENV LANG en_US.UTF-8
 ENV LANGUAGE en_US:en
 ENV LC_ALL en_US.UTF-8
 
-# renovate: datasource=custom.miniconda depName=Linux-x86_64.sh
+# renovate: datasource=custom.miniconda_installer depName=Linux-x86_64.sh
 ARG INSTALLER_VERSION=py312_24.3.0-0
 
 RUN curl -sSL -o /tmp/miniconda.sh \

diff --git a/anaconda-pkg-build/linux/cuda/Dockerfile b/anaconda-pkg-build/linux/cuda/Dockerfile
@@ -79,7 +79,7 @@ ENV LANG en_US.UTF-8
 ENV LANGUAGE en_US:en
 ENV LC_ALL en_US.UTF-8
 
-# renovate: datasource=custom.miniconda depName=Linux-x86_64.sh
+# renovate: datasource=custom.miniconda_installer depName=Linux-x86_64.sh
 ARG INSTALLER_VERSION=py312_24.3.0-0
 
 RUN curl -sSL -o /tmp/miniconda.sh \

diff --git a/anaconda3/amazonlinux/Dockerfile b/anaconda3/amazonlinux/Dockerfile
@@ -3,7 +3,7 @@ FROM amazonlinux:2
 ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
 ENV PATH /opt/conda/bin:$PATH
 
-# renovate: datasource=custom.anaconda
+# renovate: datasource=custom.anaconda_installer
 ARG INSTALLER_URL="https://repo.anaconda.com/archive/Anaconda3-2024.02-1-Linux-aarch64.sh"
 ARG SHA256SUM="28c5bed6fba84f418516e41640c7937514aabd55e929a8f66937c737303c7bba"
 

diff --git a/anaconda3/debian/Dockerfile b/anaconda3/debian/Dockerfile
@@ -3,13 +3,13 @@ FROM debian:bullseye-slim
 ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
 ENV PATH /opt/conda/bin:$PATH
 
-# renovate: datasource=custom.anaconda
+# renovate: datasource=custom.anaconda_installer
 ARG INSTALLER_URL_LINUX64="https://repo.anaconda.com/archive/Anaconda3-2024.02-1-Linux-x86_64.sh"
 ARG SHA256SUM_LINUX64="c536ddb7b4ba738bddbd4e581b29308cb332fa12ae3fa2cd66814bd735dff231"
-# renovate: datasource=custom.anaconda
+# renovate: datasource=custom.anaconda_installer
 ARG INSTALLER_URL_S390X="https://repo.anaconda.com/archive/Anaconda3-2024.02-1-Linux-s390x.sh"
 ARG SHA256SUM_S390X="3e2e8b17ea9a5caafd448f52e01435998b2e1ce102040a924d5bd6e05a1d735b"
-# renovate: datasource=custom.anaconda
+# renovate: datasource=custom.anaconda_installer
 ARG INSTALLER_URL_AARCH64="https://repo.anaconda.com/archive/Anaconda3-2024.02-1-Linux-aarch64.sh"
 ARG SHA256SUM_AARCH64="28c5bed6fba84f418516e41640c7937514aabd55e929a8f66937c737303c7bba"
 

diff --git a/miniconda3/debian/Dockerfile b/miniconda3/debian/Dockerfile
@@ -27,13 +27,13 @@ ENV PATH /opt/conda/bin:$PATH
 CMD [ "/bin/bash" ]
 
 # Leave these args here to better use the Docker build cache
-# renovate: datasource=custom.miniconda
+# renovate: datasource=custom.miniconda_installer
 ARG INSTALLER_URL_LINUX64="https://repo.anaconda.com/miniconda/Miniconda3-py312_24.3.0-0-Linux-x86_64.sh"
 ARG SHA256SUM_LINUX64="96a44849ff17e960eeb8877ecd9055246381c4d4f2d031263b63fa7e2e930af1"
-# renovate: datasource=custom.miniconda
+# renovate: datasource=custom.miniconda_installer
 ARG INSTALLER_URL_S390X="https://repo.anaconda.com/miniconda/Miniconda3-py312_24.3.0-0-Linux-s390x.sh"
 ARG SHA256SUM_S390X="b601cb8e3ea65a4ed1aecd96d4f3d14aca5b590b2e1ab0ec5c04c825f5c5e439"
-# renovate: datasource=custom.miniconda
+# renovate: datasource=custom.miniconda_installer
 ARG INSTALLER_URL_AARCH64="https://repo.anaconda.com/miniconda/Miniconda3-py312_24.3.0-0-Linux-aarch64.sh"
 ARG SHA256SUM_AARCH64="05f70cbc89b6caf84e22db836f7696a16b617992eb23d6102acf7651eb132365"
 

diff --git a/renovate.json b/renovate.json
@@ -1,21 +1,8 @@
 {
   "$schema": "https://docs.renovatebot.com/renovate-schema.json",
   "extends": [
-    "github>anaconda/renovate-config"
-  ],
-  "packageRules": [
-    {
-      "matchDatasources": ["custom.anaconda"],
-      "groupName": "Anaconda Distribution",
-      "commitMessageTopic": "Anaconda Distribution",
-      "separateMajorMinor": false
-    },
-    {
-      "matchDatasources": ["custom.miniconda"],
-      "groupName": "Miniconda",
-      "commitMessageTopic": "Miniconda",
-      "separateMajorMinor": false
-    }
+    "github>anaconda/renovate-config",
+    "github>anaconda/renovate-config:anaconda-installers"
   ],
   "customManagers": [
     {
@@ -29,19 +16,5 @@
       ],
       "versioningTemplate": "regex:^(py\\d+_)?(?<major>\\d+)\\.(?<minor>\\d+)(\\.(?<patch>\\d+))?(\\-(?<build>\\d+))?$"
     }
-  ],
-  "customDatasources": {
-    "miniconda": {
-      "defaultRegistryUrlTemplate": "https://repo.anaconda.com/miniconda/.files.json",
-      "transformTemplates": [
-        "{ \"releases\": [ $map($sift($, function($v, $k) { $not($k.$contains(\"latest\")) and $v.sha256 = $lookup($, \"Miniconda3-latest-{{ packageName }}\").sha256}).$keys(), function($v) { { \"digest\": $lookup($, $v).sha256, \"version\": $contains($v.$split(\"-\")[2], /^[0-9]+$/) ? $v.$split(\"-\")[1] & \"-\" & $v.$split(\"-\")[2] : $v.$split(\"-\")[1], \"changeLogUrl\": \"https://docs.anaconda.com/free/miniconda/miniconda-release-notes/#miniconda-\" & ($contains( $v.$split(\"-\")[2], /^[0-9]+$/) ? $v.$split(\"-\")[1].$split(\"_\")[1] & \"-\" & $v.$split(\"-\")[2] : $v.$split(\"-\")[1].$split(\"_\")[1]).$replace(\".\", \"-\"), \"sourceUrl\": \"https://repo.anaconda.com/miniconda/\" & $v, \"releaseTimestamp\": $fromMillis($lookup($, $v).mtime * 1000, \"[Y0001]-[M01]-[D01]T[H01]:[m01]\", \"-6\") } } ) ], \"sourceUrl\": \"https://repo.anaconda.com/miniconda/\", \"homepage\": \"https://anaconda.com\"}"
-      ]
-    },
-    "anaconda": {
-      "defaultRegistryUrlTemplate": "https://repo.anaconda.com/archive/.files.json",
-      "transformTemplates": [
-        "{ \"releases\": $map($sift($, function($v, $k){ $k.$contains(\"{{ packageName }}\") } ).$keys(), function($v) { { \"digest\": $lookup($, $v).sha256, \"version\": $contains($v.$split(\"-\")[2], /^[0-9]+$/) ? $v.$split(\"-\")[1] & \"-\" & $v.$split(\"-\")[2] : $v.$split(\"-\")[1], \"changeLogUrl\": \"https://docs.anaconda.com/free/anaconda/release-notes/#anaconda-\" & ($contains( $v.$split(\"-\")[2], /^[0-9]+$/) ? $v.$split(\"-\")[1] & \"-\" & $v.$split(\"-\")[2] : $v.$split(\"-\")[1]).$replace(\".\", \"-\"), \"sourceUrl\": \"https://repo.anaconda.com/archive/\" & $v, \"releaseTimestamp\": $fromMillis($lookup($, $v).mtime * 1000, \"[Y0001]-[M01]-[D01]T[H01]:[m01]\", \"-6\") } } ), \"sourceUrl\": \"https://repo.anaconda.com/archive/\", \"homepage\": \"https://anaconda.com\" }"
-      ]
-    }
-  }
+  ]
 }