feat: add tech_stack to the Canister Metadata Standard (#3670)

* wip * reuse run_command to support version_command * e2e tests in metadata.bash * rename to tech_stack * command can have pips/redirections * add docs pages * error cases coverage * improve the doc * changelog * fix * turn off ui_test on ubuntu-20.04 * fix run_command * remove dfx_patchelf * fix e2e on ubuntu * redesign * update docs * re-enable ui_test * fix shellcheck * revert * schema * fix jq * execute command can handle file name with whitespace * cover local command with whitespace * add an argument to guarantee `sh -c` execution * words * fix error_context * add bitcoin addree for illustration * improve words * update docs for the final design * impl the final design * update schema * set default tech_stack for rust/motoko * set tech_stack in azle/kybra templates * improve docs * fix e2e * refactor: categories are visible in schema * missing comment doc * minor
dfinity · Apr 9, 2024 · 321f404 · 321f404
1 parent 0a1d469
commit 321f404
Show file tree

Hide file tree

Showing 20 changed files with 676 additions and 72 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,6 +2,12 @@
 
 # UNRELEASED
 
+### feat: add tech_stack to the Canister Metadata Standard
+
+The standardized `dfx` metadata is extended with another object: `tech_stack`.
+
+Please check [tech-stack](docs/concepts/tech-stack.md) for more details.
+
 ### chore: updated management canister .did file
 
 ### feat: added `dfx completion` command

diff --git a/docs/concepts/canister-metadata.md b/docs/concepts/canister-metadata.md
@@ -95,9 +95,14 @@ For Motoko canisters, if you specify a `path` for candid:service metadata (repla
 
 ### `dfx`
 
-A json text for `dfx` usage.
+A json text for `dfx` usage. It consists of independent optional objects.
 
-Currently it contains the [`pullable`](pull-dependencies.md#service-provider-workflow) object.
+#### `pullable`
+
+The `pullable` object is necessary for the canister to be pullable.
+
+Check [pull-dependencies](pull-dependencies.md#service-provider-workflow) for more details of each field
+and how to set it in `dfx.json`.
 
 ```json
 {
@@ -112,6 +117,41 @@ Currently it contains the [`pullable`](pull-dependencies.md#service-provider-wor
 }
 ```
 
+#### `tech_stack`
+
+The `tech_stack` object provides a standard format to show the technologies involved to build the canister.
+
+Check [tech-stack](tech-stack.md) for more details and how to set it in `dfx.json`.
+
+```json
+{
+  "tech_stack": {
+    "language": {
+      "rust": {
+        "version": "1.75.0"
+      }
+    },
+    "cdk": {
+      "ic-cdk": {
+        "version": "0.13.0"
+      }
+    },
+    "lib": {
+      "ic-cdk-timers": {},
+      "ic-stable-structures": {}
+    },
+    "other": {
+      "bitcoin": {
+        "address": "bcrt1qfe264m0ycx2vcqvqyhs0gpxk6tw8ug6hqeps2d"
+      }
+    },
+    "tool": {
+      "dfx": {}
+    }
+  }
+}
+```
+
 ## A more complex example
 
 In this example, we change the visibility of the `candid:service` metadata on the ic and staging networks to private, but leave it public for the local network.

diff --git a/docs/concepts/tech-stack.md b/docs/concepts/tech-stack.md
@@ -0,0 +1,116 @@
+# Tech Stack
+
+## Overview
+
+Canister authors can opt in to display the tech stack of the canister.
+
+Providing a standard format of such information makes it easier to build tools like a Canister Explorer.
+
+## JSON schema
+
+`tech_stack` has 5 top-level optional categories.
+
+- cdk
+- language
+- lib
+- tool
+- other
+
+Each category is a map keyed by the names of tech stack items, where each value is a map containing optional fields.
+
+### Example
+
+```json
+{
+  "tech_stack": {
+    "language": {
+      "rust": {
+        "version": "1.75.0"
+      }
+    },
+    "cdk": {
+      "ic-cdk": {
+        "version": "0.13.0"
+      }
+    },
+    "lib": {
+      "ic-cdk-timers": {},
+      "ic-stable-structures": {}
+    },
+    "other": {
+      "bitcoin": {
+        "address": "bcrt1qfe264m0ycx2vcqvqyhs0gpxk6tw8ug6hqeps2d"
+      }
+    },
+    "tool": {
+      "dfx": {}
+    }
+  }
+}
+```
+
+## Configuration in `dfx.json`
+
+While the only way to configure this is in `dfx.json`, we don't envision that canister developers will define these values in `dfx.json` by themselves.
+
+Upcoming work will enable CDK providers to set `tech_stack` fields for their users. Please check the Q&A below for more explanation.
+
+The example above was generated from the `dfx.json` configuration below.
+
+```json
+{
+  "canisters": {
+    "canister_foo": {
+      "type": "custom",
+      "tech_stack": {
+        "cdk": {
+          "ic-cdk": {
+            "version": "0.13.0"
+          }
+        },
+        "language": {
+          "rust": {
+            "version": "$(rustc --version | cut -d ' ' -f 2)"
+          }
+        },
+        "lib": {
+          "ic-cdk-timers": {},
+          "ic-stable-structures": {}
+        },
+        "tool": {
+          "dfx": {}
+        },
+        "other": {
+          "bitcoin": {
+            "address": "bcrt1qfe264m0ycx2vcqvqyhs0gpxk6tw8ug6hqeps2d"
+          }
+        }
+      }
+    }
+  }
+}
+```
+
+The `"tech_stack"` object in `dfx.json` is almost the same as the generated metadata.
+
+The only difference is that the `language->rust->version` field is `"$(rustc --version | cut -d ' ' -f 2)"` instead of `"1.75.0"`.
+
+Besides directly setting the value of custom fields, it's also possible to obtain the value by executing a command.
+
+If the content of a custom field value begins with the prefix `$(` and ends with the postfix `)`, the inner text will be interpreted as a command.
+
+- The command will be executed in the workspace root directory, which contains the `dfx.json` file.
+- The stdout should be a valid UTF-8 string.
+- The field value will be obtained by trimming the stdout, removing any leading and trailing whitespace.
+
+## Q&A
+
+### Who should set `tech_stack`?
+
+In the near future, CDK will be able to set `tech_stack` without requiring extra configuration in `dfx.json`.
+
+Currently, `dfx` sets `tech_stack` for Rust and Motoko canisters if they don't define `tech_stack` explicitly in `dfx.json`.
+
+For Azle and Kybra projects created with `dfx new`, the corresponding `tech_stack` configuration will be added `dfx.json` by default.
+
+Canister developers can always add/overwrite/remove the `tech_stack` fields set by CDK.
diff --git a/docs/dfx-json-schema.json b/docs/dfx-json-schema.json
@@ -478,6 +478,19 @@
             "string",
             "null"
           ]
+        },
+        "tech_stack": {
+          "title": "Tech Stack",
+          "description": "Defines the tech stack used to build this canister.",
+          "default": null,
+          "anyOf": [
+            {
+              "$ref": "#/definitions/TechStack"
+            },
+            {
+              "type": "null"
+            }
+          ]
         }
       }
     },
@@ -1067,6 +1080,78 @@
         }
       ]
     },
+    "TechStack": {
+      "title": "Tech Stack",
+      "description": "The tech stack used to build a canister.",
+      "type": "object",
+      "properties": {
+        "cdk": {
+          "title": "cdk",
+          "type": [
+            "object",
+            "null"
+          ],
+          "additionalProperties": {
+            "type": "object",
+            "additionalProperties": {
+              "type": "string"
+            }
+          }
+        },
+        "language": {
+          "title": "language",
+          "type": [
+            "object",
+            "null"
+          ],
+          "additionalProperties": {
+            "type": "object",
+            "additionalProperties": {
+              "type": "string"
+            }
+          }
+        },
+        "lib": {
+          "title": "lib",
+          "type": [
+            "object",
+            "null"
+          ],
+          "additionalProperties": {
+            "type": "object",
+            "additionalProperties": {
+              "type": "string"
+            }
+          }
+        },
+        "other": {
+          "title": "other",
+          "type": [
+            "object",
+            "null"
+          ],
+          "additionalProperties": {
+            "type": "object",
+            "additionalProperties": {
+              "type": "string"
+            }
+          }
+        },
+        "tool": {
+          "title": "tool",
+          "type": [
+            "object",
+            "null"
+          ],
+          "additionalProperties": {
+            "type": "object",
+            "additionalProperties": {
+              "type": "string"
+            }
+          }
+        }
+      }
+    },
     "WasmOptLevel": {
       "title": "Wasm Optimization Levels",
       "description": "Wasm optimization levels that are passed to `wasm-opt`. \"cycles\" defaults to O3, \"size\" defaults to Oz. O4 through O0 focus on performance (with O0 performing no optimizations), and Oz and Os focus on reducing binary size, where Oz is more aggressive than Os. O3 and Oz empirically give best cycle savings and code size savings respectively.",

diff --git a/e2e/assets/metadata/tech_stack/a custom script.sh b/e2e/assets/metadata/tech_stack/a custom script.sh
@@ -0,0 +1,2 @@
+#!/usr/bin/env bash
+echo 1.75.0
diff --git a/e2e/assets/metadata/tech_stack/a.did b/e2e/assets/metadata/tech_stack/a.did
@@ -0,0 +1 @@
+service: {}