How should we generate metrics-catalog.html

[pritikin]

The current model is to combine data/front-matter.md and data/primary-dataset.yml using the tools/metrics_validator.rb. The ruby script leverages ‘redcarpet’ to process the markdown and nokogiri to manipulate the html document in memory. The final step is to write the doc to a file.

@rajkrishnamurthy has submitted a PR to discuss an alternate approach. The proposed alternative is to combine a data/metrics-catalog.template with primary-dataset.yml using ERB, “a templating system” often used when generating HTML.

(Commit of the generated html is handled by .github/workflows/actions.yml

It felt pre-mature to discuss in the PR so I created this issue thread to capture my thoughts.

Tagging @apannetrat as well as he wrote the original.

[pritikin]

I’m agnostic concerning the choice of generation toolchain. @rajkrishnamurthy what benefits do you see in ERB vs the existing redcarpet/nokogiri chain? Can you provide a specific example re your comment that “The current implementation uses custom html generation based on our own tag parsing. ERB provides an alternative option, and possibly cleaner” ?

[rajkrishnamurthy]

@pritikin @apannetrat
ERB does much of the DOM work for us without us having to explicitly code for generating the html file. The template allows us to maintain the html structure easier, and can include the static content such as the front-matter. The action workflow can remain intact, generating the html automatically and committed using test-room-7/action-update-file plugin. This replaces nokogiri however retains redcarpet for markdown to html translation.

In summary;

1. Easier to read and maintain
2. Less code
3. No direct DOM manipulation
4. Avoid vulnerabilities with nokogiri

PS: traveling this week and delayed in my responses.

[apannetrat]

I don't have a strong preference between the two approaches. I see pros and cons with both.

A couple of points:

• The Nokigiri issue is now fixed; there is no vulnerability.
• If using ERB, the font-matter.md file should remain separate (not inserted into the ERB template) because it should be easily editable by anyone.
• The current approach seems to make it easier to perform schema validation for our YAML data (see YAML Syntax Errors in Continuous Audit Metrics #57 (comment)).

[pritikin]

I propose we close this thread. @rajkrishnamurthy provided a good suggestion but I don't see this as a blocking issue currently.