Auto generate featured chapters

[tunetheweb]

As discussed in #1016 (comment) this allows the featured_chapter.html to be auto-generated as part of npm run generate.

I've just marked up 3 English chapters for now to show how this would work, and make reviewing easier, but would need to mark up the other chapters either as part of this, or a separate PR shortly after merging. Until then I've commented out the actual generation call, so the 20 chapter featured_chapter.json file is not overwritten with a 3 chapter one.

[rviscomi]

My first impression reviewing this is that it's strange to see these implemented in the markup and intentionally hidden from rendering, for the sole purpose of conveying metadata. That feels like the job of the YAML.

[tunetheweb]

My first impression reviewing this is that it's strange to see these implemented in the markup and intentionally hidden from rendering, for the sole purpose of conveying metadata. That feels like the job of the YAML.

OK I've move it to YAML. Having it in the chapter would be fine if the hidden was the exception, but it's very much the norm.

However, it's pretty messy:

Colons

YAML cannot contain colons so it must be changed to : or the whole quote breaks for that chapter - this is very fragile (see example in media.md). If you don't do that, then this:

featured_quote: Images...  are important for many reasons: they help...

becomes:

"featured_quote: Images...  are important for many reasons": "they help..."

instead of:

"featured_quote": "Images...  are important for many reasons: they help..."

Ampersands

Showdown escapes & to & in meta data, meaning media's third label:

Use of <code>&lt;picture&gt;</code> element

becomes:

Use of <code>&amp;lt;picture&amp;gt;</code> element

So we have to un-escape these back.

Not sure if these are the only two things but appear to be so far.

EJS

The EJS template is horrible. But there doesn't seem to be a better way of formatting it to display the featured_chapter.html nicely. We could go format the EJS template nicer, and accept that featured_chapter.html formatting would be worse but that feels just as bad if not worse to be honest.

We could also move to JSON as in #1103 which would make it a bit cleaner and also save us an extra EJS template.

Conclusion

All, in all, I just think this is too much code, too much fragility, and not enough gains. Yes the quotes and data are closer to the chapter - but they are still duplicated in YAML and far enough away from the quote itself that there is no guarantee they will be kept in sync.

I honestly think we should close this PR and just leave it the way it is, with quotes and stats hand picked from chapters and stored manually in featured_chapters.html is they are now. It works, it's a once-off effort each year, and it allows all the HTML you want, and is easily translated.

WDYT @rviscomi ?

[ibnesayeed]

You can quote the string if it contains colons:

featured_quote: "Images...  are important for many reasons: they help..."

Is equivalent to:

{
  "featured_quote": "Images...  are important for many reasons: they help..."
}

[tunetheweb]

You can quote the string if it contains colons:

featured_quote: "Images...  are important for many reasons: they help..."

Is equivalent to:

{
  "featured_quote": "Images...  are important for many reasons: they help..."
}

Unfortunately that is not the case here, as showdown does not recognise this. So that becomes:

"featured_quote: "Images... are important for many reasons": "they help...""

The problem is we're mixing too many technologies: Showdown, JavaScript, YAML, JSON, EJS, HTM and they all escape things differently.

[tunetheweb]

Any thoughts on this @rviscomi ?

It kind of works with the YAML but with the caveats on certain escaping needing to be done, so we could go with it. Or we could just close this and leave it as is. WDYT?

[rviscomi]

Sorry, not clear on the YAML-Showdown incompatibility. Does that imply that we're running YAML through the markdown compiler?

[tunetheweb]

Sorry, not clear on the YAML-Showdown incompatibility. Does that imply that we're running YAML through the markdown compiler?

Yes Showndown takes the YAML at the top and loads it into metadata object for you to do as you please. We then JSON.stringify it to an Jinja2 template variable:

almanac.httparchive.org/src/templates/base/2019/chapter.ejs.html

Line 13 in 038f145

{% set metadata = <%- JSON.stringify(metadata) %> %}

It's also not exactly YAML. See here: #1104

[rviscomi]

How feasible is it to replace the YAML with pure JSON in the markdown files? And would that help?

[tunetheweb]

How feasible is it to replace the YAML with pure JSON in the markdown files? And would that help?

I wouldn't be a fan to be honest. The YAML-ish meta data is part of Showdown and is presented simply, and so is easy to understand for authors and developers alike. It's worked well for us.

What we've got here works, with just a few caveats to be aware of. So if we really want to move these to the featured chapter quotes to the chapter files, then I think this is mergeable as is. Personally I still question the value so am still still more on the side of keeping it as is and scrapping this PR. But I also wouldn't be against merging it as is (with the other chapters moved of course) if you still feel strongly about this.