Is there a reason h isn't allowed by lang-wiki.js?

[jsoref]

codeql/docs/codeql/ql-training/_static-training/slides-semmle-2/static/js/prettify/lang-wiki.js

Line 1 in 5cadd3c

PR.registerLangHandler(PR.createSimpleLexer([["pln",/^[\d\t a-gi-z\xa0]+/,null,"\t Â\xa0abcdefgijklmnopqrstuvwxyz0123456789"],["pun",/^[*=[\]^~]+/,null,"=*~^[]"]],[["lang-wiki.meta",/(?:^^|\r\n?|\n)(#[a-z]+)\b/],["lit",/^[A-Z][a-z][\da-z]+[A-Z][a-z][^\W_]+\b/],["lang-",/^{{{([\S\s]+?)}}}/],["lang-",/^`([^\n\r`]+)`/],["str",/^https?:\/\/[^\s#/?]*(?:\/[^\s#?]*)?(?:\?[^\s#]*)?(?:#\S*)?/i],["pln",/^(?:\r\n|[\S\s])[^\n\r#*=A-[^`h{~]*/]]),["wiki"]);

I looked at the history and it wasn't enlightening.

https://github.com/github/codeql/blame/faf4727a78ef3be68c9490ba51afe208d46818eb/docs/ql-documentation/ql-training-rst/_static-training/slides-semmle-2/static/js/prettify/lang-wiki.js

[jketema]

Thanks for your question. It looks like this is somewhere deep in a side deck in the docs folder, so it's probably no necessary for the slide deck and this might just have been the default contents for a lang-wiki.js file at the time it was created. Is there there some place where you see this causing problems?

[jsoref]

I'm running a spell checker against the repository and trying to determine which anomalies are interesting and which aren't.

In general runs of a-z are usually intended to be a-z even when people manage to lose a character or double a character etc. -- Sometimes it's intentional, but then, in general, my preference is for a comment or code to make it clear to the reader why the run is not actually a-z.

I picked this one because it was the simplest odd run. It does seem to more or less correspond to the a-gi-z in the earlier argument, but, again, a comment explaining what made h special would improve the readability of the content.

Mostly running the spell checker against repositories helps me identify bugs in the spell checker itself (and I've hit a couple here, including some codeql record limits). But there are of course typos in the codeql content, and once I have a vague sense of interesting (e.g. 83ab0c2) corrections, I may send them along.

[jketema]

Looking at this further: I think you need to fine tune your heuristics somewhat. This seems boilerplate code that was just committed by us, but not created by us. The reason the h is missing is because http and https (both start with an h) are special here.

[jsoref]

Indeed, it appears to come from https://github.com/googlearchive/code-prettify/blob/master/src/lang-wiki.js (which is of course a dead project), and presumably somewhere along the way the comments were stripped. Although as it happens, there wasn't even a particularly helpful comment there explaining that the omission of h was to help catch https?.

Yes, I'm likely to drop /test/ content for this repository (although I'll probably leave one or two changes...), and I sometimes drop /static/ content, as it's often sourced akin to vendor/.

For larger projects it can take me weeks to adjust the heuristics...

[jketema]

The /test/ directories can indeed be a bit tricky for the CodeQL repo, as typos might or might not be intentional. This might require some manual inspection in some cases. I would still expect most typos in those directories to be unintentional.

[jsoref]

I'm already at https://github.com/github/codeql/compare/5cadd3c0e68334ef96c70d12e743f3489377beb5..d32a2b2aed31c87d2aeb7560d54493b83a8cc2e7

I think I'll start w/ this sub-slice and see how it fares. If it's accepted, I can revisit a larger slice of the repository.

In general, I try to inspect everything before I make a PR. But my goal is to have some vague sense of what's worth touching at all first... And here, that means I need to figure out if, e.g. /static/ is reliably worth ignoring.