Update TextMate bundle for the good of humanity

[cannikin]

https://github.com/textmate/toml.tmbundle

This bundle is used by GitHub's own Linguist for syntax highlighting TOML. It's pretty rudimentary. Let's update it to be v1.0 compliant.

Time to brush up on your regexes!

[cannikin]

I was able to update HighlightJS's TOML highlighter (originally shared with ini files) to be more accurate. Not sure how well the grammar syntax matches to TextMate's:

https://github.com/toml-lang/toml.io/blob/wip/code/javascripts/highlight/languages/toml.js

[workingjubilee]

I'm digging into the syntax now.

[cannikin]

@workingjubilee Any progress on this one?

[workingjubilee]

Yes, I should be able to turn in a PR today, actually, I just had a few sessions of looong stares into the abyss (well, the internals of text editors) to get things properly tested, first.

…

[workingjubilee]

My current status: "today" was obviously optimistic (I had ground things down to "eh, should only be an hour or two to get done, right?" hahaha). I got a bit jammed on fixing some issues, but they will get done. My current lead is on a malformation in the key_pair rules.

As far as I can tell, the problem with the datetime was never that dates were not represented in the syntax, but that there was a bug in the syntax's parsing rules. So the good news is the syntax actually already is ready to parse and correctly highlight datetimes! The bad news is that what is stopping it is a parsing bug which has been much more tricky to isolate and slowly peel out, as opposed to just "add a thing".

[eksortso]

@workingjubilee, Hi, how's progress going on this? Any recent news?

[pyfisch]

I submitted a PR: textmate/toml.tmbundle/pull/14

It resolves all issues with the highlighter I am aware of and makes it v1.0 compliant. If you have complex or difficult to parse TOML documents please test the parser and let me know of wrong highlighting.

[marzer]

Per this comment, it seems as though the toml.tmbundle repo (and the other textmate tmbundle repositories) are borderline-abandonware at this point (there's very little PR reviewing going on, with many waiting unactioned for months, sometimes years now).

Mercifully it seems pretty easy to redirect github's Linguist to use a different grammar; they outline how to do this in their contributor guidelines. Might be a good time for the toml-lang org to take ownership of this with a toml-lang/grammars repo or similar.

CC @pradyunsg

[pradyunsg]

It does look like Textmate's repositories aren't getting much maintenance anymore. I do think the best course of action is to identify some other up-to-date TOML syntax bundle, and suggest GitHub to switch to.

I'm not sure if toml-lang is the best option or place to put TOML grammers. I mean, it'd be nice for sure, but there's basically only me doing maintainery stuff in toml-lang right now, on my limited free time, and taking up responsibility for maintaining language grammers for TOML isn't something that I have the time budget for.

I think there's some parallels to why TOML doesn't have a canonical implementation, and, while there's clear differences, I do think we're better off not adding a toml-lang/grammers repository in the short term.

GitHub's poor highlighting in the README is... suboptimal right now but we can stop using toml for those code blocks (if someone feels strongly about this, you're welcome to file a PR doing this). I don't feel too strongly about this, since we're gonna have a website soon™-ish.

[abelbraaksma]

I've pinged the maintainers, maybe it'll help. They seem to still be active on github, at least a bit.

[pradyunsg]

Oh, and @cannikin and I just put https://toml.io/ online. :)

[abelbraaksma]

@pradyunsg, that looks awesome! Niiiice! :)

[pyfisch]

GitHub's poor highlighting in the README is... suboptimal right now but we can stop using toml for those code blocks (if someone feels strongly about this, you're welcome to file a PR doing this). I don't feel too strongly about this, since we're gonna have a website soon™-ish.

My issue is not that the README is poorly highlighted but all TOML files published on GitHub are. While not all files are affected those using date/time and some newer syntax definitely are.

[abelbraaksma]

@pyfisch, it's a sad truth for basically all highlighters, as I've yet to find one that is up to date with the latest if any given language, or that is capable of highlighting all features of a certain language.

Though I fully agree that it would be wonderful that this is updated. I don't know if it's possible to fork and just switch highlighting to that fork? Not sure where the highlighting libraries are connected to repos, if it is configurable for individual repos and if it's easy to change this globally (I assume that requires github personnel to take action).

We could also ask for someone to be added to the list of maintainers.

[marzer]

@abelbraaksma

Though I fully agree that it would be wonderful that this is updated. I don't know if it's possible to fork and just switch highlighting to that fork? Not sure where the highlighting libraries are connected to repos, if it is configurable for individual repos and if it's easy to change this globally (I assume that requires github personnel to take action).

They explain how to switch here: Changing the source of a syntax highlighting grammar

[pyfisch]

The PR to textmate/toml.tmbundle got merged! 🥳

[pradyunsg]

Awesome! Looks like we can close this issue now. :)