WISH: JSON decoder to skip comments

[hiiamboris]

A lot of projects in the wild are using JSON for their config files (despite it never having been intended as config format). As a consequence they support // and /* */ style comments, because otherwise it would be annoying to work with such configs. There are even a few JSON extension formats with comment support. Red JSON decoder, being compliant, just fails to load such files.

I propose adding comment support as a decoder option (refinement).

[greggirwin]

It's a seductive idea, but a tough call. I talked to Crockford about this years ago and his stance was firm that, when he removed comments from the spec, it was intentional and those using them were non-compliant (and misusing comments). The idea being that if there is important information that should be there, put it in the data.

Then we have to ask if they should be loaded, how they appear in loaded data, can they be round-tripped, can they be nested, are they just for docs or also allowed for commenting out data. Does a malformed comment affect the entire file. Is it both // and /* */ syntaxes? Any others?

XML has the same question. People ask for non-disappearing comments in Redbol from time to time as well.

Let's say we add it. Great. Now you load the data and...what happened to the comments? To keep it simple, assume they vanish, as in loaded Red. What are the specific use cases, other than "people are doing it and we may want to." Then ask how you know you need to use the refinement, because one project you support does. In that case, a JSON pre-processor comment-stripper would work just as well, wouldn't it? I agree that it's nicer to have help for it, and make it easy to use (in general), but it also affects what may be the most widely used codec in the world.

[hiiamboris]

Any others?

// and /* */ are common (probably also #, and forgiving last comma - this needs more research on what extensions are common enough to support).

To keep it simple, assume they vanish

As the title says ;)

What are the specific use cases

IIRC I was trying to validate some JSON in Red when the other program gave me a dumb error report, not even mentioning in which line the error lies or how it looks like.

a JSON pre-processor comment-stripper would work just as well

A separate stripper makes no sense because it won't work outside JSON context, because to support comments it has to understand JSON syntax, because otherwise it may cripple strings if it finds // inside them.

[hiiamboris]

An example of wheel reinvention (state machine JSON comment filter): https://github.com/SagerNet/sing/blob/dev/common/json/comment.go

[greggirwin]

A separate stripper makes no sense because it won't work outside JSON context, because to support comments it has to understand JSON syntax, because otherwise it may cripple strings if it finds // inside them.

Only needs to understand strings and comments.

[endo64]

I'm with the "simply remove comments and then load" idea, inside the codec.