Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON output no longer quotes all values #1298

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

parkera101093
Copy link
Contributor

JSONInfosetInputter and Outputter were treating all values as strings. This meant everyhing was quoted and was messier for the user to read. In addition, not all json comes from dfdl, so it makes sense to support numbers.

As of the time of this commit 156 tests fail as the changes cause some issues with dates and NaN, INF, and -INF values as well as numbers with a large amount of decimals such as 0.1234567898765432123456789 from test decimal_constructor_06.

DAFFODIL-2362

JSONInfosetInputter and Outputter were treating all values as strings.
This meant everyhing was quoted and was messier for the user to read.
In addition, not all json comes from dfdl, so it makes sense to support numbers.

The JSONInfosetInputter's getSimpleText function also throws a
NonTextFoundInSimpleContentException if a node in a nonscalar value, since
finding a nonscalar (such as an object or array) when your expecting a simple value
means the infoset does not match.

 DAFFODIL-2362

deleteme
Copy link
Member

@stevedlawrence stevedlawrence left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just some suggestions for minor clean up and add a couple of tests to make sure we handle an error case correctly. I don't think we did before this change so it would be good to verify it's fixed and get a reasonable diagnostic now.

@@ -118,10 +123,11 @@ class JsonInfosetInputter(input: java.io.InputStream) extends InfosetInputter {
primType: NodeInfo.Kind,
runtimeProperties: java.util.Map[String, String]
): String = {
if (jsp.getCurrentToken() == JsonToken.VALUE_NULL) {
if (!jsp.getCurrentToken().isScalarValue()) {
throw new NonTextFoundInSimpleContentException(jsp.getCurrentToken().asString())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can make this diagnostic more helpful to users. Maybe something like this:

"Unexpected array or object '" + getLocalName + "' on line " + jsp.getTokenLocation().getLineNr()

We should have a test that this fails correctly for both arrays and objects.

(simple.metadata.dfdlType == DFDLPrimType.Double && text.equals("-INF")) ||
(simple.metadata.dfdlType == DFDLPrimType.Float && text.equals("NaN")) ||
(simple.metadata.dfdlType == DFDLPrimType.Float && text.equals("INF")) ||
(simple.metadata.dfdlType == DFDLPrimType.Float && text.equals("-INF"))) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's usually best to avoid string comparisons when there's something more specific available. String comparisons can silently break if we ever decide to change the string representation but don't check the condition. For example, Double and Float have isInfinite and isNaN functions.

This is also a pretty big if-statement. I might suggest moving this all to a separate private function, something like this:

private def needsQuote(simple: InofsetSimpleElement): Boolean = {
  simple.metadata.dfdlType match {
    case DFDLPrimType.String => true
    case DFDLPrimType.HexBinary => true
    case DFDLprimType.AnyURI => true
    ...
    // json does not support inf/nan double/float so they must be quoted
    case DFDLPrimType.Double => {
      val d = simple.getDouble
      d.isInfinite || d.isNaN
    }
    case DFDLPrimType.Float => {
      val f = simple.getFloat
      f.isInfinity || f.isNan
    }
    case _ => false
}

Then this is statement just becomes if (needsQuotes(simple)) ..... This makes it much easier to understand what this if-statement is checking for without having to decipher it. And it makes it easier to add comments for unexpected edgecases like inf/nan support.

@stevedlawrence stevedlawrence marked this pull request as ready for review September 27, 2024 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants