RFC: Split generic deriving into specific cases, powered by DerivingVia.

[phadej]

A problem I find in improving aeson is dealing with generic and TH code. It's monolithic, trying to cater for all needs at once.
Issues like #824 and #827 is hard to incorporate as everything is coupled. Compiling generics code is also slow (#810) as simplifier needs to work out a lot of code.

I propose a long term strategy of splitting various kinds of encoding formats into separate newtype wrappers to be used with DerivingVia

• The Feature Request: deriving via newtype wrapper offering efficient toEncoding #909 issue would be a good start, to provide the "correct" Generic deriving. (would resolve Inconsistent encoding of Double depending on container structure #660)
• The following step would be to provide deriving newtypes for product types. We might introduce To/FromJSONObject classes here.
• I propose to not touch the current deriving code for a while, until the alternatives are sufficiently expressive.
• ... but when they are, remove the current generics code and default implementations. That would be a major breaking change at that point, but on the other hand, any further changes to generics code will be much more localized.
  • That also will allow to move generics code out of ToJSON & FromJSON modules, making working on aeson slightly more pleasant.

"simplifying" generics story would also help to improve other aspects of aeson, e.g. #839 is somewhat stuck as getting the change through current generics code is very hard.

Note: TH code should be adopted to be configurable similarly to DerivingVia code, if people would prefer to use TH. This shouldn't be difficult.

In summary: 1. Add an alternative generics code to be used with DerivingVia. 2. Remove old generics code. 3. Profit.

ping @bergmark @Lysxia

[bergmark]

Sounds good!

Does DerivingVia compose well? E.g. if I want to use both fieldLabelModifier and rejectUnknownFields together, how would that look?

[phadej]

Sounds good!

Does DerivingVia compose well? E.g. if I want to use both fieldLabelModifier and rejectUnknownFields together, how would that look?

It doesn't. Each "newtype" has to implement the features, but can use the same auxiliary code for that.

[treeowl]

That also will allow to move generics code out of ToJSON & FromJSON modules, making working on aeson slightly more pleasant.

That approach tends to lead to orphan instances. Are those considered okay here?

[phadej]

@treeowl they won't be orphans, as in my plan there are no default definitions.

[treeowl]

But there need to be instances, right? And presumably the newtype representing the current default will be used for some of them?

[phadej]

And presumably the newtype representing the current default will be used for some of them?

It isn't currently, I don't see why situation would dramatically change. aeson doesn't define instances for compound types (e.g. records with fields) where Generics would be useful.

[treeowl]

Oh, didn't realize they weren't used for Either, tuples, Ordering, etc. Never mind!

[phadej]

2.0.3.0 added instance for Generically a which is the first DerivingVia newtype. More to follow.

[blackheaven]

Sounds good!

Does DerivingVia compose well? E.g. if I want to use both fieldLabelModifier and rejectUnknownFields together, how would that look?

No, but we can have options to compose, such as in: deriving-aeson

[ulidtko]

Good plan @phadej, +1 !

I just had a use-case for an extended deriving strategy... but upon reading through the Data.Aeson.Types.FromJSON module — I'd stopped in my tracks, realizing I'll need to duplicate / reimplement half of that module if not more.

To sketch the idea, imagine numerous serializable record-datatypes with instances of Default:

data MyAirInfo = MyAirInfo
  { airTemperature :: Float
  , airPressurePa  :: !Int
  , airPlanet      :: Text
  }
  deriving Generic
  deriving ToJSON via CustomJSON (KebabCaseWithoutPrefix "air") MyAirInfo

instance Default MyAirInfo where
  def = MyAirInfo
    { airTemperature = 19.0
    , airPressurePa  = 108900
    , airPlanet      = "Earth"
    }

So, given that all fields have sane defaults, and can be omitted from input json...

How to write the FromJSON instance?

-- either the default values are repeated in an hand-rolled parser instance:
--
instance FromJSON MyAirInfo where
  parseJSON = withObject "MyAirInfo" $ \o -> do
    airTemperature <- o .:? "temperature" .!= 19.0
    airPressurePa  <- o .:? "pressure-pa" .!= 108900
    airPlanet      <- o .:? "planet"      .!= "Earth"
    pure MyAirInfo{..}

-- or we leverage the Default instance, but repeat field names and still hand-roll:
--
instance FromJSON MyAirInfo where
  parseJSON = withObject "MyAirInfo" $ \o -> do
    airTemperature <- o .:? "temperature" .!= airTemperature def
    airPressurePa  <- o .:? "pressure-pa" .!= airPressurePa def
    airPlanet      <- o .:? "planet"      .!= airPlanet def
    pure MyAirInfo{..}

-- instead I'd want to say:
--
deriving via CustomJSONwithDefault (KebabCaseWithoutPrefix "air") MyAirInfo instance FromJSON MyAirInfo

— but CustomJSONwithDefault doesn't exist, and is currently hard to write on top of public interface of aeson.

---

Does DerivingVia compose well? E.g. if I want to use both fieldLabelModifier and rejectUnknownFields together, how would that look?

It doesn't. Each "newtype" has to implement the features, but can use the same auxiliary code for that.

I constructively disagree. Check out the CustomJson deriving-strategy here 👉 fumieval/deriving-aeson
— just whip up a type-list of void marker types a-la AesonOptions, then interpret them into respective modifiers of aeson's Options; 200 lines, done.

And although via-strategy newtypes can also compose by nesting — I'd suggest to not restrict any particular composition style in the core aeson library. Leave it on value-level; that's sufficient. Because 👉 fieldstrength/aeson-deriving — implements similar idea, but with a different UI (or like they say in javascript circles, "DX", developer experience).

---

BTW, was this bug ever noticed?.. The ± infinities are flipped here.

aeson/src/Data/Aeson/Types/FromJSON.hs

Lines 1577 to 1582 in 98875e0

	instance FromJSONKey Double where
	fromJSONKey = FromJSONKeyTextParser $ \t -> case t of
	"NaN" -> pure (0/0)
	"-inf" -> pure (1/0)
	"+inf" -> pure (negate 1/0)
	_ -> Scientific.toRealFloat <$> parseScientificText t

[phadej]

CustomJSON is not what I had in mind.

But specific encodings like deriving (FromJSON) via Aeson.TaggedEnum [...modifiers...].

And if the type cannot be encoded as TaggedEnum it would fail.

Modifiers may be shared (e.g. how to mangle constructor names), but that's about it.

Similarly, record encoding as object would have options, etc.

EDIT: In particular I dislike and will remove a (default) option where aeson tries to guess which serialization format to use based on the data. That makes derived instances unstable (if type is modified). E.g. I have really think hard to remember how to make unary constructor type to encode as tagged enum (I.e. constant string).

[ulidtko]

@phadej so what about supporting generic ideas like my CustomJsonWithDefault, do you think that'll become easy to implement? Can FromJSON1 somehow help with that?