Skip to content

Commit

Permalink
Merge branch 'LeonPoon-bug-using-either-for-coproduct-v1.5.3'
Browse files Browse the repository at this point in the history
  • Loading branch information
julianpeeters committed Sep 24, 2023
2 parents a96c295 + 9b2161b commit 7836b97
Show file tree
Hide file tree
Showing 25 changed files with 655 additions and 28 deletions.
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,8 +172,8 @@ To reassign Scala types to Avro types, use the following (e.g. for customizing `
* `record` can be assigned to `ScalaCaseClass` and `ScalaCaseClassWithSchema`(with schema in a companion object)
* `array` can be assigned to `ScalaSeq`, `ScalaArray`, `ScalaList`, and `ScalaVector`
* `enum` can be assigned to `JavaEnum`, `ScalaCaseObjectEnum`, `EnumAsScalaString`, and `ScalaEnumeration`
* `fixed` can be assigned to , `ScalaCaseClassWrapper` and `ScalaCaseClassWrapperWithSchema`(with schema in a companion object)
* `union` can be assigned to `OptionShapelessCoproduct`, `OptionEitherShapelessCoproduct`, or `OptionalShapelessCoproduct`
* `fixed` can be assigned to `ScalaCaseClassWrapper` and `ScalaCaseClassWrapperWithSchema`(with schema in a companion object)
* `union` can be assigned to `OptionShapelessCoproduct`, `OptionEitherShapelessCoproduct`, or `OptionalShapelessCoproduct`
* `int`, `long`, `float`, `double` can be assigned to `ScalaInt`, `ScalaLong`, `ScalaFloat`, `ScalaDouble`
* `protocol` can be assigned to `ScalaADT` and `NoTypeGenerated`
* `decimal` can be assigned to e.g. `ScalaBigDecimal(Some(BigDecimal.RoundingMode.HALF_EVEN))` and `ScalaBigDecimalWithPrecision(None)` (via Shapeless Tagged Types)
Expand Down Expand Up @@ -267,9 +267,9 @@ Depends on [Avro](https://github.com/apache/avro) and [Treehugger](https://githu

Contributors:

| | | | |
| :--- | :--- | :--- | :--- |
| [Marius Soutier](https://github.com/mariussoutier) </br> [Brian London](https://github.com/BrianLondon) </br> [alancnet](https://github.com/alancnet) </br> [Matt Coffin](https://github.com/mcoffin) </br> [Ryan Koval](http://github.ryankoval.com) </br> [Simonas Gelazevicius](https://github.com/simsasg) </br> [Paul Snively](https://github.com/PaulAtBanno) </br> [Marco Stefani](https://github.com/inafets) </br> [Andrew Gustafson](https://github.com/agustafson) </br> [Kostya Golikov](https://github.com/lazyval) </br> [Plínio Pantaleão](https://github.com/plinioj) </br> [Sietse de Kaper](https://github.com/targeter) </br> [Martin Mauch](https://github.com/nightscape) </br> [LeonPoon](https://github.com/LeonPoon) | [Paul Pearcy](https://github.com/ppearcy) </br> [Matt Allen](https://github.com/Matt343) </br> [C-zito](https://github.com/C-Zito) </br> [Tim Chan](https://github.com/timchan-lumoslabs) </br> [Saket](https://github.com/skate056) </br> [Daniel Davis](https://github.com/wabu) </br> [Zach Cox](https://github.com/zcox) </br> [Diego E. Alonso Blas](https://github.com/diesalbla) </br> [Fede Fernández](https://github.com/fedefernandez) </br> [Rob Landers](https://github.com/withinboredom) </br> [Simon Petty](https://github.com/simonpetty) </br> [Andreas Drobisch](https://github.com/adrobisch) </br> [Timo Schmid](https://github.com/timo-schmid) | [Stefano Galarraga](https://github.com/galarragas) </br> [Lars Albertsson](https://github.com/lallea) </br> [Eugene Platonov](https://github.com/jozic) </br> [Jerome Wacongne](https://github.com/ch4mpy) </br> [Jon Morra](https://github.com/jon-morra-zefr) </br> [Raúl Raja Martínez](https://github.com/raulraja) </br> [Kaur Matas](https://github.com/kmatasflp) </br> [Chris Albright](https://github.com/chrisalbright) </br> [Francisco Díaz](https://github.com/franciscodr) </br> [Bobby Rauchenberg](https://github.com/bobbyrauchenberg) </br> [Leonard Ehrenfried](https://github.com/leonardehrenfried) </br> [François Sarradin](https://github.com/fsarradin) </br> [niqdev](https://github.com/niqdev) | [Julien BENOIT](https://github.com/jbenoit2011) </br> [Adam Drakeford](https://github.com/dr4ke616) </br> [Carlos Silva](https://github.com/alchimystic) </br> [ismail Benammar](https://github.com/ismailBenammar) </br> [mcenkar](https://github.com/mcenkar) </br> [Luca Tronchin](https://github.com/ltronky) </br> [LydiaSkuse](https://github.com/LydiaSkuse) </br> [Algimantas Milašius](https://github.com/AlgMi) </br> [Leonard Ehrenfried](https://github.com/leonardehrenfried) </br> [Massimo Siani](https://github.com/massimosiani) </br> [Konstantin](https://github.com/tyger) </br> [natefitzgerald](https://github.com/natefitzgerald) </br> [Victor](https://github.com/gafiatulin) |
| | | | |
|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| :--- | :--- | :--- |
| [Marius Soutier](https://github.com/mariussoutier) </br> [Brian London](https://github.com/BrianLondon) </br> [alancnet](https://github.com/alancnet) </br> [Matt Coffin](https://github.com/mcoffin) </br> [Ryan Koval](http://github.ryankoval.com) </br> [Simonas Gelazevicius](https://github.com/simsasg) </br> [Paul Snively](https://github.com/PaulAtBanno) </br> [Marco Stefani](https://github.com/inafets) </br> [Andrew Gustafson](https://github.com/agustafson) </br> [Kostya Golikov](https://github.com/lazyval) </br> [Plínio Pantaleão](https://github.com/plinioj) </br> [Sietse de Kaper](https://github.com/targeter) </br> [Martin Mauch](https://github.com/nightscape) </br> [Leon Poon](https://github.com/LeonPoon) | [Paul Pearcy](https://github.com/ppearcy) </br> [Matt Allen](https://github.com/Matt343) </br> [C-zito](https://github.com/C-Zito) </br> [Tim Chan](https://github.com/timchan-lumoslabs) </br> [Saket](https://github.com/skate056) </br> [Daniel Davis](https://github.com/wabu) </br> [Zach Cox](https://github.com/zcox) </br> [Diego E. Alonso Blas](https://github.com/diesalbla) </br> [Fede Fernández](https://github.com/fedefernandez) </br> [Rob Landers](https://github.com/withinboredom) </br> [Simon Petty](https://github.com/simonpetty) </br> [Andreas Drobisch](https://github.com/adrobisch) </br> [Timo Schmid](https://github.com/timo-schmid) | [Stefano Galarraga](https://github.com/galarragas) </br> [Lars Albertsson](https://github.com/lallea) </br> [Eugene Platonov](https://github.com/jozic) </br> [Jerome Wacongne](https://github.com/ch4mpy) </br> [Jon Morra](https://github.com/jon-morra-zefr) </br> [Raúl Raja Martínez](https://github.com/raulraja) </br> [Kaur Matas](https://github.com/kmatasflp) </br> [Chris Albright](https://github.com/chrisalbright) </br> [Francisco Díaz](https://github.com/franciscodr) </br> [Bobby Rauchenberg](https://github.com/bobbyrauchenberg) </br> [Leonard Ehrenfried](https://github.com/leonardehrenfried) </br> [François Sarradin](https://github.com/fsarradin) </br> [niqdev](https://github.com/niqdev) | [Julien BENOIT](https://github.com/jbenoit2011) </br> [Adam Drakeford](https://github.com/dr4ke616) </br> [Carlos Silva](https://github.com/alchimystic) </br> [ismail Benammar](https://github.com/ismailBenammar) </br> [mcenkar](https://github.com/mcenkar) </br> [Luca Tronchin](https://github.com/ltronky) </br> [LydiaSkuse](https://github.com/LydiaSkuse) </br> [Algimantas Milašius](https://github.com/AlgMi) </br> [Leonard Ehrenfried](https://github.com/leonardehrenfried) </br> [Massimo Siani](https://github.com/massimosiani) </br> [Konstantin](https://github.com/tyger) </br> [natefitzgerald](https://github.com/natefitzgerald) </br> [Victor](https://github.com/gafiatulin) |


##### Criticism is appreciated.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -116,16 +116,11 @@ object SpecificImporter extends Importer {
case OptionShapelessCoproduct => 1
case OptionEitherShapelessCoproduct => 2 // unions of one nullable type become Option, two become Either
}
val unionContainsNull: Schema => Boolean = _.getType == Schema.Type.NULL
def shapelessCoproductTest(unionTypes: List[Schema], maxNonNullTypes: Int): Boolean =
(unionTypes.length > maxNonNullTypes && !unionTypes.exists(unionContainsNull)) ||
(unionTypes.length > maxNonNullTypes + 1 && unionTypes.exists(unionContainsNull))
def defaultValueTest(field: Schema.Field, unionTypes: List[Schema], maxNonNullTypes: Int) =
(unionTypes.length > maxNonNullTypes + 1 && !unionTypes.exists(unionContainsNull)) ||
(unionTypes.length > maxNonNullTypes + 2 && unionTypes.exists(unionContainsNull))
val unionTypes = unionSchema.getTypes().asScala.toList
val isShapelessCoproduct: Boolean = shapelessCoproductTest(unionTypes, thresholdNonNullTypes)
val hasDefaultValue: Boolean = defaultValueTest(field, unionTypes, thresholdNonNullTypes)
val unionNonNullTypes = unionTypes.filterNot(_.getType == Schema.Type.NULL)
val unionContainsNull: Boolean = unionNonNullTypes.length < unionTypes.length
val isShapelessCoproduct: Boolean = unionNonNullTypes.length > thresholdNonNullTypes
val hasDefaultValue: Boolean = !unionContainsNull
val unionImports = if (isShapelessCoproduct && hasDefaultValue)
List(":+:", "CNil", "Coproduct")
else if (isShapelessCoproduct)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,8 +68,9 @@ object DefaultParamMatcher {
case Schema.Type.UNION =>
val schemas = avroSchema.getTypes.asScala.toList
if (avroSchema.isNullable) NONE
else if (schemas.size == 2) LEFT(asDefaultParam(classStore, schemas.head, typeMatcher))
else COPRODUCT(asDefaultParam(classStore, schemas.head, typeMatcher), schemas.map(_.getName))
else if (schemas.size == 2 && typeMatcher.avroScalaTypes.union.useEitherForTwoNonNullTypes)
LEFT(asDefaultParam(classStore, schemas.head, typeMatcher))
else COPRODUCT(asDefaultParam(classStore, schemas.head, typeMatcher), schemas.map(typeMatcher.toScalaType(classStore, None, _).safeToString))
case Schema.Type.ARRAY =>
CustomDefaultParamMatcher.checkCustomArrayType(typeMatcher.avroScalaTypes.array) DOT "empty"
case Schema.Type.MAP =>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -177,13 +177,14 @@ object DefaultValueMatcher {
def unionsArityStrategy(
classStore: ClassStore,
namespace: Option[String],
typeMatcher: TypeMatcher) =
typeMatcher: TypeMatcher,
useEither: Boolean) =
nonNullableSchemas match {
case Nil =>
UNIT
case List(schemaA) => //Option
treeMatcher(node, schemaA)
case List(schemaA, schemaB) if typeMatcher.avroScalaTypes.union == OptionEitherShapelessCoproduct => //Either
case List(schemaA, _) if useEither => //Either
LEFT(treeMatcher(node, schemaA))
case firstSchema :: _ => //Coproduct
COPRODUCT(firstSchema,
Expand All @@ -192,8 +193,8 @@ object DefaultValueMatcher {
}

def matchedTree(classStore: ClassStore, namespace: Option[String]) = typeMatcher.avroScalaTypes.union match {
case OptionShapelessCoproduct => unionsArityStrategy(classStore, namespace, typeMatcher)
case OptionEitherShapelessCoproduct => unionsArityStrategy(classStore, namespace, typeMatcher)
case OptionShapelessCoproduct | OptionEitherShapelessCoproduct =>
unionsArityStrategy(classStore, namespace, typeMatcher, typeMatcher.avroScalaTypes.union.useEitherForTwoNonNullTypes)
case OptionalShapelessCoproduct => unionsAsOptionalShapelessCoproductStrategy
}

Expand Down
11 changes: 9 additions & 2 deletions avrohugger-core/src/main/scala/types/ComplexAvroScalaTypes.scala
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,17 @@ case object JavaEnum extends AvroScalaEnumType
case object ScalaCaseObjectEnum extends AvroScalaEnumType
case object EnumAsScalaString extends AvroScalaEnumType
// union
sealed trait AvroScalaUnionType extends Product with Serializable
sealed trait AvroScalaUnionType extends Product with Serializable {
val useEitherForTwoNonNullTypes: Boolean = false

}

case object OptionalShapelessCoproduct extends AvroScalaUnionType
case object OptionShapelessCoproduct extends AvroScalaUnionType
case object OptionEitherShapelessCoproduct extends AvroScalaUnionType
case object OptionEitherShapelessCoproduct extends AvroScalaUnionType {
override val useEitherForTwoNonNullTypes: Boolean = true

}
// array
sealed trait AvroScalaArrayType extends Product with Serializable
case object ScalaArray extends AvroScalaArrayType
Expand Down
76 changes: 76 additions & 0 deletions avrohugger-core/src/test/avro/unions_with_coproduct2.avsc
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
[
{
"namespace": "com.example.avrohugger.unions_with_coproduct_avsc2",
"name": "UnionOfNullWithOneNonNullType",
"type": "record",
"fields": [
{
"name": "f3",
"type": [
"null",
"double"
]
}
]
},
{
"namespace": "com.example.avrohugger.unions_with_coproduct_avsc2",
"name": "UnionOfTwoNonNullTypes",
"type": "record",
"fields": [
{
"name": "r4",
"type": [
"int",
"string"
]
}
]
},
{
"namespace": "com.example.avrohugger.unions_with_coproduct_avsc2",
"name": "UnionOfNullWithTwoNonNullTypes",
"type": "record",
"fields": [
{
"name": "r5",
"type": [
"null",
"int",
"string"
]
}
]
},
{
"namespace": "com.example.avrohugger.unions_with_coproduct_avsc2",
"name": "UnionOfMoreThanTwoNonNullTypes",
"type": "record",
"fields": [
{
"name": "r6",
"type": [
"boolean",
"int",
"string"
]
}
]
},
{
"namespace": "com.example.avrohugger.unions_with_coproduct_avsc2",
"name": "UnionOfNullWithMoreThanTwoNonNullTypes",
"type": "record",
"fields": [
{
"name": "r7",
"type": [
"null",
"boolean",
"int",
"string"
]
}
]
}
]
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
/** MACHINE-GENERATED FROM AVRO SCHEMA. DO NOT EDIT DIRECTLY */
package com.example.avrohugger.unions_with_coproduct_avsc2

import scala.annotation.switch

import shapeless.{:+:, CNil, Coproduct}

final case class UnionOfMoreThanTwoNonNullTypes(var r6: Boolean :+: Int :+: String :+: CNil) extends org.apache.avro.specific.SpecificRecordBase {
def this() = this(shapeless.Coproduct[Boolean :+: Int :+: String :+: CNil](false))
def get(field$: Int): AnyRef = {
(field$: @switch) match {
case 0 => {
r6
}.asInstanceOf[AnyRef]
case _ => new org.apache.avro.AvroRuntimeException("Bad index")
}
}
def put(field$: Int, value: Any): Unit = {
(field$: @switch) match {
case 0 => this.r6 = {
value
}.asInstanceOf[Boolean :+: Int :+: String :+: CNil]
case _ => new org.apache.avro.AvroRuntimeException("Bad index")
}
()
}
def getSchema: org.apache.avro.Schema = com.example.avrohugger.unions_with_coproduct_avsc2.UnionOfMoreThanTwoNonNullTypes.SCHEMA$
}

object UnionOfMoreThanTwoNonNullTypes {
val SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"UnionOfMoreThanTwoNonNullTypes\",\"namespace\":\"com.example.avrohugger.unions_with_coproduct_avsc2\",\"fields\":[{\"name\":\"r6\",\"type\":[\"boolean\",\"int\",\"string\"]}]}")
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
/** MACHINE-GENERATED FROM AVRO SCHEMA. DO NOT EDIT DIRECTLY */
package com.example.avrohugger.unions_with_coproduct_avsc2

import scala.annotation.switch

import shapeless.{:+:, CNil}

final case class UnionOfNullWithMoreThanTwoNonNullTypes(var r7: Option[Boolean :+: Int :+: String :+: CNil]) extends org.apache.avro.specific.SpecificRecordBase {
def this() = this(None)
def get(field$: Int): AnyRef = {
(field$: @switch) match {
case 0 => {
r7 match {
case Some(x) => x
case None => null
}
}.asInstanceOf[AnyRef]
case _ => new org.apache.avro.AvroRuntimeException("Bad index")
}
}
def put(field$: Int, value: Any): Unit = {
(field$: @switch) match {
case 0 => this.r7 = {
value match {
case null => None
case _ => Some(value)
}
}.asInstanceOf[Option[Boolean :+: Int :+: String :+: CNil]]
case _ => new org.apache.avro.AvroRuntimeException("Bad index")
}
()
}
def getSchema: org.apache.avro.Schema = com.example.avrohugger.unions_with_coproduct_avsc2.UnionOfNullWithMoreThanTwoNonNullTypes.SCHEMA$
}

object UnionOfNullWithMoreThanTwoNonNullTypes {
val SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"UnionOfNullWithMoreThanTwoNonNullTypes\",\"namespace\":\"com.example.avrohugger.unions_with_coproduct_avsc2\",\"fields\":[{\"name\":\"r7\",\"type\":[\"null\",\"boolean\",\"int\",\"string\"]}]}")
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/** MACHINE-GENERATED FROM AVRO SCHEMA. DO NOT EDIT DIRECTLY */
package com.example.avrohugger.unions_with_coproduct_avsc2

import scala.annotation.switch

final case class UnionOfNullWithOneNonNullType(var f3: Option[Double]) extends org.apache.avro.specific.SpecificRecordBase {
def this() = this(None)
def get(field$: Int): AnyRef = {
(field$: @switch) match {
case 0 => {
f3 match {
case Some(x) => x
case None => null
}
}.asInstanceOf[AnyRef]
case _ => new org.apache.avro.AvroRuntimeException("Bad index")
}
}
def put(field$: Int, value: Any): Unit = {
(field$: @switch) match {
case 0 => this.f3 = {
value match {
case null => None
case _ => Some(value)
}
}.asInstanceOf[Option[Double]]
case _ => new org.apache.avro.AvroRuntimeException("Bad index")
}
()
}
def getSchema: org.apache.avro.Schema = com.example.avrohugger.unions_with_coproduct_avsc2.UnionOfNullWithOneNonNullType.SCHEMA$
}

object UnionOfNullWithOneNonNullType {
val SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"UnionOfNullWithOneNonNullType\",\"namespace\":\"com.example.avrohugger.unions_with_coproduct_avsc2\",\"fields\":[{\"name\":\"f3\",\"type\":[\"null\",\"double\"]}]}")
}
Loading

0 comments on commit 7836b97

Please sign in to comment.