Skip to content

Commit 6148a77

Browse files
gengliangwangHyukjinKwon
authored andcommitted
[SPARK-25883][BACKPORT][SQL][MINOR] Override method prettyName in from_avro/to_avro
Back port apache#22890 to branch-2.4. It is a bug fix for this issue: https://issues.apache.org/jira/browse/SPARK-26063 ## What changes were proposed in this pull request? Previously in from_avro/to_avro, we override the method `simpleString` and `sql` for the string output. However, the override only affects the alias naming: ``` Project [from_avro('col, ... , (mode,PERMISSIVE)) AS from_avro(col, struct<col1:bigint,col2:double>, Map(mode -> PERMISSIVE))#11] ``` It only makes the alias name quite long: `from_avro(col, struct<col1:bigint,col2:double>, Map(mode -> PERMISSIVE))`). We should follow `from_csv`/`from_json` here, to override the method prettyName only, and we will get a clean alias name ``` ... AS from_avro(col)#11 ``` ## How was this patch tested? Manual check Closes apache#23047 from gengliangwang/backport_avro_pretty_name. Authored-by: Gengliang Wang <[email protected]> Signed-off-by: hyukjinkwon <[email protected]>
1 parent 96834fb commit 6148a77

File tree

2 files changed

+2
-14
lines changed

2 files changed

+2
-14
lines changed

external/avro/src/main/scala/org/apache/spark/sql/avro/AvroDataToCatalyst.scala

+1-7
Original file line numberDiff line numberDiff line change
@@ -51,13 +51,7 @@ case class AvroDataToCatalyst(child: Expression, jsonFormatSchema: String)
5151
deserializer.deserialize(result)
5252
}
5353

54-
override def simpleString: String = {
55-
s"from_avro(${child.sql}, ${dataType.simpleString})"
56-
}
57-
58-
override def sql: String = {
59-
s"from_avro(${child.sql}, ${dataType.catalogString})"
60-
}
54+
override def prettyName: String = "from_avro"
6155

6256
override protected def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
6357
val expr = ctx.addReferenceObj("this", this)

external/avro/src/main/scala/org/apache/spark/sql/avro/CatalystDataToAvro.scala

+1-7
Original file line numberDiff line numberDiff line change
@@ -52,13 +52,7 @@ case class CatalystDataToAvro(child: Expression) extends UnaryExpression {
5252
out.toByteArray
5353
}
5454

55-
override def simpleString: String = {
56-
s"to_avro(${child.sql}, ${child.dataType.simpleString})"
57-
}
58-
59-
override def sql: String = {
60-
s"to_avro(${child.sql}, ${child.dataType.catalogString})"
61-
}
55+
override def prettyName: String = "to_avro"
6256

6357
override protected def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
6458
val expr = ctx.addReferenceObj("this", this)

0 commit comments

Comments
 (0)