You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have an @autovalue class with the @DefaultSchema(AutoValueSchema.class). One of the getters/setters is a Map<Integer, ByteBuffer>. When using elements of this class in my pipeline, I get the following error:
Error stack trace
java.lang.ClassCastException: class [B cannot be cast to class java.nio.ByteBuffer ([B and java.nio.ByteBuffer are in module java.base of loader 'bootstrap')
at org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.Function$SchemaCodeGen$sIwEd1pP.apply(Unknown Source)
at org.apache.beam.sdk.schemas.utils.ByteBuddyUtils$TransformingMap.<init>(ByteBuddyUtils.java:525)
at org.apache.beam.sdk.schemas.utils.ByteBuddyUtils.getTransformingMap(ByteBuddyUtils.java:508)
at org.apache.beam.sdk.io.iceberg.FieldValueGetter$SchemaCodeGen$c6I5wjZP.get(Unknown Source)
at org.apache.beam.sdk.io.iceberg.FieldValueGetter$SchemaCodeGen$c6I5wjZP.get(Unknown Source)
at org.apache.beam.sdk.values.RowWithGetters$1.apply(RowWithGetters.java:82)
at org.apache.beam.sdk.values.RowWithGetters$1.apply(RowWithGetters.java:75)
at java.base/java.util.Map.computeIfAbsent(Map.java:1003)
at org.apache.beam.sdk.values.RowWithGetters.getValue(RowWithGetters.java:73)
at org.apache.beam.sdk.coders.RowCoderGenerator$EncodeInstruction.encodeDelegate(RowCoderGenerator.java:419)
at org.apache.beam.sdk.coders.Coder$ByteBuddy$w90nPOHK.encode(Unknown Source)
at org.apache.beam.sdk.coders.Coder$ByteBuddy$w90nPOHK.encode(Unknown Source)
at org.apache.beam.sdk.schemas.SchemaCoder.encode(SchemaCoder.java:121)
at org.apache.beam.sdk.coders.RowCoderGenerator$EncodeInstruction.encodeDelegate(RowCoderGenerator.java:437)
at org.apache.beam.sdk.coders.Coder$ByteBuddy$1ANxtLai.encode(Unknown Source)
at org.apache.beam.sdk.coders.Coder$ByteBuddy$1ANxtLai.encode(Unknown Source)
at org.apache.beam.sdk.schemas.SchemaCoder.encode(SchemaCoder.java:121)
at org.apache.beam.sdk.coders.Coder.encode(Coder.java:135)
at org.apache.beam.sdk.util.CoderUtils.encodeToSafeStream(CoderUtils.java:86)
at org.apache.beam.sdk.util.CoderUtils.encodeToByteArray(CoderUtils.java:70)
at org.apache.beam.sdk.util.CoderUtils.encodeToByteArray(CoderUtils.java:55)
at org.apache.beam.sdk.util.CoderUtils.clone(CoderUtils.java:168)
at org.apache.beam.runners.direct.CloningBundleFactory$CloningBundle.add(CloningBundleFactory.java:87)
at org.apache.beam.runners.direct.FlattenEvaluatorFactory$FlattenEvaluator.processElement(FlattenEvaluatorFactory.java:75)
at org.apache.beam.runners.direct.DirectTransformExecutor.processElements(DirectTransformExecutor.java:165)
at org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:129)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
I think I was able to track the problem down to this code in ByteBuddy:
From what I understand, Beam does some conversions between ByteBuffer and byte[] (for reasons I am not aware of). In the above code, the Map's key and value element types are not transparent. Instead, they show up as generics. Because of this, we never enter the if block, which contains code that applies conversion logic.
Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
Issue Components
Component: Python SDK
Component: Java SDK
Component: Go SDK
Component: Typescript SDK
Component: IO connector
Component: Beam YAML
Component: Beam examples
Component: Beam playground
Component: Beam katas
Component: Website
Component: Infrastructure
Component: Spark Runner
Component: Flink Runner
Component: Samza Runner
Component: Twister2 Runner
Component: Hazelcast Jet Runner
Component: Google Cloud Dataflow Runner
The text was updated successfully, but these errors were encountered:
What happened?
I have an @autovalue class with the
@DefaultSchema(AutoValueSchema.class)
. One of the getters/setters is aMap<Integer, ByteBuffer>
. When using elements of this class in my pipeline, I get the following error:Error stack trace
I think I was able to track the problem down to this code in ByteBuddy:
beam/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/utils/ByteBuddyUtils.java
Lines 1113 to 1125 in e4ab66d
From what I understand, Beam does some conversions between
ByteBuffer
andbyte[]
(for reasons I am not aware of). In the above code, the Map's key and value element types are not transparent. Instead, they show up as generics. Because of this, we never enter theif
block, which contains code that applies conversion logic.Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
Issue Components
The text was updated successfully, but these errors were encountered: