Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

imp: ByteBufferSerde #604

Merged
merged 7 commits into from
Oct 20, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
/*
* Copyright 2017-2023 original authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.micronaut.serde.support.serdes;

import io.micronaut.core.annotation.NonNull;
import io.micronaut.core.annotation.Nullable;
import io.micronaut.core.type.Argument;
import io.micronaut.serde.Decoder;
import io.micronaut.serde.Encoder;
import io.micronaut.serde.Serde;
import jakarta.inject.Singleton;

import java.io.IOException;
import java.io.InputStream;
import java.nio.ByteBuffer;

/**
* {@link Serde} implementation of {@link java.nio.ByteBuffer}.
* This is a based on `com.fasterxml.jackson.databind.ser.std.ByteBufferSerializer` which is licenced under the Apache 2.0 licence.
*/
@Singleton
public class ByteBufferSerde implements Serde<ByteBuffer> {
@Override
public @Nullable ByteBuffer deserialize(@NonNull Decoder decoder,
@NonNull DecoderContext context,
@NonNull Argument<? super ByteBuffer> type) throws IOException {
byte[] b = decoder.decodeBinary();
return ByteBuffer.wrap(b);
}

@Override
public void serialize(@NonNull Encoder encoder,
@NonNull EncoderContext context,
@NonNull Argument<? extends ByteBuffer> type,
@NonNull ByteBuffer value) throws IOException {
encodeByteBuffer(encoder, value);
}

private void encodeByteBuffer(@NonNull Encoder encoder,
@NonNull ByteBuffer value) throws IOException {
if (value.hasArray()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and what if not

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could take inspiration from Jackson ad do something like

diff --git a/serde-support/src/main/java/io/micronaut/serde/support/serdes/ByteBufferSerde.java b/serde-support/src/main/java/io/micronaut/serde/support/serdes/ByteBufferSerde.java
index 06e03a2f..12bc6b8d 100644
--- a/serde-support/src/main/java/io/micronaut/serde/support/serdes/ByteBufferSerde.java
+++ b/serde-support/src/main/java/io/micronaut/serde/support/serdes/ByteBufferSerde.java
@@ -58,6 +58,43 @@ public class ByteBufferSerde implements Serde<ByteBuffer> {
             byte[] copy = new byte[len];
             System.arraycopy(value.array(), offset, copy, 0, len);
             encoder.encodeBinary(copy);
+            return;
+        }
+        try (InputStream s = new ByteBufferInputStream(value.asReadOnlyBuffer().rewind())) {
+            encoder.encodeBinary(s.readAllBytes());
+        }
+    }
+
+    private static class ByteBufferInputStream extends InputStream {
+
+        private static final int EOS = -1;
+        private final ByteBuffer buffer;
+
+        private ByteBufferInputStream(ByteBuffer buffer) {
+            this.buffer = buffer;
+        }
+
+        @Override
+        public int available() {
+            return buffer.remaining();
+        }
+
+        @Override
+        public int read() {
+            if (buffer.hasRemaining()) {
+                return buffer.get() & 0xFF;
+            }
+            return EOS;
+        }
+
+        @Override
+        public int read(byte[] b, int off, int len) {
+            if (buffer.hasRemaining()) {
+                len = Math.min(len, buffer.remaining());
+                buffer.get(b, off, len);
+                return len;
+            }
+            return EOS;
         }
     }
 }

?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I'm not sure my use of rewind is correct here... 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding the following commit efe6b37

final int pos = value.position();
int len = value.limit() - pos;
int offset = value.arrayOffset() + pos;
byte[] copy = new byte[len];
System.arraycopy(value.array(), offset, copy, 0, len);
encoder.encodeBinary(copy);
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
/*
* Copyright 2017-2023 original authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.micronaut.serde.tck.tests;

import io.micronaut.context.annotation.Property;
import io.micronaut.core.util.StringUtils;
import io.micronaut.json.JsonMapper;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;
import java.io.IOException;
import static org.junit.jupiter.api.Assertions.assertEquals;

@Property(name = "micronaut.serde.write-binary-as-array", value = StringUtils.FALSE)
@MicronautTest(startApplication = false)
public class ByteArrayWriteValueAsStringTest {

/**
* byte[] is written as base64 string
* @param jsonMapper JSONMapper either Jackson or Serde implementation
* @throws IOException If an unrecoverable error occurs
*/
@Test
public void testByteArrayAsString(JsonMapper jsonMapper) throws IOException {
final byte[] INPUT_BYTES = new byte[] { 1, 2, 3, 4, 5 };
assertEquals("\"AQIDBAU=\"", jsonMapper.writeValueAsString(INPUT_BYTES));
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
import java.net.InetAddress;
import static org.junit.jupiter.api.Assertions.assertEquals;

@MicronautTest
@MicronautTest(startApplication = false)
public class InetAddressTest {

@Test
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
/*
* Copyright 2017-2023 original authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.micronaut.serde.tck.tests.bytebuffer;

import io.micronaut.context.annotation.Property;
import io.micronaut.core.util.StringUtils;
import io.micronaut.json.JsonMapper;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;

import java.io.IOException;
import java.nio.ByteBuffer;

import static org.junit.jupiter.api.Assertions.assertEquals;

@Property(name = "micronaut.serde.write-binary-as-array", value = StringUtils.FALSE)
@MicronautTest(startApplication = false)
public class ByteBufferDuplicatedTest {
/**
* Test ported from com.fasterxml.jackson.databind.ser.jdk.JDKTypeSerializationTest
* @param jsonMapper JSONMapper either Jackson or Serde implementation
* @throws IOException If an unrecoverable error occurs
*/
@Test
public void testDuplicatedByteBufferWithCustomPosition(JsonMapper jsonMapper) throws IOException {
final byte[] INPUT_BYTES = new byte[] { 1, 2, 3, 4, 5 };

String exp = jsonMapper.writeValueAsString(new byte[] { 3, 4, 5 });
ByteBuffer bbuf = ByteBuffer.wrap(INPUT_BYTES);
bbuf.position(2);
ByteBuffer duplicated = bbuf.duplicate();
assertEquals(exp, jsonMapper.writeValueAsString(duplicated));

// also check differently constructed bytebuffer (noting that
// offset given is the _position_ to use, NOT array offset
exp = jsonMapper.writeValueAsString(new byte[] { 2, 3, 4 });
bbuf = ByteBuffer.wrap(INPUT_BYTES, 1, 3);
assertEquals(exp, jsonMapper.writeValueAsString(bbuf.duplicate()));
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
package io.micronaut.serde.tck.tests.bytebuffer;

import io.micronaut.context.annotation.Property;
import io.micronaut.core.util.StringUtils;
import io.micronaut.json.JsonMapper;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;

import java.io.IOException;
import java.nio.ByteBuffer;

import static org.junit.jupiter.api.Assertions.assertEquals;

@Property(name = "micronaut.serde.write-binary-as-array", value = StringUtils.FALSE)
@MicronautTest(startApplication = false)
public class ByteBufferNativeTest {
/**
* Test ported from com.fasterxml.jackson.databind.ser.jdk.JDKTypeSerializationTest
* @param jsonMapper JSONMapper either Jackson or Serde implementation
* @throws IOException If an unrecoverable error occurs
*/
@Test
public void testByteBufferNative(JsonMapper jsonMapper) throws IOException {
final byte[] INPUT_BYTES = new byte[]{1, 2, 3, 4, 5};
String exp = jsonMapper.writeValueAsString(INPUT_BYTES);
// so far so good, but must ensure Native buffers also work:
ByteBuffer bbuf2 = ByteBuffer.allocateDirect(5);
bbuf2.put(INPUT_BYTES);
assertEquals(exp, jsonMapper.writeValueAsString(bbuf2));
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
/*
* Copyright 2017-2023 original authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.micronaut.serde.tck.tests.bytebuffer;

import io.micronaut.context.annotation.Property;
import io.micronaut.core.util.StringUtils;
import io.micronaut.json.JsonMapper;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;

import java.io.IOException;
import java.nio.ByteBuffer;

import static org.junit.jupiter.api.Assertions.assertEquals;

@Property(name = "micronaut.serde.write-binary-as-array", value = StringUtils.FALSE)
@MicronautTest(startApplication = false)
public class ByteBufferSlicedTest {

/**
* Test ported from com.fasterxml.jackson.databind.ser.jdk.JDKTypeSerializationTest
* @param jsonMapper JSONMapper either Jackson or Serde implementation
* @throws IOException If an unrecoverable error occurs
*/
@Test
public void testSlicedByteBuffer(JsonMapper jsonMapper) throws IOException {
final byte[] INPUT_BYTES = new byte[] { 1, 2, 3, 4, 5 };
ByteBuffer bbuf = ByteBuffer.wrap(INPUT_BYTES);

bbuf.position(2);
ByteBuffer slicedBuf = bbuf.slice();

assertEquals(jsonMapper.writeValueAsString(new byte[] { 3, 4, 5 }),
jsonMapper.writeValueAsString(slicedBuf));

// but how about offset within?
slicedBuf.position(1);
assertEquals(jsonMapper.writeValueAsString(new byte[] { 4, 5 }),
jsonMapper.writeValueAsString(slicedBuf));
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
package io.micronaut.serde.tck.tests.bytebuffer;

import io.micronaut.context.annotation.Property;
import io.micronaut.core.util.StringUtils;
import io.micronaut.json.JsonMapper;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;

import java.io.IOException;
import java.nio.ByteBuffer;

import static org.junit.jupiter.api.Assertions.assertEquals;
@Property(name = "micronaut.serde.write-binary-as-array", value = StringUtils.FALSE)
@MicronautTest(startApplication = false)
public class ByteBufferTest {

/**
* Test ported from com.fasterxml.jackson.databind.ser.jdk.JDKTypeSerializationTest
* @param jsonMapper JSONMapper either Jackson or Serde implementation
* @throws IOException If an unrecoverable error occurs
*/
@Test
public void testByteBuffer(JsonMapper jsonMapper) throws IOException {
final byte[] INPUT_BYTES = new byte[]{1, 2, 3, 4, 5};
String exp = jsonMapper.writeValueAsString(INPUT_BYTES);
ByteBuffer bbuf = ByteBuffer.wrap(INPUT_BYTES);
assertEquals(exp, jsonMapper.writeValueAsString(bbuf));
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
/*
* Copyright 2017-2023 original authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/**
* Tests ported from Jackson Databind project.
* Jackson-databind is licensed under Apache 2.0. License
*/
package io.micronaut.serde.tck.tests.bytebuffer;
10 changes: 10 additions & 0 deletions test-suite-tck-jackson-databind/src/test/resources/logback.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<root level="info">
<appender-ref ref="STDOUT" />
</root>
</configuration>
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
package io.micronaut.serde.tck.tests.serde;

import org.junit.platform.suite.api.ExcludeClassNamePatterns;
timyates marked this conversation as resolved.
Show resolved Hide resolved
import org.junit.platform.suite.api.Suite;
import org.junit.platform.suite.api.SelectPackages;
import org.junit.platform.suite.api.SuiteDisplayName;

@Suite
@ExcludeClassNamePatterns(
"io.micronaut.serde.tck.tests.bytebuffer.ByteBufferNativeTest"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since youre copying the data anyway, you can just copy it for direct buffers too. but ideally there should be overloads for encodeBinary for ByteBuffers.

)
@SelectPackages("io.micronaut.serde.tck.tests")
@SuiteDisplayName("Serialization TCK Serde")
public class SerializationSerdeSuite {
Expand Down
10 changes: 10 additions & 0 deletions test-suite-tck-serde/src/test/resources/logback.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<root level="info">
<appender-ref ref="STDOUT" />
</root>
</configuration>
Loading