-
Notifications
You must be signed in to change notification settings - Fork 13
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add package extension to support InlineStrings in Arrow.jl
Fixes apache/arrow-julia#196. This utilizes the new package extension feature of Julia 1.9 to add a conditional dependency on the ArrowTypes.jl package. With ArrowTypes.jl, it adds the necessary overloads to allow round- tripping of inline strings through the arrow format. Other language implementations will read them as normal strings, but in the Julia implementation, the additional type metadata signal that these strings were originally inline strings and can be deserialized as such. I'm explicitly not using the Requires.jl hack for backwards compat w/ older Julia versions because I like the idea of this being sort of a "beta" feature for users already using 1.9 to see if there are any unexpected issues that pop up for inline strings in the arrow format.
- Loading branch information
Showing
4 changed files
with
33 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
module ArrowTypesExt | ||
|
||
using ArrowTypes, InlineStrings | ||
|
||
for sz in (1, 4, 8, 16, 32, 64, 128, 256) | ||
nm = Symbol(:InlineString, max(1, sz - 1)) | ||
arrow_nm = Symbol("JuliaLang.InlineStrings.", nm) | ||
@eval begin | ||
ArrowTypes.arrowname(::Type{$nm}) = $(Meta.quot(arrow_nm)) | ||
ArrowTypes.JuliaType(::Val{$(Meta.quot(arrow_nm))}) = $nm | ||
ArrowTypes.fromarrow(::Type{$nm}, ptr::Ptr{UInt8}, len::Int) = $nm(ptr, len) | ||
end | ||
end | ||
|
||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
using Test, Arrow, InlineStrings | ||
|
||
@testset "basic Arrow.jl interop" begin | ||
t = (x = inlinestrings(["a", "b", "sailor"]),) | ||
t2 = Arrow.Table(Arrow.tobuffer(t)) | ||
@test isequal(t.x, t2.x) | ||
@test t2.x[1] isa InlineString | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters