Skip to content

Add Utf8View to TypeCategory::Unknown #13350

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 13, 2024
Merged

Add Utf8View to TypeCategory::Unknown #13350

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7f8d5f3
add utf8view
jayzhan211 Nov 11, 2024
908f610
add test
jayzhan211 Nov 11, 2024
2ae09b3
fix comment
jayzhan211 Nov 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions datafusion/expr-common/src/type_coercion/binary.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -330,11 +330,13 @@ impl From<&DataType> for TypeCategory {
return TypeCategory::Array;
}

// String literal is possible to cast to many other types like numeric or datetime,
// therefore, it is categorized as a unknown type
// It is categorized as unknown type because the type will be resolved later on
if matches!(
data_type,
DataType::Utf8 | DataType::LargeUtf8 | DataType::Null
DataType::Utf8
| DataType::LargeUtf8
| DataType::Utf8View
| DataType::Null
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding Utf8View LGTM

but the comment above "String literal is possible to cast to many other types like numeric or datetime," is worrisome. I don't think it's true.

Would it make sense to update that comment?

) {
return TypeCategory::Unknown;
}
Expand Down
8 changes: 8 additions & 0 deletions datafusion/sqllogictest/test_files/coalesce.slt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,14 @@ none_set
statement ok
drop table test1

# coalesce with utf8view
query TTT
select coalesce(arrow_cast(null, 'Utf8View'), arrow_cast('t', 'Utf8')),
arrow_typeof(coalesce(arrow_cast(null, 'Utf8View'), arrow_cast('t', 'Utf8'))),
arrow_typeof(coalesce(arrow_cast(null, 'Utf8'), arrow_cast('t', 'Utf8View')));
----
t Utf8View Utf8View

# test dict coercion with value
statement ok
create table t(c varchar) as values ('a'), (null);
Expand Down