Skip to content

datafusion-proto crate feature json is broken #15157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
alamb opened this issue Mar 11, 2025 · 4 comments · Fixed by #15172
Closed

datafusion-proto crate feature json is broken #15157

alamb opened this issue Mar 11, 2025 · 4 comments · Fixed by #15172
Labels
bug Something isn't working

Comments

@alamb
Copy link
Contributor

alamb commented Mar 11, 2025

Describe the bug

I am not sure what the json feature is supposed to do in datafusion-proto but it is broken

To Reproduce

cargo check --profile ci --all-targets --no-default-features -p datafusion-proto --features=json

Results in pages of errors like this

error[E0277]: the trait bound `datafusion_proto_common::Schema: Serialize` is not satisfied
    --> datafusion/proto/src/generated/pbjson.rs:159:55
     |
159  |             struct_ser.serialize_field("inputSchema", v)?;
     |                        ---------------                ^ the trait `Serialize` is not implemented for `datafusion_proto_common::Schema`
     |                        |
     |                        required by a bound introduced by this call
     |
     = note: for local types consider adding `#[derive(serde::Serialize)]` to your `datafusion_proto_common::Schema` type
     = note: for types from other crates check whether the crate offers a `serde` feature flag
     = help: the following other types implement trait `Serialize`:
               &'a T
               &'a mut T
               ()
               (T,)
               (T0, T1)
               (T0, T1, T2)
               (T0, T1, T2, T3)
               (T0, T1, T2, T3, T4)
             and 303 others
note: required by a bound in `serde::ser::SerializeStruct::serialize_field`
    --> /usr/local/cargo/registry/src/index.crates.io-1949cf8c6b5b557f/serde-1.0.218/src/ser/mod.rs:18[66](https://github.com/apache/datafusion/actions/runs/13788882508/job/38563713616?pr=15156#step:6:67):21
     |
1864 |     fn serialize_field<T>(&mut self, key: &'static str, value: &T) -> Result<(), Self::Error>
     |        --------------- required by a bound in this associated function
1865 |     where
1866 |         T: ?Sized + Serialize;
     |                     ^^^^^^^^^ required by this bound in `SerializeStruct::serialize_field`

error[E0277]: the trait bound `datafusion_proto_common::Schema: Deserialize<'_>` is not satisfied
    --> datafusion/proto/src/generated/pbjson.rs:320:51
     |
320  | ...                   input_schema__ = map_.next_value()?;
     |                                             ^^^^^^^^^^ the trait `Deserialize<'_>` is not implemented for `datafusion_proto_common::Schema`
     |
     = note: for local types consider adding `#[derive(serde::Deserialize)]` to your `datafusion_proto_common::Schema` type
     = note: for types from other crates check whether the crate offers a `serde` feature flag
     = help: the following other types implement trait `Deserialize<'de>`:
               &'a [u8]
               &'a std::path::Path
               &'a str
               ()
               (T,)
               (T0, T1)
               (T0, T1, T2)
               (T0, T1, T2, T3)
             and 4[80](https://github.com/apache/datafusion/actions/runs/13788882508/job/38563713616?pr=15156#step:6:81) others
     = note: required for `std::option::Option<datafusion_proto_common::Schema>` to implement `Deserialize<'de>`
note: required by a bound in `next_value`

See more here https://github.com/apache/datafusion/actions/runs/13788882508/job/38563713616?pr=15156

Expected behavior

I expect the feature to work, or to be removed

Additional context

Found while working on

@timsaucer
Copy link
Contributor

Recommend adding to this issue that we will want to re-enable the Check datafusion-proto (json) as part of closing the issue

@apache apache deleted a comment from piyushgarg5021 Mar 11, 2025
@timsaucer
Copy link
Contributor

Deleting comments that appear to be from an AI model.

@Shreyaskr1409
Copy link
Contributor

https://github.com/apache/datafusion/pull/15185/files#r1991797793
#15089 (comment)

@irenjj maybe it is due to this issue, we are facing problems with JsonSink #15089 , as I said in one of my comments, no matter what I did, JsonSink is not getting used whereas CsvSink and others do get used. Maybe we can look into that issue after this issue is solved.

@irenjj
Copy link
Contributor

irenjj commented Mar 13, 2025

https://github.com/apache/datafusion/pull/15185/files#r1991797793 #15089 (comment)

@irenjj maybe it is due to this issue, we are facing problems with JsonSink #15089 , as I said in one of my comments, no matter what I did, JsonSink is not getting used whereas CsvSink and others do get used. Maybe we can look into that issue after this issue is solved.

Thanks @Shreyaskr1409 , explain works in #15185(even though I wasn't able to construct a non-empty file_group case), I'd like to try to add more tests after this issue resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants