Skip to content

Simpler / easier to see expressions in tree explain mode #15107

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Tracked by #14914
alamb opened this issue Mar 9, 2025 · 3 comments · Fixed by #15163
Closed
Tracked by #14914

Simpler / easier to see expressions in tree explain mode #15107

alamb opened this issue Mar 9, 2025 · 3 comments · Fixed by #15163
Assignees
Labels
enhancement New feature or request

Comments

@alamb
Copy link
Contributor

alamb commented Mar 9, 2025

Is your feature request related to a problem or challenge?

Thanks to @Standing-Man , tree explains are looking nice

The current display of expressions makes it possible to understand their inner structure (what is casted, etc)

However, it is quite hard for normal users to read. For example

| physical_plan | ┌───────────────────────────┐                                                                         |
|               | │       ProjectionExec      │                                                                         |
|               | │    --------------------   │                                                                         |
|               | │ foo.int_column + Int64(100│                                                                         |
|               | │             ):            │                                                                         |
|               | │ CAST(int_column@0 AS Int64│                                                                         |
|               | │          ) + 100          │                                                                         |
|               | │                           │                                                                         |
|               | │  substr(foo.string_column │                                                                         |
|               | │    ,Int64(1),Int64(2)):   │                                                                         |
|               | │ substr(string_column@1, 1,│                                                                         |
|               | │             2)            │                                                                         |
|               | └─────────────┬─────────────┘      

For example

> explain select int_column + 100, substr(string_column, 1, 2) from foo where string_column = 'bar';
+---------------+-------------------------------------------------------------------------------------------------------+
| plan_type     | plan                                                                                                  |
+---------------+-------------------------------------------------------------------------------------------------------+
| logical_plan  | Projection: CAST(foo.int_column AS Int64) + Int64(100), substr(foo.string_column, Int64(1), Int64(2)) |
|               |   Filter: foo.string_column = Utf8("bar")                                                             |
|               |     TableScan: foo projection=[int_column, string_column]                                             |
| physical_plan | ┌───────────────────────────┐                                                                         |
|               | │       ProjectionExec      │                                                                         |
|               | │    --------------------   │                                                                         |
|               | │ foo.int_column + Int64(100│                                                                         |
|               | │             ):            │                                                                         |
|               | │ CAST(int_column@0 AS Int64│                                                                         |
|               | │          ) + 100          │                                                                         |
|               | │                           │                                                                         |
|               | │  substr(foo.string_column │                                                                         |
|               | │    ,Int64(1),Int64(2)):   │                                                                         |
|               | │ substr(string_column@1, 1,│                                                                         |
|               | │             2)            │                                                                         |
|               | └─────────────┬─────────────┘                                                                         |
|               | ┌─────────────┴─────────────┐                                                                         |
|               | │      RepartitionExec      │                                                                         |
|               | └─────────────┬─────────────┘                                                                         |
|               | ┌─────────────┴─────────────┐                                                                         |
|               | │    CoalesceBatchesExec    │                                                                         |
|               | └─────────────┬─────────────┘                                                                         |
|               | ┌─────────────┴─────────────┐                                                                         |
|               | │         FilterExec        │                                                                         |
|               | │    --------------------   │                                                                         |
|               | │         predicate:        │                                                                         |
|               | │   string_column@1 = bar   │                                                                         |
|               | └─────────────┬─────────────┘                                                                         |
|               | ┌─────────────┴─────────────┐                                                                         |
|               | │       DataSourceExec      │                                                                         |
|               | │    --------------------   │                                                                         |
|               | │        bytes: 1320        │                                                                         |
|               | │       format: memory      │                                                                         |
|               | │          rows: 1          │                                                                         |
|               | └───────────────────────────┘                                                                         |
|               |                                                                                                       |
+---------------+-------------------------------------------------------------------------------------------------------+
2 row(s) fetched.
Elapsed 0.012 seconds.

Describe the solution you'd like

I would like the expressions to be nicely formatted

For example instead of above it would be great to see

| physical_plan | ┌───────────────────────────┐                                                                         |
|               | │       ProjectionExec      │                                                                         |
|               | │    --------------------   │                                                                         |
|               | │ foo.int_column + 100      │                                                                         |
|               | │                           │                                                                         |
|               | │ substr(string_column, 1, 2)│                                                                         |
|               | └─────────────┬─────────────┘      

Describe alternatives you've considered

PhysicalExpr already has Debug and Display impls with varying levels of detail.

One way to add another way of displaying PhysicalExpr, would be to add a new method to the PhysicalExpr trait for "simplified display"

Maybe something like

    fn simply_fmt(&self, f: &mut Formatter<'_>) -> Result;

We could then make this work with normal write! style macros

Additional context

No response

@alamb alamb added the enhancement New feature or request label Mar 9, 2025
@alamb
Copy link
Contributor Author

alamb commented Mar 9, 2025

If anyone is interested in this I think the first thing to do would be to get a function like this working:

fn simple_display(expr: &dyn PhysicalExpr) -> String

Then we can integrate it into the PhysicalExpr trait

@alamb alamb changed the title Simpler / easier to see expressions in tree explain mode Simpler / easier to see expressions in tree explain mode Mar 9, 2025
@irenjj
Copy link
Contributor

irenjj commented Mar 9, 2025

take

@alamb
Copy link
Contributor Author

alamb commented Mar 10, 2025

I was thinking about this last night -- I was thinking a good name for this function might be "fmt_sql` or something -- with the goal that the expressions would (eventually) be valid SQL expressions

That would be human readable and might have potential value for other usecases

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants