You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I observed this recently in #12429 (review). This likely means we have forgotten to document some functions or that the documentation has drifted over time
Also this means the help text for various functions can only be found on the DataFusion website, and not, for example within the function itself.
It would be awesome if you could do something like this from SQL:
> DESCRIBE sqrt;
Returns the square root of a number.
sqrt(numeric_expression)
Arguments
* numeric_expression: Numeric expression to operate on. Can be a constant, column, or function, and any combination of arithmetic operators.
Then we can work in multiple PRs to port the remaining documentation over to the code (which will automatically result in the new page getting updated)
And then finally we can remove the old page when all functions are ported.
If we start working on this project, we (I) can file follow tickets to track porting the remaining functions / doing the same thing for aggregate functions, etc.
Additional context
Also, similarly, GlareDB has a way to automatically annotate functions with documentation, and @universalmind303 proposed something similar here #8366
Also, @findepi is considering implementing SHOW FUNCTIONS as part of #12144 that could also likely take advantage of this documentation if it was present
The text was updated successfully, but these errors were encountered:
I took a quick look at this and I believe we need to add a doc_category for each udf to be able to slot it into the appropriate section in the documentation. For example, for scalar udf's that could be math, conditional, string, etc
The doc_category fn could return either be a simple string or more properly an enum, one for each type of UDF (scalar, aggregate, window, ...)
I don't like the fn name 'doc_category' but I couldn't come up with something better.
Is your feature request related to a problem or challenge?
When we add a new function to datafusion's library we have to remember to document that function in the documentation, for example in https://datafusion.apache.org/user-guide/sql/scalar_functions.html
I observed this recently in #12429 (review). This likely means we have forgotten to document some functions or that the documentation has drifted over time
Also this means the help text for various functions can only be found on the DataFusion website, and not, for example within the function itself.
It would be awesome if you could do something like this from SQL:
Describe the solution you'd like
I would like:
DataFusion already does something like this for
ConfigOptions
For example, the comments in https://docs.rs/datafusion/latest/datafusion/config/struct.SqlParserOptions.html are automatically added to the documentation programatically:
datafusion/datafusion/core/src/bin/print_config_docs.rs
Line 4 in 3ece7a7
Describe alternatives you've considered
I suggest this as a high level approach
ScalarUDFImpl
trait as proposed by @universalmind303 in add examples and description to scalar/aggregate functions? #8366ScalarUDFImpl::description
andScalarUDFImpl::sql_example
ConfigOptions
to generate the sql reference from those functions.In terms of implementation order I would personally suggest breaking this project into smaller parts:
A first PR that does:
Then we can work in multiple PRs to port the remaining documentation over to the code (which will automatically result in the new page getting updated)
And then finally we can remove the old page when all functions are ported.
If we start working on this project, we (I) can file follow tickets to track porting the remaining functions / doing the same thing for aggregate functions, etc.
Additional context
Also, similarly, GlareDB has a way to automatically annotate functions with documentation, and @universalmind303 proposed something similar here #8366
Also, @findepi is considering implementing
SHOW FUNCTIONS
as part of #12144 that could also likely take advantage of this documentation if it was presentThe text was updated successfully, but these errors were encountered: