Skip to content

Implement a dbgen compatible command line API #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
alamb opened this issue Mar 10, 2025 · 3 comments
Closed

Implement a dbgen compatible command line API #9

alamb opened this issue Mar 10, 2025 · 3 comments
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@alamb
Copy link
Collaborator

alamb commented Mar 10, 2025

The tpch data generator also includes a command line utility:

For example, to make scale factor 0.1 (100M):

 ./dbgen -f -s 0.1

The code and flag meaning is on https://github.com/databricks/tpch-dbgen

Here are some examples if its use from the datafusion repo:

          mkdir -p datafusion/sqllogictest/test_files/tpch/data
          git clone https://github.com/databricks/tpch-dbgen.git
          cd tpch-dbgen
          make
          ./dbgen -f -s 0.1
          mv *.tbl ../datafusion/sqllogictest/test_files/tpch/data

https://github.com/apache/datafusion/blob/9382add72b929c553ca4976d1423d8ebbc80889d/benchmarks/bench.sh#L358-L357

@clflushopt
Copy link
Owner

That's next after I add a test suite thanks for opening the issue I forgot to add it for tracking !

@clflushopt clflushopt self-assigned this Mar 11, 2025
@clflushopt clflushopt added the enhancement New feature or request label Mar 11, 2025
@clflushopt clflushopt added this to the v0.1.0 milestone Mar 11, 2025
@alamb
Copy link
Collaborator Author

alamb commented Mar 11, 2025

I would love to help with this -- and I will with wahtever is needed once I have made a bit more progress on

my current obsession:

@clflushopt
Copy link
Owner

Closed in #12 any further work to improve the UX will be via a separate issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants