Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sqlite 去掉 Nondeterministic & 生成result #1

Open
hazelnut-99 opened this issue May 2, 2024 · 1 comment
Open

sqlite 去掉 Nondeterministic & 生成result #1

hazelnut-99 opened this issue May 2, 2024 · 1 comment

Comments

@hazelnut-99
Copy link
Owner

hazelnut-99 commented May 2, 2024

background:
我把sqlite的test cases清理好啦 https://github.com/hazelnut-99/SQL-dialects/tree/main/test-suite-generated/sqlite
目录结构是这样:
db.sql是这个目录下所有测试用例共用的setup语句
然后每个测试用例有一个自己的目录
image

todo

  1. 把每个测试用例执行N次 (maybe N=3),去掉 Nondeterministic 的 test cases
  2. 每个测试用例生成一个result.txt文件作为后续结果比较的基准
    这个问题比较open-ended, 需要设计一下查询结果的存储格式 (我直觉的一个方案是按csv格式存 然后后面比较的时候比文件的md5, maybe you have better ideas)
@yimingwang123
Copy link
Collaborator

yimingwang123 commented May 4, 2024

todo

把每个测试用例执行N次 (maybe N=3),去掉 Nondeterministic 的 test cases
=> done
每个测试用例生成一个result.txt文件作为后续结果比较的基准
这个问题比较open-ended, 需要设计一下查询结果的存储格式 (我直觉的一个方案是按csv格式存 然后后面比较的时候比文件的md5, maybe you have better ideas)
=> saved in sql_results.zip
=> for each test case, one result.csv for human reading, one result.md5 for hashing comparison with other databases

yiming's notes:
1. 如果将来要跑别的database来和sqlite的结果相对比, 只需要改变run-test.py里面的line 15 line 27的sqlite3就可
2. 我们将来的results都可以存成.zip,因为我们的results file数量太多,git直接卡死在commit阶段😂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants