Skip to content

Semver-checks for all crate on merge and push #16078

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

logan-keede
Copy link
Contributor

@logan-keede logan-keede commented May 18, 2025

Which issue does this PR close?

Rationale for this change

Detect accidental breaking changes.

What changes are included in this PR?

A github workflow, that uses cargo semver-checks for all crates. divided into three parts because 1 job goes on for too long + crashes before completion(can probably be fixed by emptying/deleting cache in the middle.)

Are these changes tested?

On local branch by CI.

Are there any user-facing changes?

NO.

@github-actions github-actions bot added the development-process Related to development process of DataFusion label May 18, 2025
@logan-keede
Copy link
Contributor Author

cargo semver-checks is probably too heavy to run on every push but I have kept it for testing purpose.
I would also like to know other contributor/committer`s opinion.

@logan-keede logan-keede changed the title Semver-checks for all crate on merge and PR Semver-checks for all crate on merge and push May 18, 2025
@logan-keede logan-keede marked this pull request as ready for review May 19, 2025 12:08
@logan-keede
Copy link
Contributor Author

cc @crepererum

@logan-keede
Copy link
Contributor Author

cc @xudong963 @Omega359

@Omega359
Copy link
Contributor

I unfortunately do not have the time currently to do a proper review of this PR. Hopefully I'll have time in June if it's not reviewed by then.

Comment on lines +52 to +69
- name: Run semver-check datasource
run: cargo semver-checks check-release -p datafusion-datasource -p datafusion-datasource-avro -p datafusion-datasource-csv -p datafusion-datasource-json -p datafusion-datasource-parquet
continue-on-error: true
- name: Run semver-check common
run: cargo semver-checks check-release -p datafusion-common -p datafusion-common-runtime
continue-on-error: true
- name: Run semver-check catalog
run: cargo semver-checks check-release -p datafusion-catalog -p datafusion-catalog-listing
continue-on-error: true
- name: Run semver-check execution
run: cargo semver-checks check-release -p datafusion-execution -p datafusion-session
continue-on-error: true
- name: Run semver-check expr
run: cargo semver-checks check-release -p datafusion-expr -p datafusion-expr-common -p datafusion-physical-expr
continue-on-error: true
- name : Run semver-check functions
run: cargo semver-checks check-release -p datafusion-functions -p datafusion-functions-aggregate -p datafusion-functions-aggregate-common -p datafusion-functions-nested -p datafusion-functions-table -p datafusion-functions-window -p datafusion-functions-window-common
continue-on-error: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we can somehow automate this listing? Otherwise we will likely never update this list when we add new crates.

continue-on-error: true


semver-checks-2:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are there multiple jobs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because a single job runs out of disk space.

@crepererum
Copy link
Contributor

The check also fails currently but is marked as "green". I think this way it has little value, because nobody will likely check the logs.

@logan-keede
Copy link
Contributor Author

logan-keede commented May 26, 2025

The check also fails currently but is marked as "green". I think this way it has little value, because nobody will likely check the logs.

I think that is the only way to do it, otherwise it will always show red till you change the Version of the crate according to SemVer, which is not possible for each PR or merge.

It is more of a reviewer convenience thing to spot API changes IMO.

also it is more convenient to have all the API changes in job then to wait because it failed in one crate before.

@crepererum
Copy link
Contributor

I think the job should compare against current main, not the latest released version. Otherwise the logs are too noisy to be useful. Then the job should be red if it fails but a user may add a PR label (e.g. the existing "api breakage" label) to accept the breakage.

@logan-keede logan-keede marked this pull request as draft May 26, 2025 10:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
development-process Related to development process of DataFusion
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add CI check for cargo-semver-checks Use cargo semver-checks for release testing
3 participants