Add container jobs performing chores #31
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This will avoid repeating this operation multiple times. Technically auto[run|build|test].sh create a dir for output in 'spdk/../', but making it explicit makes it more clear. Additionally the output_dir is set by default one directory up from spdk. Use that to shorten the already unwieldy one-liner. Signed-off-by: Tomasz Zawadzki <[email protected]>
Some of the tests perform git operations on the submodules. Signed-off-by: Tomasz Zawadzki <[email protected]>
This is intended to match unittest-main-docker-autotest. Originaly this job was intended to run inside a container to make better use of resources than VMs. For jobs that do not run functional tests with SPDK. Meanwhile container images are not ready for spdk-ci, this is still executed in VM. Job was renamed to better match the intent: - 'main' to 'gcc', other jobs will be similar and have 'clang' and 'vagrant' - 'docker' to 'container' As for the steps, they do not use autorun.sh in order to skip parts that are required for functional tests - ex. setup.sh. On old CI setup it was demonstrably faster in execution, than just full autorun.sh. Yet it might not hold up today. To keep changes on SPDK repo to minimum, container jobs will still call functions from tests scripts directly. At some point it will need to re-evaluted if this is best approach. One note here is that $rootdir is usually set by autorun.sh script. Since we are not calling it, we need to set it manually. OCF is turned on, to match prior CI setup. Signed-off-by: Tomasz Zawadzki <[email protected]>
Signed-off-by: Tomasz Zawadzki <[email protected]>
s/main/gcc s/docker/container Signed-off-by: Tomasz Zawadzki <[email protected]>
Note that empty.conf was added to avoid creating empty config files for jobs that require no such file. We might want to revist this requirement on some of the scripts in SPDK. s/docker/container Signed-off-by: Tomasz Zawadzki <[email protected]>
Signed-off-by: Tomasz Zawadzki <[email protected]>
Signed-off-by: Tomasz Zawadzki <[email protected]>
Signed-off-by: Tomasz Zawadzki <[email protected]>
Tests like release build or scan-build jobs take around ~30min already, with longest running one I've seen at 32min. Let's extend it for now, and once we manage to get all required jobs in - figure out how to reduce the execution time. Signed-off-by: Tomasz Zawadzki <[email protected]>
Probably '*-autotest_artifacts' would be better in the future, but still need to change the nvme job to match. Signed-off-by: Tomasz Zawadzki <[email protected]>
karlatec
reviewed
Mar 7, 2025
| - name: autorun_nvme | ||
| run: | | ||
| /opt/spdk/autorun.sh /opt/configs/nvme.conf | ||
|
|
||
| - name: unittest-gcc-container-autotest | ||
| run: | | ||
| rootdir=/opt/spdk && cd $rootdir && source ./test/common/autobuild_common.sh /opt/configs/unittest-gcc-container-autotest.conf && ocf_precompile && unittest_build && ./test/unit/unittest.sh |
There was a problem hiding this comment.
Total nit: break the line a bit?
It's ~190 chars, so a bit unwieldy when displayed in editor with split scree.
There was a problem hiding this comment.
I see jobs in other commits having similar line length, so applies there as well.
There was a problem hiding this comment.
https://cijoe.readthedocs.io/en/latest/workflows/index.html#inline-commands
Each line is separate command intentionally. This implies all the variables being set and workdir has to be in one line.
I could have created separate script that does all of that, but then it goes a bit against the simplicity we wanted here. We are already three 'workflows' deep.
|
|
||
| - name: check-so-deps-container-autotest | ||
| run: | | ||
| rootdir=/opt/spdk && cd $rootdir && source ./test/common/autobuild_common.sh /opt/configs/check-so-deps-container-autotest.conf && ocf_precompile && SPDK_ABI_DIR=/opt/spdk-abi check_so_deps |
There was a problem hiding this comment.
That's a lot of repetitions of this rootdir=/opt/spdk && cd $rootdir && source ./test/common/autobuild_common.sh.
This is ci-joe specific config so I have no idea how to handle that. Using yaml's aliases may reduce the number of repetitions, but overall generates more total lines in this file (lol). Something to clean up later I guess.
karlatec
approved these changes
Mar 7, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds following jobs to the per-patch:
Series on SPDK here, along with image rebuild is required.
This part of the work on spdk/spdk#3615