Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI Improvements #125

Merged
merged 15 commits into from
Nov 21, 2024
Merged

CI Improvements #125

merged 15 commits into from
Nov 21, 2024

Conversation

brenns10
Copy link
Member

@brenns10 brenns10 commented Oct 30, 2024 •
edited
Loading

This pull request contains several improvements to our CI.

  1. Removing tox, which hasn't been useful, and it's a bit confusing and inflexible. This simplifies a fair bit of our CI and speeds things up too.
  2. Enabling CTF testing in the two live test case type (litevm and heavyvm). We weren't actually doing any CTF testing against live kernels in CI previously.
  3. Dropping dependencies for the test suite. The goal is that the tests that developers may want to run (litevm, vmcore) should only require python, drgn, and pytest, all of which could even be satisfied by the Yum versions. To do this, I've dropped the pytest-cov (which was causing issues in CI for no benefit), junitparser (it can be replaced by a few lines of XML parsing).
  4. Discovered that CTF is broken for UEK4, enough that we shouldn't mark it as supported.
  5. Added CTF testing in Github, by building drgn from source. (This necessitated removing Python 3.6 from the Github tests, but is well covered elsewhere)
  6. Eliminating the OCI SDK from the vmcore test runner, and allowing it to run with a PAR (pre-authenticated URL) which has far less access than an SDK API key. This is a security improvement, but also of course makes it simpler for developers to run the vmcore tests because they don't need the third-party dependencies.

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Oct 30, 2024
@brenns10 brenns10 force-pushed the remove_coverage branch 5 times, most recently from 597def6 to ec0f0cd Compare October 30, 2024 23:03
@brenns10
testing: remove coverage
174d010 
The Gitlab CI tests have intermittently failed for vmcore tests due to
race conditions with the parallel execution for coverage. The fact of
the matter is that the coverage information is not currently reported by
any of our CI tests, and no developers use it locally. It's just more
trouble than it's worth, so remove it.

Orabug: 37307170
Signed-off-by: Stephen Brennan <[email protected]>
@brenns10
testing: rpm: call sys.executable
64a0410 
Using "python3" runs the risk of a different python version getting
used. The whole point of testing/rpm.py is to run *within* the current
Python environment, so use sys.executable to make that certain.

Orabug: 37307170
Signed-off-by: Stephen Brennan <[email protected]>
@brenns10
testing: unpin oci-cli
972ce91 
There have been many releases since the annotated ones. It should be
fine to unpin now.

Orabug: 37307170
Signed-off-by: Stephen Brennan <[email protected]>
@brenns10
tests: heavyvm: run with CTF as well
c7073f9 
Currently the only CTF-related tests for CI are the ones which run
against vmcores. The heavyvm tests can easily support CTF, so run it. It
should not add enough time to cause any issues with CI.

Orabug: 37307170
Signed-off-by: Stephen Brennan <[email protected]>
@brenns10
testing: heavyvm: skip CTF for UEK4
afd847b 
The CTF data for UEK4 is fully broken; we cannot support it. Skip
running live tests for UEK4 CTF.

Orabug: 37307170
Signed-off-by: Stephen Brennan <[email protected]>
@brenns10
conftest: fix test suite names for heavyvm CTF
2aafc80 
The test suite name was being reported as "CTF" only if the tests were
run against vmcores. With heavyvm (and possibly litevm) tests running
against CTF, this means that the XML test reports no longer contain the
correct debuginfo in their name.

Live tests also simply report "live" which makes it very difficult to
understand which test suite is for which version. Report the uname in
this case, which includes both UEK and OL version, though it's not as
nice as simply saying "OLX UEKY".

Orabug: 37307170
Signed-off-by: Stephen Brennan <[email protected]>
@brenns10
testing: stop using tox
12d1d0c 
The value proposition of tox is allowing you to run your tests against a
lot of different Python versions at the same time. It can be really
useful, but honestly it hasn't been helpful in drgn-tools. We typically
only have one (maybe two) Python versions available, and frequently we
can only run tests against one anyway. Testing multiple Python versions
is done in Github CI, but tricking tox into only running tests for one
version is actually a pain!

All told, it's caused more pain than gain, so let's do away with it in
testing. The tox.ini still exists and is used for generating docs, but
we can eliminate that smaller issue in a subsequent commit.

Orabug: 37307170
Signed-off-by: Stephen Brennan <[email protected]>
@brenns10
testing: remove junitparser
862011f 
This is a totally unnecessary dependency. We only need to have one
operation: concatenating two test results. Thankfully, that can be done
directly with the python XML support. Drop the dependency.

Orabug: 37307170
Signed-off-by: Stephen Brennan <[email protected]>
@brenns10
testing.rpm: remove the --coverage and --xml
bbabbc7 
The coverage option has been removed from the rest of the tests. It
looks like the xml option was simply never implemented.

Orabug: 37307170
Signed-off-by: Stephen Brennan <[email protected]>
@brenns10
Mark UEK4 as incompatible with CTF
f179be8 
Orabug: 37307170
Signed-off-by: Stephen Brennan <[email protected]>
@brenns10
actions: Remove 3.6 testing, add CTF testing
18278c1 
Orabug: 37307170
Signed-off-by: Stephen Brennan <[email protected]>
@brenns10
testing: litevm: fix litevm runner
075ecfe 
It got broken by the removal of tox. It also needs to handle CTF in the
same run for some cases too. Fix it all up.

Orabug: 37307170
Signed-off-by: Stephen Brennan <[email protected]>
@brenns10
testing: add parlib for OCI pre-authenticated requests
7415bff 
Pre-authenticated requests allow a user to download and upload files to
OCI Object Storage without needing to have a full OCI SDK and API key
setup. More importantly, they can do it without sharing somebody's API
keys. They are a great way to grant tightly scoped access to a shared
bucket or bucket prefix, for things like vmcore storage in a shared
environment.

Orabug: 37307170
Signed-off-by: Stephen Brennan <[email protected]>
@brenns10
testing.vmcore: split out management and simplify
a0db0d8 
The testing.vmcore system requires the OCI SDK to be installed, which
has its own dependencies. Mostly, this is due to the management system
relying on OCI Object Storage, but also we require the "rich" library
for progress bars.

It really doesn't make sense to have extra third-party dependencies
required for testing. They add runtime during CI, there are security
questions, and it is more confusing for developers. Further, using the
OCI SDK for Object Storage requires a full set of OCI API keys, which is
not really suitable for shared systems.

So this commit does two things: first, it splits testing.vmcore into two
sub-components: testing.vmcore.test and testing.vmcore.manage. The
former actually runs the tests, and the latter manages vmcores. The
second task is to migrate vmcore management from the OCI SDK to the
parlib tool introduced in the previous commit. This drops several
dependencies. Further, we drop the rich dependency, resulting in a very
minimal testing script with no more dependencies than necessary.

Orabug: 37307170
Signed-off-by: Stephen Brennan <[email protected]>
@brenns10 brenns10 marked this pull request as ready for review November 20, 2024 23:47
@brenns10
testing: ensure we provide the test dir to pytest
304f0a5 
The RPM and vmcore test runners omitted the "tests" directoy. This meant
that pytest was searching the entire working directory for tests. This
is fine in CI tests, because there isn't much extra data in the git
checkout. But for local/developer test runs, this is a problem, because
the "testdata" directory contains lots of extra data, including
extracted RPMs which have some python code. This causes errors when
pytest attempts to collect and run tests from those files.

The heavyvm and litevm tests already provide the tests/ directory as a
command line argument.

Orabug: 37307170
Signed-off-by: Stephen Brennan <[email protected]>
@brenns10 brenns10 changed the title [draft] CI Improvements CI Improvements Nov 21, 2024
@brenns10 brenns10 merged commit 61704f2 into oracle-samples:main Nov 21, 2024
4 checks passed
@brenns10 brenns10 deleted the remove_coverage branch November 21, 2024 17:40
@brenns10 brenns10 mentioned this pull request Nov 21, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@biger410 biger410 biger410 approved these changes

Assignees
No one assigned
Labels
OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@brenns10 @biger410