Skip to content

OTA-1403: Update and Verify Test Metadata Using Makefile #1179

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 15, 2025
Merged

OTA-1403: Update and Verify Test Metadata Using Makefile #1179

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
289e976
hack: Extract build variables to separate file
DavidHurta May 6, 2025
54da06c
hack: Create script to update test metadata
DavidHurta May 6, 2025
15bdbcc
Makefile: Add update target
DavidHurta May 6, 2025
8a5364f
Run `make update`
DavidHurta May 6, 2025
a90e78e
Makefile: Add verify-update target
DavidHurta May 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .openshift-tests-extension/openshift_payload_cluster-version-operator.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The .openshift-tests-extension/openshift_payload_cluster-version-operator.json path here diverges from the hack/.openshift-tests-exension/product/type/component/* pattern claimed by the extension you'd linked from the pull-request opening comment. Is the extension text stale? Is our cluster-version-operator-tests binary feeding the wrong location? Other? I guess the update logic living in a vendored directory tips us towards "enhancement text is stale"?

Copy link
Contributor Author

@DavidHurta DavidHurta May 14, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the text in the enhancement is stale. The .openshift-tests-extension/ directory is present at the root of a repository in openshift/kubernetes and in openshift-eng/openshift-tests-extension as well.

No --basedir flag (mentioned by the enhancement) is present for the update subcommand as of the last package bump.

$ ./../_output/linux/amd64/cluster-version-operator-tests update --help
Update test metadata

Usage:
   update [flags]

Flags:
      --component string   specify the component to enable (default "default")
  -h, --help               help for update

We can execute the extension while being in the hack/ directory to create the .openshift-tests-extension/ directory there. I do not have a strong opinion regarding the placement.

{
"name": "[cvo-testing] cluster-version-operator-tests should support passing tests",
"labels": {},
"resources": {
"isolation": {}
},
"source": "openshift:payload:cluster-version-operator",
"lifecycle": "blocking",
"environmentSelector": {}
}
]
10 changes: 9 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,17 +15,25 @@ integration-test:
./hack/integration-test.sh
.PHONY: integration-test

update: build
hack/update-test-metadata.sh
.PHONY: update

format:
go fmt ./...
.PHONY: format

verify: verify-yaml
verify: verify-yaml verify-update
.PHONY: verify

verify-yaml:
hack/verify-yaml.sh
.PHONY: verify-yaml

verify-update: update
git diff --exit-code HEAD
.PHONY: verify-update

clean:
rm -rf _output/
rm -rf bin
Expand Down
22 changes: 2 additions & 20 deletions hack/build-go.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,11 @@

set -eu

REPO=github.com/openshift/cluster-version-operator
GOFLAGS=${GOFLAGS:--mod=vendor}
GLDFLAGS=${GLDFLAGS:-}

eval $(go env | grep -e "GOHOSTOS" -e "GOHOSTARCH")

GOOS=${GOOS:-${GOHOSTOS}}
GOARCH=${GOARCH:-${GOHOSTARCH}}

# Go to the root of the repo
cd "$(git rev-parse --show-cdup)"

VERSION_OVERRIDE=${VERSION_OVERRIDE:-${OS_GIT_VERSION:-}}
if [ -z "${VERSION_OVERRIDE:-}" ]; then
echo "Using version from git..."
VERSION_OVERRIDE=$(git describe --abbrev=8 --dirty --always)
fi

eval $(go env)

if [ -z ${BIN_PATH+a} ]; then
export BIN_PATH=_output/${GOOS}/${GOARCH}
fi
# Source build variables
source hack/build-info.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering if there was a way to set up this import without relying on "assume build-go.sh is being called from the repository's root directory". But it seems like there may not be, so probably fine to stick with that assumption and not worry about the BASH_SOURCE and similar semi-reliable approaches.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we are probably fine with the assumption and the given nature of how the scripts are mainly executed (via make while being in the repository's root directory.

Assuming the build-info.sh is run from the repository, maybe we can do:

build-info.sh:

#!/usr/bin/env bash

set -eu

REPO=github.com/openshift/cluster-version-operator
GOFLAGS=${GOFLAGS:--mod=vendor}
GLDFLAGS=${GLDFLAGS:-}

eval $(go env | grep -e "GOHOSTOS" -e "GOHOSTARCH")

GOOS=${GOOS:-${GOHOSTOS}}
GOARCH=${GOARCH:-${GOHOSTARCH}}

VERSION_OVERRIDE=${VERSION_OVERRIDE:-${OS_GIT_VERSION:-}}
if [ -z "${VERSION_OVERRIDE:-}" ]; then
	echo "Using version from git..."
	VERSION_OVERRIDE=$(git describe --abbrev=8 --dirty --always)
fi

eval $(go env)

if [ -z ${BIN_PATH+a} ]; then
-	export BIN_PATH=_output/${GOOS}/${GOARCH}
+       export BIN_PATH=$(git rev-parse --show-toplevel)_output/${GOOS}/${GOARCH}
fi
       --show-toplevel
           Show the (by default, absolute) path of the top-level directory of the working tree. If there is no working tree, report an error.

To obtain the absolute path.


echo "Building binaries into ${BIN_PATH}"
mkdir -p ${BIN_PATH}
Expand Down
24 changes: 24 additions & 0 deletions hack/build-info.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/usr/bin/env bash

set -eu

REPO=github.com/openshift/cluster-version-operator
GOFLAGS=${GOFLAGS:--mod=vendor}
GLDFLAGS=${GLDFLAGS:-}

eval $(go env | grep -e "GOHOSTOS" -e "GOHOSTARCH")

GOOS=${GOOS:-${GOHOSTOS}}
GOARCH=${GOARCH:-${GOHOSTARCH}}

VERSION_OVERRIDE=${VERSION_OVERRIDE:-${OS_GIT_VERSION:-}}
if [ -z "${VERSION_OVERRIDE:-}" ]; then
echo "Using version from git..."
VERSION_OVERRIDE=$(git describe --abbrev=8 --dirty --always)
fi

eval $(go env)

if [ -z ${BIN_PATH+a} ]; then
export BIN_PATH=_output/${GOOS}/${GOARCH}
fi
8 changes: 8 additions & 0 deletions hack/update-test-metadata.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/usr/bin/env bash

set -eu

source hack/build-info.sh

# Update test metadata
eval "${BIN_PATH}/cluster-version-operator-tests" "update"