Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(nix): Setup a head build for the cross_compatibility integ test #4567

Merged
merged 22 commits into from
Jul 26, 2024
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 35 additions & 16 deletions codebuild/bin/install_s2n_head.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/bash
#!/usr/bin/env bash
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License").
Expand All @@ -12,31 +12,50 @@
# express or implied. See the License for the specific language governing
# permissions and limitations under the License.

set -ex
pushd "$(pwd)"
set -e

usage() {
echo "install_s2n_head.sh build_dir"
exit 1
}

BUILD_DIR=$1
SRC_ROOT=${SRC_ROOT:-$(pwd)}

if [ "$#" -ne "1" ]; then
usage
fi

BUILD_DIR=$1
source codebuild/bin/jobs.sh
cd "$BUILD_DIR"

# Clone the most recent s2n commit
git clone --depth=1 https://github.com/aws/s2n-tls s2n_head
cmake ./s2n_head -Bbuild -DCMAKE_PREFIX_PATH="$LIBCRYPTO_ROOT" -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_SHARED_LIBS=on -DBUILD_TESTING=on
cmake --build ./build -- -j $JOBS

# Copy new executables to bin directory
cp -f "$BUILD_DIR"/build/bin/s2nc "$BASE_S2N_DIR"/bin/s2nc_head
cp -f "$BUILD_DIR"/build/bin/s2nd "$BASE_S2N_DIR"/bin/s2nd_head
# CMake(nix) and Make are using different directory structures.
if [[ "$IN_NIX_SHELL" ]]; then
export DEST_DIR="$SRC_ROOT"/build/bin
# Safety measure
mkdir -p "$DEST_DIR"
else
export DEST_DIR="$SRC_ROOT"/bin
fi

popd
if [[ ! -x "$DEST_DIR/s2nc_head" ]]; then
if [[ ! -d "s2n_head" ]]; then
# Clone the most recent s2n commit
git clone --branch main --single-branch . s2n_head
else
cd s2n_head
echo "Checking the age of s2n_head..."
test $(date -d '-3 days' +%s) -lt $(git log -1 --format="%at") || echo "s2n_head is too old, refusing to use it";exit 1
cd ..
fi
if [[ "$IN_NIX_SHELL" ]]; then
cmake ./s2n_head -B"$BUILD_DIR" -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_SHARED_LIBS=on -DBUILD_TESTING=on
else
cmake ./s2n_head -B"$BUILD_DIR" -DCMAKE_PREFIX_PATH="$LIBCRYPTO_ROOT" -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_SHARED_LIBS=on -DBUILD_TESTING=on
fi
cmake --build "$BUILD_DIR" -- -j "$(nproc)"
# Copy head executables for make build
cp -f "$BUILD_DIR"/bin/s2nc "$DEST_DIR"/s2nc_head
cp -f "$BUILD_DIR"/bin/s2nd "$DEST_DIR"/s2nd_head
else
echo "s2nc_head already exists; not rebuilding s2n_head"
fi

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will this work locally / with nix? Like, I'm imagining that I'm developing a feature that requires me to run the cross_compatibility test. s2n_head could be quite old if I haven't run clean lately, and my understanding of this behavior is that I wouldn't really have any visibility into that beyond this one stdout message in the big block of build stdout messages.

Maybe we don't have to build every time, but should we have some sort of freshness check? Checking the version against github might be a bit too much-- Maybe as simple as the creation time for s2nc_head? At the bare minimum I'd try to add some visibility to this message, maybe noting which commit is in s2nc_head, how to force s2nc_head to rebuild, maybe some color.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had started down the path of checking s2n_head's commit, but thought, at least initially, it's a waste of time since the CI will always be running on a clean instance. Checking the age of the last commit in s2n_head might be lighter weight, let me try a few things.

exit 0
11 changes: 4 additions & 7 deletions nix/shell.sh
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ function build {
banner "Running Build"
javac tests/integrationv2/bin/SSLSocketClient.java
cmake --build ./build -j $(nproc)
# Build s2n from HEAD
$SRC_ROOT/codebuild/bin/install_s2n_head.sh $(mktemp -d)
}

function unit {
Expand All @@ -60,21 +62,16 @@ function unit {
function integ {
if [ "$1" == "help" ]; then
echo "The following tests are not supported:"
echo " - cross_compatibility"
echo " This test depends on s2nc_head and s2nd_head. To run"
echo " the test build s2n-tls from the main branch on github."
echo " Change the names of s2n[cd] to s2n[cd]_head and add those"
echo " binaries to \$PATH."
echo "- renegotiate_apache"
echo " This test requires apache to be running. See codebuild/bin/s2n_apache.sh"
echo " for more info."
return
fi
if [[ -z "$1" ]]; then
banner "Running all integ tests except cross_compatibility, renegotiate_apache."
banner "Running all integ tests except renegotiate_apache."
(cd $SRC_ROOT/build; ctest -L integrationv2 -E "(integrationv2_cross_compatibility|integrationv2_renegotiate_apache)" --verbose)
else
banner "Warning: cross_compatibility & renegotiate_apache are not supported in nix for various reasons integ help for more info."
banner "Warning: renegotiate_apache is not supported in nix for various reasons integ help for more info."
for test in $@; do
ctest --test-dir ./build -L integrationv2 --no-tests=error --output-on-failure -R "$test" --verbose
if [ "$?" -ne 0 ]; then
Expand Down
Loading