From 4ce937148d3989112087888d0f34b56f7aa0cc25 Mon Sep 17 00:00:00 2001 From: Prabhjot Singh <86856448+psbrar99@users.noreply.github.com> Date: Mon, 7 Aug 2023 11:47:14 -0600 Subject: [PATCH] Tetrate workflow v1 (#1018) * Retry when east-west gateway hostname resolution fails (#44192) * retry when gateway hostnames resolution failed * add a ut * add a release note * test * test retry period * test * test * update retry period * distinguish server failure * fix * fix ttl * remove named return values * address comments * Automator: update proxy@master in istio/istio@master (#44295) * gateway: skip un-managed types in deploymentcontroller (#43541) * Automator: update proxy@master in istio/istio@master (#44298) * Add nil-check for WASM validation (#44296) * kube-probe: avoid duplicate probe headers (#44297) This is a regression of https://github.com/istio/istio/issues/28466 from https://github.com/istio/istio/pull/31866 The 31866 fixed the k8s spec having duplicates, but we end up duplicating all headers twice - the kubelet adds the header from HTTPHeaders and we also add from HTTPHeaders, resulting in 2x every header. Instead, we simply copy the incoming request headers and host, making us a fairly transparent proxy. * Refactor multi network manage (#44190) * Refactor multi network * Change reloadXX to networkManager's private method * Automator: update go-control-plane in istio/istio@master (#44301) * Automator: update proxy@master in istio/istio@master (#44302) * Automator: update common-files@master in istio/istio@master (#44305) * skip gateway.istio.io/controller-version anno message that is not created by users (#44306) * Automator: update istio/client-go@master dependency in istio/istio@master (#44312) * Fix the metric and log reports for the dryrun gateway config (#44303) * fix reporting metric for dryrun gateway config * add releasnotes * improve log * Automator: update common-files@master in istio/istio@master (#44314) * Early update_deps.sh before branching (#44317) * e2e: refactor grpc stats test (#44307) * e2e: refactor grpc stats test * fix gen * fix build * fix analyze * fix buildGRPCQuery Signed-off-by: hejianpeng --------- Signed-off-by: hejianpeng * Automator: update proxy@master in istio/istio@master (#44320) * Fix system namespace filtering (#44325) * update istio.io/api (#44330) * validation: add another nil check (#44332) * Fix type at shared.go (#44326) Signed-off-by: jongwooo * Update istio.io modules (#44331) * Update istio.io modules * Run `mske gen` * Add update_ztunnel script (#44334) * remove sync all (#44111) * remove syncall * update * update * Fix namespace equal check * Fix ClusterExternalAddresses race * Fix * Add bookinfo demo for PSA (#44214) Signed-off-by: Kalya Subramanian * Integration tests for ENABLE_ENHANCED_RESOURCE_SCOPING feature (#44246) * Integration tests for ENABLE_ENHANCED_RESOURCE_SCOPING feature Signed-off-by: Faseela K * fix lint Signed-off-by: Faseela K * add build tag Signed-off-by: Faseela K * fix lint Signed-off-by: Faseela K * skip VMs Signed-off-by: Faseela K * Require singlecluster Signed-off-by: Faseela K * SkipExternalControlPlaneTopology Signed-off-by: Faseela K * Require Multi Primary Signed-off-by: Faseela K * add build tag Signed-off-by: Faseela K * replace httpbin.org with echo external service Signed-off-by: Faseela K --------- Signed-off-by: Faseela K * Update deps 2023 04 10 (#44319) * ./bin/update_deps.sh * update go deps * Remove default.yaml * Make gen * Revert change to distroless * Fix controller runtime * Update istio.io/api and istio.io/pkg * Run go mod tidy * Update release prow * Update istio.io/api * Automator: update istio/client-go@master dependency in istio/istio@master (#44339) * Use official 0.27.0 release (#44340) * reduced tokenWaitBackoff from 1 second to 10 millis (#44338) * Fix the ztunnel iop specification for resources leads to a patch error. (#44322) * fix ztunnel iop resource * add test * Automator: update proxy@master in istio/istio@master (#44343) * Fix ServiceEntry WorkloadInstanceHandler for label change in Pods (#42922) * Fix ServiceEntry WorkloadInstanceHandler for label change in Pods For a label change in WorkloadInstance, which results in a mismatch with a previously matching ServiceEntry, the whole update event is skipped. Which results in the WorkloadInstance not getting removed from the ServiceEntry. This fix changes the behaviour for such cases to handle those updates similar to a delete event and clean up the WorkloadInstance from the list. Fixes [#42921] * use DeepEquals to avoid SubsetOf iteration * gocritic linting fix * go back to the first approach * fix workloadInstanceDiff * fix linting errors * change test cases to table format * fix testcase for new Event type * fix Event Type * go fmt * remove wi diff * Update BASE_VERSION to master-2023-04-12T19-02-00 (#44357) * Add docs for Calico, set the annotation by default. (#44259) * Add docs for Calico, set the annotation by default. * Finish the comment. * Remove debug, kubectl debug works * indent * Update manifests/charts/istio-cni/README.md --------- Co-authored-by: John Howard * update_deps.sh change (#44373) * fix gateway service name (#44365) * Automator: update proxy@master in istio/istio@master (#44380) * validation: add small nil check (#44360) * Remove kustomize and precompute manifests (#44376) See https://github.com/istio/istio/issues/44237 for more info * Add a better check for ztunnel pod guess (#44292) * add a better check for ztunnel guess * rebise based on comments * cni: drop experimental taint controller (#44377) This controller was added many years ago as an experiment and hasn't progressed. Unless there are users who have remained hidden during this time, I think its effectively dead code we should cleanup. * kube: use protobuf in client (#44379) * kube: use protobuf in client This was attempted in #38658 but the `config` set is never used there. Also add an opt-out and AcceptContentTypes to allow json fallback as recommended by k8s * fix writes * Remove release note approval by release-managers (#44395) * gateway: prevent duplicate `istio_authn` network filter in the filter chain (#44388) * gateway: prevent duplicate `istio_authn` network filter in the filter chain Signed-off-by: Yaroslav Skopets * add release notes Signed-off-by: Yaroslav Skopets * fix lint Signed-off-by: Yaroslav Skopets * fixup Signed-off-by: Yaroslav Skopets * ensure stability of the unit test Signed-off-by: Yaroslav Skopets --------- Signed-off-by: Yaroslav Skopets * Add response headers for grpc (#44394) * Add response headers for grpc * Update the output * Automator: update istio/client-go@master dependency in istio/istio@master (#44396) * Automator: update common-files@master in istio/istio@master (#44398) * Use k8s 1.27.0 as the default node image for integration testing (#44400) * Automator: update proxy@master in istio/istio@master (#44403) * remove deadcode from operator proto (#44397) Was missed in previous PR * fix:productpage build fail (#44405) Signed-off-by: xin.li * Run update_deps.sh (#44404) * chore: Fix function name of comment (#44406) Correct name is runAllTypes but comment has pruneAllTypes * Automator: update go-control-plane in istio/istio@master (#44407) * Automator: update proxy@master in istio/istio@master (#44408) * Automator: update ztunnel@master in istio/istio@master (#44409) * Remove hard coded Istio namespace (#44410) * Automator: update proxy@master in istio/istio@master (#44411) * Analyze: add cert check for gateway credential (#43921) * add cert check for gateway credential * add releasenotes and lint * reuse some logics and add more checks * make gen * remove unnecessary releasenotes * Automator: update proxy@master in istio/istio@master (#44419) * Fix webhook issues in installation process (#44345) * fix webhook creation in install process * add releasenotes * Only set the WorkloadSelector of ServiceEntry if the label is not empty (#44420) Signed-off-by: Yanqiang Miao * Setting the control plane lazily (#44417) Change-Id: I31adfcb808d04aef51c42ddaed5b3473db2397a5 * Automator: update proxy@master in istio/istio@master (#44431) * "istioctl pc route" output add "VHOST NAME" (#44414) * "istioctl pc route" output add "VHOST NAME" * add releasenote * fix * Update releasenotes/notes/44414.yaml Co-authored-by: Yossi Mesika --------- Co-authored-by: Yossi Mesika * Temper severity of extraneous errors (#44416) When the CNI initially runs on a node that is clean of the ztunnel chains errors and warnings are logged that are extraneous. This change reduces the severity and adds comments in the code. * cleanup message.yaml (#44421) * update_deps for 1.18 branch cut (#44436) * Automator: update proxy@master in istio/istio@master (#44442) * fix wrong example for admin log (#44438) Signed-off-by: xin.li * add validation for empty prefix header match (#44428) * add validation for empty prefix header match * fix linter issues * update error message + add a release note * add "prefix" to the error message * >fix `istioctl analyze` to panic when the server port in gateway is nil. (#44321) * >fix `istioctl analyze` to panic when the server port in gateway is nil. * Update releasenotes/notes/fix-44318.yaml Co-authored-by: Xiaopeng Han --------- Co-authored-by: Xiaopeng Han * Automator: update ztunnel@release-1.18 in istio/istio@release-1.18 (#44527) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#44550) * [release-1.18] Automated branching step 4 (#44553) * Automator: update istio/pkg@release-1.18 dependency in istio/istio@release-1.18 (#44559) * Automator: update ztunnel@release-1.18 in istio/istio@release-1.18 (#44560) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#44564) * Automator: update istio/client-go@release-1.18 dependency in istio/istio@release-1.18 (#44563) * Automator: update common-files@release-1.18 in istio/istio@release-1.18 (#44567) * Automator: update istio/client-go@release-1.18 dependency in istio/istio@release-1.18 (#44569) * Update BASE_VERSION to release-1.18-2023-04-26T19-01-40 (#44576) * Automator: update ztunnel@release-1.18 in istio/istio@release-1.18 (#44607) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#44610) * Update master to 1.18 (#44615) * gateway-api: start reading ReferenceGrant beta (#44619) Co-authored-by: John Howard * update to kiali 1.67.0 (#44504) Co-authored-by: John Mazzitelli * revise waypoint examples (#44511) Co-authored-by: xiaopeng * Use safer dedupe for config (#44521) This is just appending them, there is no guarantee of conflicts being avoided Co-authored-by: John Howard * disable automount SA token only on tests with min istio revisions >= 1.16 (#44533) Testing multiple istio versions involves older istio versions which doesn't support sidecars with disable automount SA token. This was enabled form 1.16 onwards. Co-authored-by: akshayjnambiar * Automator: update ztunnel@release-1.18 in istio/istio@release-1.18 (#44625) * [release-1.18] Fix pilot using wrong readinessprobe check, should check if /validate and /inject endpoints are ready. (#44632) * fix validationcontroller not having readinessprobe * add releasenotes * revise based on comments * revise based on comments * Delete 44526.yaml --------- Co-authored-by: xiaopeng * typo fix for failover validation (#44638) Co-authored-by: Greg Hanson * telemetry: deflake access log tests (#44645) Example failure: https://prow.istio.io/view/gs/istio-prow/pr-logs/directory/integ-telemetry_istio/1652008195079540736 It looks like the XDS push is just causing too much load and it takes more than 10s to process. We see during XDS push /stats/prometheus also times out. Co-authored-by: John Howard * [release-1.18] Support p384 curves (#44628) * support p384 curves * code review and make gen * cleanup test * fix linter * only support 256 and 384 * cleanup tests --------- Co-authored-by: Jacob Delgado * Automator: update go-control-plane in istio/istio@release-1.18 (#44651) * [release-1.18] Skip runtime resources when analyzing files (#44663) * Skip runtime resources when analyzing files * add test data and fmt * add support for tests to analyze pods * update analyze test to respect file exclusions * show failed json in message * fix json formatting * differentiate json analyzer tests * add release note --------- Co-authored-by: Mitch Connors * [release-1.18] gateway: fix and test unmanaged skipping (#44508) * gateway: fix and test unmanaged skipping https://github.com/istio/istio/pull/43541 didn't work quite right due to a rebasing issue. Fix it and add better tests * fix test * fix log * use unique name (#44528) (cherry picked from commit 7033e9537a68f23a77f804f35413c7d00b6e00be) (cherry picked from commit c8f7331975c43dbd3ee7f7f084ffb2d546429b6d) --------- Co-authored-by: John Howard * vwh: speedup reconcilation and fix test flake (#44658) Fixes https://prow.istio.io/view/gs/istio-prow/logs/integ-pilot-cpp_istio_postsubmit/1651656748131422208 failure This is a 1.18 regression since we changed the queue backoff. This fixes it in 2 ways: * Make sure we don't ahve to wait 1min every time * Make sure once one webhook succeeds, all of them do This also makes the test more robust to wait for all webhooks instead of just one. Co-authored-by: John Howard * Automator: update istio/client-go@release-1.18 dependency in istio/istio@release-1.18 (#44672) * Automator: update common-files@release-1.18 in istio/istio@release-1.18 (#44669) * Use go-control-plane from the last commmit before 1.26 was cut (#44674) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#44682) * Fix new test which was broken on distroless (#44685) There is no distroless app container Co-authored-by: John Howard * Automator: update ztunnel@release-1.18 in istio/istio@release-1.18 (#44694) * update_deps output priot to beta0 build (#44699) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#44707) * Actually change the rate limiter type (#44726) Co-authored-by: John Howard * Automator: update common-files@release-1.18 in istio/istio@release-1.18 (#44727) * Automator: update istio/client-go@release-1.18 dependency in istio/istio@release-1.18 (#44724) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#44728) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#44738) * Fix multi-cluster issue by increasing the timeout of listing CRDs (#44715) (#44740) When a new secret is added, a watcher will be created based on the remote secret. The process can fail if the API server doesn't respond in 10 seconds which can be the case if the cluster contains a lot of CRDs. This PR bumps the timeout to 60 seconds which is the default timeout value (specified in --request-timeout) for requests to API server. * [release-1.18] Fix persistent sessions scale down with envoy (#44653) * Missing change for persistent session support. Without it envoy will drop the draining endpoints and scale down will break. * Update the cluster status to be more future proof, add grpc, c++ implementation requires this * Remove unhealthy --------- Co-authored-by: Costin Manolache * Automator: update ztunnel@release-1.18 in istio/istio@release-1.18 (#44760) * [release-1.18] Fix gateway hostname resolution TTL (#44768) * fix gateway hostname resolution ttl * add tests --------- Co-authored-by: dddddai * [release-1.18] Fix verify-install to work with multi iops (#44752) * fix verify-install with multi iops * Update releasenotes/notes/verify-install-multi-iops.yaml Co-authored-by: Eric Van Norman --------- Co-authored-by: xiaopeng Co-authored-by: Eric Van Norman * deployment controller: add leaderelection back (#44746) (#44771) * Add per-revision leader election * Add leader election for deployment controller (cherry picked from commit acd30f96ef24f8bfc0cdd72a996ee90c27cea8d5) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#44774) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#44780) * fix missing gateway services (#44461) Co-authored-by: dddddai * add support for security.istio.io/v1beta1 api in authz tests when testing multiple istio versions (#44806) Testing multiple istio versions involves older istio versions which uses the v1beta1 api. This change will start using v1 for 1.17+ and v1beta1 for 1.16-. Co-authored-by: Akshay J Nambiar * set delay between retry attempts (#44809) Co-authored-by: dddddai * [release-1.18] istiod: drop Alpha Gateway API types by default (#44812) * istiod: drop Alpha Gateway API types by default This is problematic because most providers will not install Alpha CRDs. So once these promote to Beta, clusters will have *only* beta version in the CRD. We would detect the CRD and attempt to watch, but fail as alpha does not exist. This makes the alpha enablement an explicit opt-in to avoid this. An alternative could be to read the actual CRD to check it has the version we want. However, this is not safe -- a user may silently stop reading critical configurations. This is not needed for Istio CRDs as we don't remove old versions. * Fix test * fix build * fix note --------- Co-authored-by: John Howard * [release-1.18] Fix precheck and analysis messages (#44832) * fix precheck and analysis messages * fix lint --------- Co-authored-by: xiaopeng * update_deps prior to beta1 build (#44846) * inject: remove unknown fields from template (#44860) We have a number of cases where we insert unknown fields into the template. This eventually gets marshalled into a `v1.Pod`, so the unknown fields are dropped. So it has no impact, but it is "wrong". However, one of the fields we have (restartPolicy) is actually going to be a valid field in future k8s, so would start breaking at that point. So this *will* be a critical bug for future k8s versions. Test with: Replace applyOverlayYAML with ``` decoder := json.NewDecoder(bytes.NewReader(patched)) decoder.DisallowUnknownFields() if err := decoder.Decode(&pod); err != nil { return nil, fmt.Errorf("unmarshal patched pod: %v", err) } ``` Its probably a nice idea to keep it as non-strict to be resilient to unexpected issues? Co-authored-by: John Howard * [release-1.18] vm: support health checks for VMs that are not using auto-registration (#44866) * vm: support health checks for VMs that are not using auto-registration Signed-off-by: Yaroslav Skopets * add release notes Signed-off-by: Yaroslav Skopets --------- Signed-off-by: Yaroslav Skopets Co-authored-by: Yaroslav Skopets * remove file from file certs before triggering call backs (#44907) Signed-off-by: Rama Chavali Co-authored-by: Rama Chavali * spiffe: fix handling of trust bundles with multiple keys (#44947) In the existing implementation, we were overriding the `cert` while iterating over doc.Keys. This commit fixes that. Further, there was an unnecessary check for the existence of `ret[trustDomain]`. We are iterating over a map with `trustDomain` as keys, maps don't have duplicate keys, and so `ret[trustDomain]` would never have had a key before we set it in this iteration. This is a cherry-pick for #44831 (with modifications in the tests to avoid conflicts). This was cherry-picked in 1.17 as #44909. Change-Id: Ibf68f75cc667a72cce68bd42e4f600bd37946222 * Automator: update common-files@release-1.18 in istio/istio@release-1.18 (#44992) * Automator: update ztunnel@release-1.18 in istio/istio@release-1.18 (#45001) * Automator: update istio/client-go@release-1.18 dependency in istio/istio@release-1.18 (#44994) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#45008) * [release-1.18] deploymentcontroller: add support for ProxyConfig CRD (#44987) * deploymentcontroller: add support for ProxyConfig CRD (#44916) * deploymentcontroller: add support for ProxyConfig CRD Signed-off-by: Jacek Ewertowski * Handle default gateway labels Signed-off-by: Jacek Ewertowski * Remove a comment Signed-off-by: Jacek Ewertowski * Handle pod annotations Signed-off-by: Jacek Ewertowski * Encapsulate EffectiveProxyConfig into GetProxyConfigOrDefault Signed-off-by: Jacek Ewertowski * Remove default pod label service.istio.io/canonical-name Signed-off-by: Jacek Ewertowski * Respect only istio.io/gateway-name label when matching ProxyConfig with Gateway Signed-off-by: Jacek Ewertowski * Add release note Signed-off-by: Jacek Ewertowski * Change area in the relase note Signed-off-by: Jacek Ewertowski * Remove support for proxy.istio.io/config annotation applied to k8s Gateway pods Signed-off-by: Jacek Ewertowski --------- Signed-off-by: Jacek Ewertowski * Fix unit tests Signed-off-by: Jacek Ewertowski --------- Signed-off-by: Jacek Ewertowski * Automator: update istio/client-go@release-1.18 dependency in istio/istio@release-1.18 (#45024) * [release-1.18] Fix bug report missing all logs for json logs (#45026) * Add json log parsing * Remove bug report file * Omit filter and add tests * Extract to parseLog * Fix test failures * Fix format --------- Co-authored-by: Siyi Wang * telemetry: enable experimental mertic expiry (#44605) (#45037) * mertic-expiry * fix test * fix lint * add release-notes * fix gen * use ProxyMetadata * fix rebase * add test * update release-notes * fix not working * reduce xds size * use pilot ENV * update with john's comment * add UT * fix lint --------- Signed-off-by: hejianpeng * [release-1.18] fix backoff and read ca file interval (#45038) * fix read ca file interval * fix backoff never stop * address comment * remove permanent error * Address comment --------- Co-authored-by: Zhonghu Xu * fix waypoint list summary and add revision (#45052) Co-authored-by: xiaopeng * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#45072) * Automator: update common-files@release-1.18 in istio/istio@release-1.18 (#45074) * Automator: update istio/client-go@release-1.18 dependency in istio/istio@release-1.18 (#45076) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#45094) * Update BASE_VERSION to release-1.18-2023-05-24T19-03-47 (#45109) * RetryWithContext should use the new NextBackOff() (#45123) Signed-off-by: Faseela K Co-authored-by: Faseela K * rc.0 step 1 - update deps (#45134) * Automator: update ztunnel@release-1.18 in istio/istio@release-1.18 (#45139) * Temporarily remove eBPF impl pending CNCF guidance (#45167) eBPF support is temporarily disabled pending CNCF establishing guidance around dual-licensed eBPF bytecode https://github.com/cncf/toc/pull/1000#issuecomment-1564289871 Signed-off-by: Benjamin Leggett * 1.17: bump docker dep (#45199) This has a "CVE". It doesn't impact Istio but makes scanners unhappy. The dep is only used in WASM code and is a small bump so pretty low risk * Update BASE_VERSION to release-1.18-2023-05-31T19-02-09 (#45225) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#45281) * Update BASE_VERSION to release-1.18-2023-06-05T19-04-11 (#45289) * Update deps prior to 1.18.0 GA (#45310) * Automator: update common-files@release-1.18 in istio/istio@release-1.18 (#45325) * Automator: update istio/client-go@release-1.18 dependency in istio/istio@release-1.18 (#45326) * Automator: update ztunnel@release-1.18 in istio/istio@release-1.18 (#45333) * precise-errorcode-debuggen (#45164) Change-Id: Ia2654e18006b7cc2d54bb86ff9c9a2abe9e28bf5 Co-authored-by: Ingwon Song * [release-1.18]Manual cherry-pick of 44481 and 44775 (#45081) * add PDB to gateway chart * add releasenotes * revise based on comments * revise values to have better control * disable PDB by default * [release-1.18] Certificate Revocation List support (#45130) * refactor KeyCertAndStaple (#44764) Doing this refactoring in preparation for CRL support Signed-off-by: Faseela K * Certificate Revocation List support (#45104) Signed-off-by: Faseela K * populate crl only when the key is present in secret (#45112) Signed-off-by: Faseela K * skip empty ocsp staple configuration (#45159) Signed-off-by: Faseela K --------- Signed-off-by: Faseela K * Adding LRS support (#45165) Change-Id: Ifd075d62a5f0dda3b4b57eb807677f1637bed04f Co-authored-by: Ingwon Song * Fix invalid XDS configuration for wildcard Ingress HTTP path (#44898) (#45168) * Fix invalid XDS configuration for wildcard Ingress path Updates Ingress to VirtualService translation to not create a HTTPRequestMatch when the URI is nil. The URI is nil when the path is a wildcard or is empty and the pathType is nil or implementationSpecific. This change prevents an Envoy failure. Envoy regex fails when the path seperated prefix is empty or has a trailing "/". * Fix failing unit tests - handle sorting HTTPRoutes with no HTTPRequestMatches * Update golden converion tests for ingress. Remove empty match. --------- Signed-off-by: jaellio * Fix Telemetry disablement matching (#45303) Co-authored-by: John Howard * Add rolling update max unavailable to CNI chart to speed up deploys (cherry pick to release-1.18) (#44934) * Add rolling update max unavailable to CNI chart to speed up deploys * Update generated code * Update chart updating instructions * Add release note * Skip config from #44642 * [release-1.18] Fix istioctl pc secret cert validity not accurate (#45343) * fix istioctl pc secret cert valid * lint and add releasenotes --------- Co-authored-by: xiaopeng * Change to use Node instead of RawMeta (#45359) Change-Id: I21117025bb99b62c18484d2f1598a001751faaa4 Co-authored-by: Ingwon Song * [release-1.18] Check the disabled status when adding a log provider (#45373) * Check the disabled status when adding a log provider By checking the disabled status when adding a log provider, this PR fixes an issue where disabling a log provider through Istio telemetry API would not work. Otherwise, a disabled log provider may still be added to the log configuration and cause the disabling to not work as expected. The test case in this PR verifies that with the fix this PR, a disabled log provider will not be added to the log configuration. * Update the variable name --------- Co-authored-by: Lei Tang <32078630+lei-tang@users.noreply.github.com> * Automator: update istio/client-go@release-1.18 dependency in istio/istio@release-1.18 (#45381) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#45450) * [release-1.18] Update min supported k8s version to 1.24 (#45444) * update min supported k9s version to 1.24 * add releasenotes --------- Co-authored-by: xiaopeng * [release-1.18] cherry-pick: add debug info when generating certs for workloads (#45194) * cherry-pick: add debug info when generating certs for workloads #45183 Signed-off-by: huabing zhao * remove signer and make ttl human readable Signed-off-by: huabing zhao --------- Signed-off-by: huabing zhao * Update BASE_VERSION to 1.18-2023-06-15T19-02-54 (#45495) * [release-1.18] improve accesslog mode e2e tests (#45519) * improve accesslog mode e2e tests * retry Signed-off-by: hejianpeng * fix --------- Signed-off-by: hejianpeng Co-authored-by: hejianpeng * Automator: update common-files@release-1.18 in istio/istio@release-1.18 (#45569) * Automator: update istio/client-go@release-1.18 dependency in istio/istio@release-1.18 (#45570) * Automator: update ztunnel@release-1.18 in istio/istio@release-1.18 (#45579) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#45587) * prow: move to use WI for auth_header in private (#45609) This replaces authentikos Co-authored-by: John Howard * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#45667) * Automator: update istio/client-go@release-1.18 dependency in istio/istio@release-1.18 (#45660) * Automator: update common-files@release-1.18 in istio/istio@release-1.18 (#45690) * Bump github.com/lestrrat-go/jwx from 1.2.25 to 1.2.26 (#45684) Signed-off-by: Kalya Subramanian * Automator: update ztunnel@release-1.18 in istio/istio@release-1.18 (#45702) * Fix auth header syntax (#45711) Co-authored-by: John Howard * gcp metadata: compute GCPClusterURL from metadata (#45741) This allows computing GCPClusterURL from GCP_METADATA env var, if it is set. This allows usage with zero dependency on the metadata server. * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#45747) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#45769) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#45771) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#45834) * [release-1.18] Fix a potential nil panic of endpointindex (#45808) * fix a potential nil panic of endpointindex * add releasenotes * revise releasenotes --------- Co-authored-by: xiaopeng * Automator: update istio/client-go@release-1.18 dependency in istio/istio@release-1.18 (#45857) * [release-1.18] Fix bug report include option not working as expected (#45860) * fix bug report include option * add releasenotes --------- Co-authored-by: xiaopeng * Automator: update istio/client-go@release-1.18 dependency in istio/istio@release-1.18 (#45876) * Automator: update common-files@release-1.18 in istio/istio@release-1.18 (#45875) * Automator: update ztunnel@release-1.18 in istio/istio@release-1.18 (#45892) * Automator: update common-files@release-1.18 in istio/istio@release-1.18 (#45936) * Automator: update istio/client-go@release-1.18 dependency in istio/istio@release-1.18 (#45938) * [release-1.18] Fix health probe port overwrite (#45873) * Remove app req host override with req host on health-check Signed-off-by: jaellio * set appReq host to prober host value Signed-off-by: jaellio * add conditional check before setting appReq host to probe host Signed-off-by: jaellio * Remove host override from app probe unit tests. The explicit override does not reflect the actual request host value on health probe requests. Prior to being processed in the istio-proxy the host of the request should not be set to the app port. Signed-off-by: jaellio --------- Signed-off-by: jaellio Co-authored-by: jaellio * Add release note for #45632 (#45927) Signed-off-by: jaellio Co-authored-by: jaellio * Automator: update ztunnel@release-1.18 in istio/istio@release-1.18 (#45948) * [release-1.18] Set inject true for compatibility tests (#45928) * Set inject true for compatibility tests * adding the fix to TestProxyProtocolTCPGateway and TestCustomGateway --------- Co-authored-by: Riya Sinha * [release-1.18] prevent port conflict with sidecar static listener like 15021 15090 (#45966) * prevent port conflict with sidecar static listener 15021 15090 * remove duplicate conflict detection * address comment --------- Co-authored-by: Zhonghu Xu * Update image from (#45958) Co-authored-by: Eric Van Norman * Automator: update common-files@release-1.18 in istio/istio@release-1.18 (#45995) * Automator: update istio/client-go@release-1.18 dependency in istio/istio@release-1.18 (#45996) * Automator: update ztunnel@release-1.18 in istio/istio@release-1.18 (#46000) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#46007) * Exit if sds socket not found (#45941) (#46014) * Exit if sds socket not found (#45941) * adding - exit if sds socket not found * fix release note for # 45941 * renaming flag to USE_EXTERNAL_WORKLOAD_SDS * rewording release note for #45941, describe usage of USE_EXTERNAL_SDS_SOCKET * fix cherry-pick * fix conflict (#46017) Signed-off-by: Kuat Yessenov Co-authored-by: Kuat Yessenov * fix concurrent map access in endpoint metadata (#44473) (#46021) * fix concurrent map access in endpoint metadata * only clone as needed * only clone as needed * remove unnecessary code * review comments * fix ut * add test case * add lock --------- Signed-off-by: Rama Chavali Co-authored-by: Rama Chavali * Fix nil map for cluster builder (#46024) Co-authored-by: Sergii Shapar * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#46025) * Automator: update proxy@release-1.18 in istio/istio@release-1.18 (#46039) * Bump proxy version (#122) * Add validation of workload entry identity (#117) (cherry picked from commit b6eefaf3045227431b60384002e5b0c57740288d) * 1.18.2 tetrate build Signed-off-by: psbrar99 * eks patch Signed-off-by: psbrar99 * 1.18.2 tetratefips build Signed-off-by: psbrar99 * 1.18.2 tetratefips build Signed-off-by: psbrar99 * 1.18.2 tetratefips build Signed-off-by: psbrar99 * 1.18.2 tetratefips build Signed-off-by: psbrar99 * bump proxy SHA Signed-off-by: psbrar99 * envoy cves Signed-off-by: psbrar99 * envoy cves Signed-off-by: psbrar99 * bump proxy SHA Signed-off-by: psbrar99 * running e2e tests for pilot Signed-off-by: psbrar99 * running e2e tests for pilot Signed-off-by: psbrar99 * running e2e tests for pilot Signed-off-by: psbrar99 * bump runners Signed-off-by: psbrar99 * bump runners Signed-off-by: psbrar99 * publish release Signed-off-by: psbrar99 * update tetrate-workflow branch Signed-off-by: psbrar99 * update tetrate-workflow branch Signed-off-by: psbrar99 * update tetrate-workflow branch Signed-off-by: psbrar99 * update tetrate-workflow branch Signed-off-by: psbrar99 * Removing unnecessary code Signed-off-by: psbrar99 * addressed comment Signed-off-by: psbrar99 * address reveiw comments Signed-off-by: psbrar99 * cleanup and updated new patch for eks Signed-off-by: psbrar99 * Update tetrateci/version_check.py Co-authored-by: zirain * fix WF for make release job Signed-off-by: psbrar99 * fips.md update Signed-off-by: psbrar99 * Update fips.md * Update .github/workflows/make_release.yml Co-authored-by: zirain * Update .github/workflows/make_release.yml Co-authored-by: zirain --------- Signed-off-by: hejianpeng Signed-off-by: jongwooo Signed-off-by: Kalya Subramanian Signed-off-by: Faseela K Signed-off-by: Yaroslav Skopets Signed-off-by: xin.li Signed-off-by: Yanqiang Miao Signed-off-by: Rama Chavali Signed-off-by: Jacek Ewertowski Signed-off-by: Benjamin Leggett Signed-off-by: jaellio Signed-off-by: huabing zhao Signed-off-by: Kuat Yessenov Signed-off-by: psbrar99 Co-authored-by: dwq Co-authored-by: Istio Automation Co-authored-by: John Howard Co-authored-by: Zhonghu Xu Co-authored-by: Xiaopeng Han Co-authored-by: Eric Van Norman Co-authored-by: zirain Co-authored-by: Jongwoo Han Co-authored-by: Kalya Subramanian <42158129+ksubrmnn@users.noreply.github.com> Co-authored-by: Faseela K Co-authored-by: jacob-delgado Co-authored-by: cebernardi Co-authored-by: Rajat Sharma Co-authored-by: Costin Manolache Co-authored-by: pmerrison Co-authored-by: Yaroslav Skopets Co-authored-by: my-git9 Co-authored-by: Ikumi Nakamura <28798279+johnmanjiro13@users.noreply.github.com> Co-authored-by: Yossi Mesika Co-authored-by: Yanqiang Miao Co-authored-by: Ingwon Song <102102227+ingwonsong@users.noreply.github.com> Co-authored-by: zengyuxing Co-authored-by: Yossi Mesika Co-authored-by: john-a-joyce Co-authored-by: pmerrison Co-authored-by: Peter Jausovec Co-authored-by: Nicole LiHui Co-authored-by: John Mazzitelli Co-authored-by: akshayjnambiar Co-authored-by: Greg Hanson Co-authored-by: Mitch Connors Co-authored-by: Rui Gu Co-authored-by: Akshay J Nambiar Co-authored-by: Rama Chavali Co-authored-by: Rohit Agarwal Co-authored-by: Jacek Ewertowski Co-authored-by: Siyi Wang Co-authored-by: Ben Leggett <854255+bleggett@users.noreply.github.com> Co-authored-by: github-actions Co-authored-by: Ingwon Song Co-authored-by: Jackie Elliott <64559656+jaellio@users.noreply.github.com> Co-authored-by: Dwayne Schultz Co-authored-by: Lei Tang <32078630+lei-tang@users.noreply.github.com> Co-authored-by: zhaohuabing Co-authored-by: jaellio Co-authored-by: Riya Sinha Co-authored-by: Adam Sayah Co-authored-by: Kuat Yessenov Co-authored-by: Hemendra Teli <8605932+hemendrateli@users.noreply.github.com> Co-authored-by: Sergii Shapar Co-authored-by: zirain --- .github/ISSUE_TEMPLATE/bug_report.yml | 70 ++++ .github/ISSUE_TEMPLATE/config.yml | 4 + .github/ISSUE_TEMPLATE/feature_request.md | 30 ++ .github/SECURITY.md | 20 + .github/pull_request_template.md | 20 + .github/workflows/e2e_tests.yml | 374 +++++++++++++++--- .github/workflows/make_release.yml | 310 +++++++++++++-- .github/workflows/tools.yml | 2 +- .gitignore | 0 tetrateci/1.12/test/skip.d/eks-arm64 | 44 +++ tetrateci/1.13/test/skip.d/eks | 6 +- tetrateci/1.13/test/skip.d/eks-arm64 | 43 ++ tetrateci/1.13/test/skip.d/gke | 11 +- tetrateci/1.14/test/skip.d/eks | 33 ++ tetrateci/1.14/test/skip.d/eks-arm64 | 27 ++ tetrateci/1.14/test/skip.d/gke | 37 ++ tetrateci/1.15/test/skip.d/eks | 27 ++ tetrateci/1.15/test/skip.d/eks-arm64 | 27 ++ tetrateci/1.15/test/skip.d/gke | 37 ++ tetrateci/1.16/test/skip.d/eks | 29 ++ tetrateci/1.16/test/skip.d/eks-arm64 | 30 ++ tetrateci/1.16/test/skip.d/gke | 37 ++ tetrateci/1.17/test/skip.d/eks | 32 ++ tetrateci/1.17/test/skip.d/eks-arm64 | 31 ++ tetrateci/1.17/test/skip.d/gke | 37 ++ tetrateci/1.18/test/skip.d/eks | 32 ++ tetrateci/1.18/test/skip.d/eks-arm64 | 31 ++ tetrateci/1.18/test/skip.d/gke | 37 ++ tetrateci/ci_workflow.md | 26 ++ tetrateci/cleanup_eks_cluster.sh | 2 +- tetrateci/create_eks_cluster.sh | 9 +- tetrateci/create_istio_release.sh | 72 +++- tetrateci/create_multiarch_images.sh | 36 ++ tetrateci/docs/release.md | 72 +--- tetrateci/fips.md | 60 +++ tetrateci/gen_iop.py | 5 +- tetrateci/gen_release_manifest.py | 12 +- tetrateci/images.sh | 25 ++ tetrateci/manifest.yaml.in | 2 +- .../patches/common/enable-arm64.1.12.patch | 24 ++ tetrateci/patches/eks/eks-ingress.1.10.patch | 61 --- tetrateci/patches/eks/eks-ingress.1.11.patch | 61 --- tetrateci/patches/eks/eks-ingress.1.13.patch | 61 --- tetrateci/patches/eks/eks-ingress.1.7.patch | 58 --- tetrateci/patches/eks/eks-ingress.1.8.patch | 62 --- tetrateci/patches/eks/eks-ingress.1.9.patch | 58 --- tetrateci/patches/eks/eks_1.15.patch | 125 ++++++ tetrateci/patches/eks/eks_1.16.patch | 124 ++++++ tetrateci/patches/eks/eks_1.17.patch | 124 ++++++ tetrateci/patches/eks/eks_1.18.patch | 123 ++++++ tetrateci/setup_boring_go.sh | 36 -- tetrateci/setup_go.sh | 22 +- tetrateci/test_1.11.sh | 14 - tetrateci/test_1.12.sh | 14 - tetrateci/test_1.13.sh | 18 +- tetrateci/test_1.14.sh | 107 +++++ tetrateci/test_1.15.sh | 108 +++++ tetrateci/test_1.15_arm.sh | 111 ++++++ tetrateci/test_1.16.sh | 107 +++++ tetrateci/test_1.16_arm.sh | 111 ++++++ tetrateci/test_1.17.sh | 109 +++++ tetrateci/test_1.17_arm.sh | 111 ++++++ tetrateci/test_1.18.sh | 109 +++++ tetrateci/test_1.18_arm.sh | 111 ++++++ tetrateci/version_check.py | 5 + 65 files changed, 3080 insertions(+), 633 deletions(-) create mode 100644 .github/ISSUE_TEMPLATE/bug_report.yml create mode 100644 .github/ISSUE_TEMPLATE/config.yml create mode 100644 .github/ISSUE_TEMPLATE/feature_request.md create mode 100644 .github/SECURITY.md create mode 100644 .github/pull_request_template.md delete mode 100644 .gitignore create mode 100644 tetrateci/1.12/test/skip.d/eks-arm64 create mode 100644 tetrateci/1.13/test/skip.d/eks-arm64 create mode 100644 tetrateci/1.14/test/skip.d/eks create mode 100644 tetrateci/1.14/test/skip.d/eks-arm64 create mode 100644 tetrateci/1.14/test/skip.d/gke create mode 100644 tetrateci/1.15/test/skip.d/eks create mode 100644 tetrateci/1.15/test/skip.d/eks-arm64 create mode 100644 tetrateci/1.15/test/skip.d/gke create mode 100644 tetrateci/1.16/test/skip.d/eks create mode 100644 tetrateci/1.16/test/skip.d/eks-arm64 create mode 100644 tetrateci/1.16/test/skip.d/gke create mode 100644 tetrateci/1.17/test/skip.d/eks create mode 100644 tetrateci/1.17/test/skip.d/eks-arm64 create mode 100644 tetrateci/1.17/test/skip.d/gke create mode 100644 tetrateci/1.18/test/skip.d/eks create mode 100644 tetrateci/1.18/test/skip.d/eks-arm64 create mode 100644 tetrateci/1.18/test/skip.d/gke create mode 100644 tetrateci/ci_workflow.md create mode 100755 tetrateci/create_multiarch_images.sh create mode 100644 tetrateci/fips.md create mode 100644 tetrateci/images.sh create mode 100644 tetrateci/patches/common/enable-arm64.1.12.patch delete mode 100644 tetrateci/patches/eks/eks-ingress.1.10.patch delete mode 100644 tetrateci/patches/eks/eks-ingress.1.11.patch delete mode 100644 tetrateci/patches/eks/eks-ingress.1.13.patch delete mode 100644 tetrateci/patches/eks/eks-ingress.1.7.patch delete mode 100644 tetrateci/patches/eks/eks-ingress.1.8.patch delete mode 100644 tetrateci/patches/eks/eks-ingress.1.9.patch create mode 100644 tetrateci/patches/eks/eks_1.15.patch create mode 100644 tetrateci/patches/eks/eks_1.16.patch create mode 100644 tetrateci/patches/eks/eks_1.17.patch create mode 100644 tetrateci/patches/eks/eks_1.18.patch delete mode 100755 tetrateci/setup_boring_go.sh create mode 100644 tetrateci/test_1.14.sh create mode 100644 tetrateci/test_1.15.sh create mode 100644 tetrateci/test_1.15_arm.sh create mode 100644 tetrateci/test_1.16.sh create mode 100644 tetrateci/test_1.16_arm.sh create mode 100644 tetrateci/test_1.17.sh create mode 100644 tetrateci/test_1.17_arm.sh create mode 100644 tetrateci/test_1.18.sh create mode 100644 tetrateci/test_1.18_arm.sh diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml new file mode 100644 index 000000000000..9f5773197221 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -0,0 +1,70 @@ +name: Bug report +description: Report a bug to help us improve Istio +body: + - type: markdown + attributes: + value: | + Thanks for taking the time to fill out this bug report! + - type: textarea + id: bug-description + attributes: + label: Bug Description + description: Tell us what issues you ran into + placeholder: Include information about what you tried, what you expected to happen, and what actually happened. The more details, the better! + validations: + required: true + - type: textarea + id: version + attributes: + label: Version + description: Include the output of `istioctl version`, `kubectl version --short`, and `helm version --short` (if you used Helm) + placeholder: | + $ istioctl version + client version: 1.0.0 + control plane version: 1.0.0 + data plane version: 1.0.0 (100 proxies) + $ kubectl version --short + Client Version: v1.0.0 + Server Version: v1.0.0 + render: prose + validations: + required: true + - type: textarea + id: additional-info + attributes: + label: Additional Information + description: | + Please include the output of [`istioctl bug-report`](http://istio.io/help/bugs/#generating-a-cluster-state-archive). + If you are unable to do so, please ensure you have collected the relevant debugging information manually and attached below; + issue without enough information will not be resolvable. + - type: checkboxes + id: area + attributes: + label: Affected product area + options: + - label: "Docs" + - label: "Installation" + - label: "Networking" + - label: "Performance and Scalability" + - label: "Extensions and Telemetry" + - label: "Security" + - label: "Test and Release" + - label: "User Experience" + - label: "Developer Infrastructure" + - label: "Upgrade" + - label: "Multi Cluster" + - label: "Virtual Machine" + - label: "Control Plane Revisions" + - type: checkboxes + id: security-check + attributes: + label: Is this the right place to submit this? + description: |- + This is used to report product bugs: + To report a security vulnerability, please visit + To ask questions about how to use Istio, please visit + options: + - label: "This is not a security vulnerability" + required: true + - label: "This is not a question about how to use Istio" + required: true \ No newline at end of file diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 000000000000..9dd8948ba0de --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,4 @@ +contact_links: +- name: "Crash bug" + url: https://istio.io/about/security-vulnerabilities/ + about: "Please file any bug causing a crash to istio-security-vulnerability-reports@googlegroups.com." diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 000000000000..5e16d4942db8 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,30 @@ +--- +name: Feature request +about: Suggest an idea to improve Istio + +--- +(This is used to request new product features, please visit for questions on using Istio) + +**Describe the feature request** + +**Describe alternatives you've considered** + +**Affected product area (please put an X in all that apply)** + +[ ] Docs +[ ] Installation +[ ] Networking +[ ] Performance and Scalability +[ ] Extensions and Telemetry +[ ] Security +[ ] Test and Release +[ ] User Experience +[ ] Developer Infrastructure + +**Affected features (please put an X in all that apply)** + +[ ] Multi Cluster +[ ] Virtual Machine +[ ] Multi Control Plane + +**Additional context** diff --git a/.github/SECURITY.md b/.github/SECURITY.md new file mode 100644 index 000000000000..ed4d3c2f01e6 --- /dev/null +++ b/.github/SECURITY.md @@ -0,0 +1,20 @@ +# Security Policy + +## Supported Versions + +Information about supported Istio versions can be found on the +[Support Announcements] page on Istio's website. + +## Reporting a Vulnerability + +Instructions for reporting a vulnerability can be found on the +[Istio Security Vulnerabilities] page. + +## Security Bulletins + +Information about previous Istio vulnerabilities can be found on the +[Security Bulletins] page. + +[Support Announcements]: https://istio.io/news/support/ +[Istio Security Vulnerabilities]: https://istio.io/about/security-vulnerabilities/ +[Security Bulletins]: https://istio.io/news/security/ diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 000000000000..672b0189a846 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,20 @@ +**Please provide a description of this PR:** + + + +**To help us figure out who should review this PR, please put an X in all the areas that this PR affects.** + +- [ ] Configuration Infrastructure +- [ ] Docs +- [ ] Installation +- [ ] Networking +- [ ] Performance and Scalability +- [ ] Policies and Telemetry +- [ ] Security +- [ ] Test and Release +- [ ] User Experience +- [ ] Developer Infrastructure + +**Please check any characteristics that apply to this pull request.** + +- [ ] Does not have any [user-facing](https://github.com/istio/istio/tree/master/releasenotes#when-to-add-release-notes) changes. This may include CLI changes, API changes, behavior changes, performance improvements, etc. diff --git a/.github/workflows/e2e_tests.yml b/.github/workflows/e2e_tests.yml index 26a84592c813..8acbaede38bf 100644 --- a/.github/workflows/e2e_tests.yml +++ b/.github/workflows/e2e_tests.yml @@ -3,7 +3,7 @@ name: e2e-tests on: push: tags: - - 'test-*' + - "test-*" env: HUB: ${{ secrets.TEST_HUB }} @@ -14,55 +14,63 @@ jobs: # Only run this job when the release type equals "fips". if: contains(github.event.ref, 'fips') name: build fips proxy binary - timeout-minutes: 1440 # 12 hours - runs-on: ubuntu-latest + timeout-minutes: 840 + runs-on: ['self-hosted','Linux','x64','c5.4xlarge'] env: RELEASE_GCS_PATH: gs://getistio-build/proxy-fips steps: - name: Get the tag id: get_tag run: echo ::set-output name=TAG::${GITHUB_REF#refs/tags/} - - name: Get normalized tag - id: get_minor_ver + id: get_normalized_ver run: echo ::set-output name=NORMALIZED_TAG::$(echo $TAG | sed 's/test-//g' | sed 's/-.*//g') shell: bash env: TAG: ${{ steps.get_tag.outputs.TAG }} - + - name: Get minor version + id: get_minor_ver + run: echo ::set-output name=REL_BRANCH_VER::$(echo $TAG | grep -Eo '[0-9]+\.[0-9]+') + shell: bash + env: + TAG: ${{ steps.get_tag.outputs.TAG }} - name: Checkout uses: actions/checkout@v2 with: - repository: 'istio/proxy' - ref: ${{ steps.get_minor_ver.outputs.NORMALIZED_TAG }} - + repository: "istio/proxy" + ref: ${{ steps.get_normalized_ver.outputs.NORMALIZED_TAG }} - name: Set up Cloud SDK uses: google-github-actions/setup-gcloud@v0 with: project_id: ${{ secrets.GCP_PROJECT_ID }} service_account_key: ${{ secrets.GCP_SA_KEY }} export_default_credentials: true - - name: Check if already built id: check_already_built + env: + REL_BRANCH_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + TAG: ${{ steps.get_tag.outputs.TAG }} run: | - SHA=$(git rev-parse --verify HEAD) + SHA=$(echo $TAG | sed 's/test-//g' | sed 's/[0-9].[0-9][0-9].[0-9]-[A-Za-z]*-//g' | sed 's/-.*//g') SHOULD_BUILD=1 - if gsutil ls ${RELEASE_GCS_PATH} | grep ${SHA} ; then + SHOULD_COPY=1 + if gsutil ls ${RELEASE_GCS_PATH} | grep "${SHA}[^-]" ; then SHOULD_BUILD=0 + fi + if [ $(bc <<< "$REL_BRANCH_VER > 1.17") -eq 1 ];then + SHOULD_COPY=0 fi echo ::set-output name=should_build::${SHOULD_BUILD} - + echo ::set-output name=should_copy::${SHOULD_COPY} - name: Tweak make recipe if: ${{ steps.check_already_built.outputs.should_build == '1' }} - # 1) remove unnecessary dependency on naive "build" target - # 2) remove "-p" flag to not push and build container, + # 1) remove unnecessary dependency on naive "build" target + # 2) remove "-p" flag to not push and build container, # 3) replace RELEASE_GCS_PATH var with the literal due to docker build. # from push_release. run: | ESCAPED_RELEASE_GCS_PATH=$(printf '%s\n' "$RELEASE_GCS_PATH" | sed -e 's/[\/&]/\\&/g') sed -i "s/\-p//g; s/push_release: build/push_release:/g; s/\"\$(RELEASE_GCS_PATH)\"/${ESCAPED_RELEASE_GCS_PATH}/g" Makefile.core.mk - - name: Tweak release-binary script if: ${{ steps.check_already_built.outputs.should_build == '1' }} # 1) Setup in-docker auth against gcloud @@ -74,7 +82,7 @@ jobs: cp scripts/release-binary.sh scripts/release-binary.sh.tmp echo gcloud auth activate-service-account --key-file="/work/${CREDENTIAL_FILE_NAME}" > scripts/release-binary.sh cat scripts/release-binary.sh.tmp >> scripts/release-binary.sh - sed -i 's/release release\-symbol debug/release release\-symbol/' scripts/release-binary.sh + sed -i 's/for config in release release-symbol asan debug/for config in release/g' scripts/release-binary.sh sed -i -n '/Build and publish Wasm plugins/q;p' scripts/release-binary.sh cat scripts/release-binary.sh @@ -85,29 +93,181 @@ jobs: - name: Build and push if: ${{ steps.check_already_built.outputs.should_build == '1' }} env: - BUILD_WITH_CONTAINER: 1 + BUILD_WITH_CONTAINER: 1 run: make push_release - + - name: Put CentOS binary + env: + REL_BRANCH_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + TAG: ${{ steps.get_tag.outputs.TAG }} # Copy the binary built on Ubuntu to CentOS path - this binar cannot run on CentOS/RHEL 7, but fine with CentOS 8. # The point is that we cannot build FIPS binary directly on CentOS 7 due to the constraints described in the BoringCrypto certification. run: | - SHA=$(git rev-parse --verify HEAD) + SHA=$(echo $TAG | sed 's/test-//g' | sed 's/[0-9].[0-9][0-9].[0-9]-[A-Za-z]*-//g' | sed 's/-.*//g') gsutil cp ${RELEASE_GCS_PATH}/envoy-alpha-${SHA}.tar.gz ${RELEASE_GCS_PATH}/envoy-centos-alpha-${SHA}.tar.gz - name: Put Wasm binary + if: ${{ steps.check_already_built.outputs.should_build == '1' && steps.check_already_built.outputs.should_copy == '1' }} # Copy the built Wasm binary in the upstream to the bucket - Wasm is nothing to do with FIPS stuff. env: UPSTREAM_GCSP_PATH: gs://istio-build/proxy + REL_BRANCH_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + TAG: ${{ steps.get_tag.outputs.TAG }} + run: | + SHA1=$(git rev-parse --verify HEAD) + SHA=$(echo $TAG | sed 's/test-//g' | sed 's/[0-9].[0-9][0-9].[0-9]-[A-Za-z]*-//g' | sed 's/-.*//g') + gsutil cp "${UPSTREAM_GCSP_PATH}/metadata_exchange-${SHA1}.compiled.wasm" "${RELEASE_GCS_PATH}/metadata_exchange-${SHA}.compiled.wasm" + gsutil cp "${UPSTREAM_GCSP_PATH}/metadata_exchange-${SHA1}.compiled.wasm.sha256" "${RELEASE_GCS_PATH}/metadata_exchange-${SHA}.compiled.wasm.sha256" + gsutil cp "${UPSTREAM_GCSP_PATH}/metadata_exchange-${SHA1}.wasm" "${RELEASE_GCS_PATH}/metadata_exchange-${SHA}.wasm" + gsutil cp "${UPSTREAM_GCSP_PATH}/metadata_exchange-${SHA1}.wasm.sha256" "${RELEASE_GCS_PATH}/metadata_exchange-${SHA}.wasm.sha256" + gsutil cp "${UPSTREAM_GCSP_PATH}/stats-${SHA1}.compiled.wasm" "${RELEASE_GCS_PATH}/stats-${SHA}.compiled.wasm" + gsutil cp "${UPSTREAM_GCSP_PATH}/stats-${SHA1}.compiled.wasm.sha256" "${RELEASE_GCS_PATH}/stats-${SHA}.compiled.wasm.sha256" + gsutil cp "${UPSTREAM_GCSP_PATH}/stats-${SHA1}.wasm" "${RELEASE_GCS_PATH}/stats-${SHA}.wasm" + gsutil cp "${UPSTREAM_GCSP_PATH}/stats-${SHA1}.wasm.sha256" "${RELEASE_GCS_PATH}/stats-${SHA}.wasm.sha256" + + build_fips_arm_proxy: + # Do not run this job when the release type equals "fips"as we do not support fips compliant ARM build yet. + # if: contains(github.event.ref, 'fips') + if: false + name: build ARM proxy binary + timeout-minutes: 1440 # 12 hours + runs-on: ['self-hosted','Linux','ARM64','arm64-32'] + env: + RELEASE_GCS_PATH: gs://getistio-build/proxy-fips + ARCH_SUFFIX: arm64 + steps: + - name: Get normalized tag + id: get_normalized_ver + run: echo ::set-output name=NORMALIZED_TAG::$(echo $TAG | sed 's/test-//g' | sed 's/-.*//g') + shell: bash + env: + TAG: ${{ steps.get_tag.outputs.TAG }} + - name: Get minor version + id: get_minor_ver + run: echo ::set-output name=REL_BRANCH_VER::$(echo $TAG | grep -Eo '[0-9]+\.[0-9]+') + shell: bash + env: + TAG: ${{ steps.get_tag.outputs.TAG }} + - name: Checkout + uses: actions/checkout@v2 + with: + repository: "istio/proxy" + ref: ${{ steps.get_normalized_ver.outputs.NORMALIZED_TAG }} + - name: Set up Cloud SDK + uses: google-github-actions/setup-gcloud@v0 + with: + project_id: ${{ secrets.GCP_PROJECT_ID }} + service_account_key: ${{ secrets.GCP_SA_KEY }} + export_default_credentials: true + - name: Check if already built + id: check_already_built + env: + REL_BRANCH_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + TAG: ${{ steps.get_tag.outputs.TAG }} + run: | + SHA=$(echo $TAG | sed 's/test-//g' | sed 's/[0-9].[0-9][0-9].[0-9]-[A-Za-z]*-//g' | sed 's/-.*//g') + SHOULD_BUILD=1 + if gsutil ls ${RELEASE_GCS_PATH} | grep "${SHA}[^-]" ; then + SHOULD_BUILD=0 + fi + echo ::set-output name=should_build::${SHOULD_BUILD} + - name: Tweak make recipe + if: ${{ steps.check_already_built.outputs.should_build == '1' }} + # 1) remove unnecessary dependency on naive "build" target + # 2) remove "-p" flag to not push and build container, + # 3) replace RELEASE_GCS_PATH var with the literal due to docker build. + # from push_release. + run: | + ESCAPED_RELEASE_GCS_PATH=$(printf '%s\n' "$RELEASE_GCS_PATH" | sed -e 's/[\/&]/\\&/g') + sed -i "s/\-p//g; s/push_release: build/push_release:/g; s/\"\$(RELEASE_GCS_PATH)\"/${ESCAPED_RELEASE_GCS_PATH}/g" Makefile.core.mk + - name: Tweak release-binary script + if: ${{ steps.check_already_built.outputs.should_build == '1' }} + # 1) Setup in-docker auth against gcloud + # 2) Remove unnecessary debug build + # 3) Remove unnecessary Wasm build and publish + # Note that /work is the mount destination of the current home of make command with BUILD_WITH_CONTAINER=1. run: | - SHA=$(git rev-parse --verify HEAD) - gsutil cp "${UPSTREAM_GCSP_PATH}/metadata_exchange-${SHA}*" ${RELEASE_GCS_PATH}/ - gsutil cp "${UPSTREAM_GCSP_PATH}/stats-${SHA}*" ${RELEASE_GCS_PATH}/ - gsutil cp "${UPSTREAM_GCSP_PATH}/attributegen-${SHA}*" ${RELEASE_GCS_PATH}/ + CREDENTIAL_FILE_NAME=$(echo ${GOOGLE_APPLICATION_CREDENTIALS} | awk -F/ '{print $NF}') + cp scripts/release-binary.sh scripts/release-binary.sh.tmp + echo gcloud auth activate-service-account --key-file="/work/${CREDENTIAL_FILE_NAME}" > scripts/release-binary.sh + cat scripts/release-binary.sh.tmp >> scripts/release-binary.sh + sed -i 's/release release\-symbol debug/release release\-symbol/' scripts/release-binary.sh + sed -i '/if \[\[ "${BUILD_ENVOY_BINARY_ONLY}" !/, +6d' scripts/release-binary.sh + sed -i -n '/Build and publish Wasm plugins/q;p' scripts/release-binary.sh + cat scripts/release-binary.sh + - name: Add FIPS flag + if: ${{ steps.check_already_built.outputs.should_build == '1' }} + run: echo "build --define boringssl=fips" >> .bazelrc + - name: Build and push + if: ${{ steps.check_already_built.outputs.should_build == '1' }} + env: + BUILD_ENVOY_BINARY_ONLY: 1 + BUILD_WITH_CONTAINER: 1 + run: make push_release - create-test-images: - name: create-test-images - runs-on: ubuntu-latest + create-test-images-arm64: + name: create-test-images-arm64 + runs-on: ["self-hosted", "arm64"] + env: + TARGETARCH: arm64 + DOCKER_ARCHITECTURES: linux/arm64 + needs: [build_fips_arm_proxy] + # 'if' condition causes this job to run even if some of the dependent jobs + # have been skipped, e.g. `build_fips_proxy`. + # see https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#example-not-requiring-dependent-jobs-to-be-successful + if: "contains(github.event.ref, 'multiarch')" + + steps: + - name: checkout + uses: actions/checkout@v2 + + - name: Get Registry + id: get_registry + run: | + [[ ${HUB} == *.* ]] && REGISTRY=$(echo ${HUB} | cut -d/ -f1) + echo REGISTRY=${REGISTRY} + echo REGISTRY=${REGISTRY} >> $GITHUB_ENV + + - name: docker-login-non-docker + uses: docker/login-action@v1 + if: ${{ env.REGISTRY != null }} + with: + registry: ${{ env.REGISTRY }} + username: ${{ secrets.TEST_HUB_USER }} + password: ${{ secrets.TEST_HUB_PASS }} + + - name: docker-login-docker + uses: docker/login-action@v1 + if: ${{ env.REGISTRY == null }} + with: + username: ${{ secrets.TEST_HUB_USER }} + password: ${{ secrets.TEST_HUB_PASS }} + + - name: Get the tag + id: get_tag + run: echo ::set-output name=TAG::${GITHUB_REF/refs\/tags\/test-/} + shell: bash + + - name: Get minor version + id: get_minor_ver + run: echo ::set-output name=REL_BRANCH_VER::$(echo $TAG | grep -Eo '[0-9]+\.[0-9]+') + shell: bash + env: + TAG: ${{ steps.get_tag.outputs.TAG }} + + - name: build and push images + run: bash ./tetrateci/create_istio_release.sh + env: + TAG: ${{ steps.get_tag.outputs.TAG }} + TEST: "true" + # if the tag contains fips we are doing a fips build + BUILD: ${{ contains(github.event.ref, 'fips') && 'fips' }} + REL_BRANCH_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + + create-test-images-amd64: + name: create-test-images-amd64 + #runs-on: ['self-hosted','Linux','x64','m5'] + runs-on: ['self-hosted','Linux','x64','c5.4xlarge'] needs: [build_fips_proxy] # 'if' condition causes this job to run even if some of the dependent jobs # have been skipped, e.g. `build_fips_proxy`. @@ -152,13 +312,64 @@ jobs: env: TAG: ${{ steps.get_tag.outputs.TAG }} - - name: apply patches required in the context of e2e tests - run: tetrateci/apply_e2e_build_patches.sh + - name: build and push images + run: bash ./tetrateci/create_istio_release.sh env: - ISTIO_MINOR_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + TAG: ${{ steps.get_tag.outputs.TAG }} + TEST: "true" + # if the tag contains fips we are doing a fips build + BUILD: ${{ contains(github.event.ref, 'fips') && 'fips' }} + REL_BRANCH_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + + create-multiarch-images: + name: create-multiarch-images + runs-on: ubuntu-latest + needs: [create-test-images-amd64, create-test-images-arm64] + # 'if' condition causes this job to run even if some of the dependent jobs + # have been skipped, e.g. `build_fips_proxy`. + # see https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#example-not-requiring-dependent-jobs-to-be-successful + if: "contains(github.event.ref, 'multiarch')" + + steps: + - name: checkout + uses: actions/checkout@v2 + + - name: Get Registry + id: get_registry + run: | + [[ ${HUB} == *.* ]] && REGISTRY=$(echo ${HUB} | cut -d/ -f1) + echo REGISTRY=${REGISTRY} + echo REGISTRY=${REGISTRY} >> $GITHUB_ENV + + - name: docker-login-non-docker + uses: docker/login-action@v1 + if: ${{ env.REGISTRY != null }} + with: + registry: ${{ env.REGISTRY }} + username: ${{ secrets.TEST_HUB_USER }} + password: ${{ secrets.TEST_HUB_PASS }} + + - name: docker-login-docker + uses: docker/login-action@v1 + if: ${{ env.REGISTRY == null }} + with: + username: ${{ secrets.TEST_HUB_USER }} + password: ${{ secrets.TEST_HUB_PASS }} + + - name: Get the tag + id: get_tag + run: echo ::set-output name=TAG::${GITHUB_REF/refs\/tags\/test-/} + shell: bash + + - name: Get minor version + id: get_minor_ver + run: echo ::set-output name=REL_BRANCH_VER::$(echo $TAG | grep -Eo '[0-9]+\.[0-9]+') + shell: bash + env: + TAG: ${{ steps.get_tag.outputs.TAG }} - name: build and push images - run: bash ./tetrateci/create_istio_release.sh + run: bash ./tetrateci/create_multiarch_images.sh env: TAG: ${{ steps.get_tag.outputs.TAG }} TEST: "true" @@ -166,10 +377,78 @@ jobs: BUILD: ${{ contains(github.event.ref, 'fips') && 'fips' }} REL_BRANCH_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + eks-e2e-arm64-test: + name: eks-e2e-arm64-test + runs-on: ubuntu-latest + needs: [create-multiarch-images] + # 'if' condition causes this job to run even if some of the dependent jobs + # have been skipped, e.g. `build_fips_proxy`. + # see https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#example-not-requiring-dependent-jobs-to-be-successful + # if: false + if: ${{ !cancelled() && !failure() }} + + strategy: + fail-fast: false + matrix: + version: ["1.25", "1.26", "1.24"] # available versions according to https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html + + steps: + - name: checkout + uses: actions/checkout@v2 + + - name: aws-config + uses: aws-actions/configure-aws-credentials@v1 + with: + aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} + aws-region: ${{ secrets.AWS_REGION }} + + - name: Get the tag + id: get_tag + run: echo ::set-output name=TAG::${GITHUB_REF/refs\/tags\/test-/} + shell: bash + + - name: Get minor version + id: get_minor_ver + run: echo ::set-output name=REL_BRANCH_VER::$(echo $TAG | grep -Eo '[0-9]+\.[0-9]+') + shell: bash + env: + TAG: ${{ steps.get_tag.outputs.TAG }} + + - uses: actions/cache@v2 + with: + path: ~/go/pkg/mod + key: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }}-go-${{ hashFiles('**/go.sum') }} + restore-keys: | + ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }}-go- + + - name: eks-cluster-create + run: bash ./tetrateci/create_eks_cluster.sh + env: + K8S_VERSION: ${{ matrix.version }} + ISTIO_MINOR_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + TEST_ARCH: arm64 + + - name: integration-tests + run: bash ./tetrateci/test_${ISTIO_MINOR_VER}_arm.sh + env: + TAG: ${{ steps.get_tag.outputs.TAG }} + ISTIO_MINOR_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + CLUSTER: eks-arm64 + K8S_VERSION: ${{ matrix.version }} + + - name: eks-cluster-delete + if: always() + run: bash ./tetrateci/cleanup_eks_cluster.sh + env: + K8S_VERSION: ${{ matrix.version }} + ISTIO_MINOR_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + TEST_ARCH: arm64 + eks-e2e-test: name: eks-e2e-test runs-on: ubuntu-latest - needs: [create-test-images] + needs: [create-multiarch-images] # 'if' condition causes this job to run even if some of the dependent jobs # have been skipped, e.g. `build_fips_proxy`. # see https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#example-not-requiring-dependent-jobs-to-be-successful @@ -178,7 +457,7 @@ jobs: strategy: fail-fast: false matrix: - version: ["1.21", "1.20", "1.19", "1.18"] # available versions according to https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html + version: ["1.26", "1.25", "1.24"] # available versions according to https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html steps: - name: checkout @@ -209,7 +488,6 @@ jobs: key: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }}-go-${{ hashFiles('**/go.sum') }} restore-keys: | ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }}-go- - - name: eks-cluster-create run: bash ./tetrateci/create_eks_cluster.sh env: @@ -230,11 +508,11 @@ jobs: env: K8S_VERSION: ${{ matrix.version }} ISTIO_MINOR_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} - + gke-e2e-test: name: gke-e2e-test runs-on: ubuntu-latest - needs: [create-test-images] + needs: [create-multiarch-images] # 'if' condition causes this job to run even if some of the dependent jobs # have been skipped, e.g. `build_fips_proxy`. # see https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#example-not-requiring-dependent-jobs-to-be-successful @@ -245,15 +523,11 @@ jobs: # available versions according to https://cloud.google.com/kubernetes-engine/versioning matrix: include: - - version: "1.22" + - version: "1.24" channel: "rapid" - - version: "1.21" + - version: "1.25" channel: "regular" - - version: "1.20" - channel: "regular" - - version: "1.19" - channel: "stable" - + steps: - name: checkout uses: actions/checkout@v2 @@ -311,7 +585,7 @@ jobs: if: false name: aks-e2e-test runs-on: ubuntu-latest - needs: [create-test-images] + needs: [create-multiarch-images] strategy: fail-fast: false @@ -372,7 +646,7 @@ jobs: eksd-e2e-test: name: eksd-e2e-test runs-on: ubuntu-latest - needs: [create-test-images, eks-e2e-test] + needs: [create-multiarch-images, eks-e2e-test] strategy: fail-fast: false @@ -433,13 +707,13 @@ jobs: ISTIO_MINOR_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} S3_BUCKET: ${{ secrets.S3_BUCKET }} AWS_REGION: ${{ secrets.AWS_REGION }} - + create-new-tag: name: create-new-tag runs-on: ubuntu-latest # since aks tests are disabled # needs: [eks-e2e-test, gke-e2e-test, aks-e2e-test] - needs: [eksd-e2e-test, gke-e2e-test] + needs: [eks-e2e-arm64-test, eksd-e2e-test, gke-e2e-test] steps: - name: Checkout code @@ -454,9 +728,9 @@ jobs: id: create_release uses: actions/create-release@v1 env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} with: tag_name: ${{ steps.get_tag.outputs.TAG }} - release_name: Tetrate Istio + release_name: Tetrate Istio draft: false - prerelease: false + prerelease: false \ No newline at end of file diff --git a/.github/workflows/make_release.yml b/.github/workflows/make_release.yml index 70540a5961a9..7c3d08b4a2ad 100644 --- a/.github/workflows/make_release.yml +++ b/.github/workflows/make_release.yml @@ -1,7 +1,7 @@ name: Make a release on: - workflow_dispatch: + workflow_dispatch: inputs: tag: description: Tag [e.g. 1.7.8-tetrate-v0] @@ -10,51 +10,65 @@ on: jobs: build_fips_proxy: # Only run this job when the release type equals "fips". - if: contains(github.event.inputs.tag, 'fips') + if: contains(github.event.ref, 'fips') name: build fips proxy binary - timeout-minutes: 1440 # 12 hours - runs-on: ubuntu-latest + timeout-minutes: 840 + runs-on: ['self-hosted','Linux','x64','c5.4xlarge'] env: RELEASE_GCS_PATH: gs://getistio-build/proxy-fips steps: + - name: Get the tag + id: get_tag + run: echo ::set-output name=TAG::${GITHUB_REF#refs/tags/} - name: Get normalized tag + id: get_normalized_ver + run: echo ::set-output name=NORMALIZED_TAG::$(echo $TAG | sed 's/test-//g' | sed 's/-.*//g') + shell: bash + env: + TAG: ${{ steps.get_tag.outputs.TAG }} + - name: Get minor version id: get_minor_ver - run: echo ::set-output name=NORMALIZED_TAG::$(echo $TAG | sed 's/-.*//g') + run: echo ::set-output name=REL_BRANCH_VER::$(echo $TAG | grep -Eo '[0-9]+\.[0-9]+') shell: bash env: - TAG: ${{ github.event.inputs.tag }} - + TAG: ${{ steps.get_tag.outputs.TAG }} - name: Checkout uses: actions/checkout@v2 with: - repository: 'istio/proxy' - ref: ${{ steps.get_minor_ver.outputs.NORMALIZED_TAG }} - + repository: "istio/proxy" + ref: ${{ steps.get_normalized_ver.outputs.NORMALIZED_TAG }} - name: Set up Cloud SDK uses: google-github-actions/setup-gcloud@v0 with: project_id: ${{ secrets.GCP_PROJECT_ID }} service_account_key: ${{ secrets.GCP_SA_KEY }} export_default_credentials: true - - name: Check if already built id: check_already_built + env: + REL_BRANCH_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + TAG: ${{ steps.get_tag.outputs.TAG }} run: | - set +e - SHA=$(git rev-parse --verify HEAD) - gsutil ls ${RELEASE_GCS_PATH} | grep ${SHA} - echo ::set-output name=should_build::$? - + SHA=$(echo $TAG | sed 's/test-//g' | sed 's/[0-9].[0-9][0-9].[0-9]-[A-Za-z]*-//g' | sed 's/-.*//g') + SHOULD_BUILD=1 + SHOULD_COPY=1 + if gsutil ls ${RELEASE_GCS_PATH} | grep "${SHA}[^-]" ; then + SHOULD_BUILD=0 + fi + if [ $(bc <<< "$REL_BRANCH_VER > 1.17") -eq 1 ];then + SHOULD_COPY=0 + fi + echo ::set-output name=should_build::${SHOULD_BUILD} + echo ::set-output name=should_copy::${SHOULD_COPY} - name: Tweak make recipe if: ${{ steps.check_already_built.outputs.should_build == '1' }} - # 1) remove unnecessary dependency on naive "build" target - # 2) remove "-p" flag to not push and build container, + # 1) remove unnecessary dependency on naive "build" target + # 2) remove "-p" flag to not push and build container, # 3) replace RELEASE_GCS_PATH var with the literal due to docker build. # from push_release. run: | ESCAPED_RELEASE_GCS_PATH=$(printf '%s\n' "$RELEASE_GCS_PATH" | sed -e 's/[\/&]/\\&/g') sed -i "s/\-p//g; s/push_release: build/push_release:/g; s/\"\$(RELEASE_GCS_PATH)\"/${ESCAPED_RELEASE_GCS_PATH}/g" Makefile.core.mk - - name: Tweak release-binary script if: ${{ steps.check_already_built.outputs.should_build == '1' }} # 1) Setup in-docker auth against gcloud @@ -66,7 +80,7 @@ jobs: cp scripts/release-binary.sh scripts/release-binary.sh.tmp echo gcloud auth activate-service-account --key-file="/work/${CREDENTIAL_FILE_NAME}" > scripts/release-binary.sh cat scripts/release-binary.sh.tmp >> scripts/release-binary.sh - sed -i 's/release release\-symbol debug/release release\-symbol/' scripts/release-binary.sh + sed -i 's/for config in release release-symbol asan debug/for config in release/g' scripts/release-binary.sh sed -i -n '/Build and publish Wasm plugins/q;p' scripts/release-binary.sh cat scripts/release-binary.sh @@ -77,31 +91,178 @@ jobs: - name: Build and push if: ${{ steps.check_already_built.outputs.should_build == '1' }} env: - BUILD_WITH_CONTAINER: 1 + BUILD_WITH_CONTAINER: 1 run: make push_release - + - name: Put CentOS binary - # Copy the binary built on Ubuntu to CentOS path - this binar cannot run on CentOS/RHEL 7, but fine with CentOS 8. + env: + REL_BRANCH_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + TAG: ${{ steps.get_tag.outputs.TAG }} + # Copy the binary built on Ubuntu to CentOS path - this binary cannot run on CentOS/RHEL 7, but fine with CentOS 8. # The point is that we cannot build FIPS binary directly on CentOS 7 due to the constraints described in the BoringCrypto certification. run: | - SHA=$(git rev-parse --verify HEAD) + SHA=$(echo $TAG | sed 's/test-//g' | sed 's/[0-9].[0-9][0-9].[0-9]-[A-Za-z]*-//g' | sed 's/-.*//g') gsutil cp ${RELEASE_GCS_PATH}/envoy-alpha-${SHA}.tar.gz ${RELEASE_GCS_PATH}/envoy-centos-alpha-${SHA}.tar.gz - name: Put Wasm binary + if: ${{ steps.check_already_built.outputs.should_build == '1' && steps.check_already_built.outputs.should_copy == '1' }} # Copy the built Wasm binary in the upstream to the bucket - Wasm is nothing to do with FIPS stuff. env: UPSTREAM_GCSP_PATH: gs://istio-build/proxy + REL_BRANCH_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + TAG: ${{ steps.get_tag.outputs.TAG }} + run: | + SHA1=$(git rev-parse --verify HEAD) + SHA=$(echo $TAG | sed 's/test-//g' | sed 's/[0-9].[0-9][0-9].[0-9]-[A-Za-z]*-//g' | sed 's/-.*//g') + gsutil cp "${UPSTREAM_GCSP_PATH}/metadata_exchange-${SHA1}.compiled.wasm" "${RELEASE_GCS_PATH}/metadata_exchange-${SHA}.compiled.wasm" + gsutil cp "${UPSTREAM_GCSP_PATH}/metadata_exchange-${SHA1}.compiled.wasm.sha256" "${RELEASE_GCS_PATH}/metadata_exchange-${SHA}.compiled.wasm.sha256" + gsutil cp "${UPSTREAM_GCSP_PATH}/metadata_exchange-${SHA1}.wasm" "${RELEASE_GCS_PATH}/metadata_exchange-${SHA}.wasm" + gsutil cp "${UPSTREAM_GCSP_PATH}/metadata_exchange-${SHA1}.wasm.sha256" "${RELEASE_GCS_PATH}/metadata_exchange-${SHA}.wasm.sha256" + gsutil cp "${UPSTREAM_GCSP_PATH}/stats-${SHA1}.compiled.wasm" "${RELEASE_GCS_PATH}/stats-${SHA}.compiled.wasm" + gsutil cp "${UPSTREAM_GCSP_PATH}/stats-${SHA1}.compiled.wasm.sha256" "${RELEASE_GCS_PATH}/stats-${SHA}.compiled.wasm.sha256" + gsutil cp "${UPSTREAM_GCSP_PATH}/stats-${SHA1}.wasm" "${RELEASE_GCS_PATH}/stats-${SHA}.wasm" + gsutil cp "${UPSTREAM_GCSP_PATH}/stats-${SHA1}.wasm.sha256" "${RELEASE_GCS_PATH}/stats-${SHA}.wasm.sha256" + + build_fips_arm_proxy: + if: contains(github.event.ref, 'fips') + name: build ARM proxy binary + timeout-minutes: 1440 # 12 hours + runs-on: ['self-hosted','Linux','ARM64','arm64-32'] + env: + RELEASE_GCS_PATH: gs://getistio-build/proxy-fips + ARCH_SUFFIX: arm64 + steps: + - name: Get normalized tag + id: get_normalized_ver + run: echo ::set-output name=NORMALIZED_TAG::$(echo $TAG | sed 's/test-//g' | sed 's/-.*//g') + shell: bash + env: + TAG: ${{ steps.get_tag.outputs.TAG }} + - name: Get minor version + id: get_minor_ver + run: echo ::set-output name=REL_BRANCH_VER::$(echo $TAG | grep -Eo '[0-9]+\.[0-9]+') + shell: bash + env: + TAG: ${{ steps.get_tag.outputs.TAG }} + - name: Checkout + uses: actions/checkout@v2 + with: + repository: "istio/proxy" + ref: ${{ steps.get_normalized_ver.outputs.NORMALIZED_TAG }} + - name: Set up Cloud SDK + uses: google-github-actions/setup-gcloud@v0 + with: + project_id: ${{ secrets.GCP_PROJECT_ID }} + service_account_key: ${{ secrets.GCP_SA_KEY }} + export_default_credentials: true + - name: Check if already built + id: check_already_built + env: + REL_BRANCH_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + TAG: ${{ steps.get_tag.outputs.TAG }} run: | - SHA=$(git rev-parse --verify HEAD) - gsutil cp "${UPSTREAM_GCSP_PATH}/metadata_exchange-${SHA}*" ${RELEASE_GCS_PATH}/ - gsutil cp "${UPSTREAM_GCSP_PATH}/stats-${SHA}*" ${RELEASE_GCS_PATH}/ - gsutil cp "${UPSTREAM_GCSP_PATH}/attributegen-${SHA}*" ${RELEASE_GCS_PATH}/ - - make_release: - name: release-builder-run (${{ github.event.inputs.tag }}) - if: ${{ ! failure() }} # Make sure that this job runs even if build == "fips". - runs-on: ubuntu-latest + SHA=$(echo $TAG | sed 's/test-//g' | sed 's/[0-9].[0-9][0-9].[0-9]-[A-Za-z]*-//g' | sed 's/-.*//g') + SHOULD_BUILD=1 + if gsutil ls ${RELEASE_GCS_PATH} | grep "${SHA}[^-]" ; then + SHOULD_BUILD=0 + fi + echo ::set-output name=should_build::${SHOULD_BUILD} + - name: Tweak make recipe + if: ${{ steps.check_already_built.outputs.should_build == '1' }} + # 1) remove unnecessary dependency on naive "build" target + # 2) remove "-p" flag to not push and build container, + # 3) replace RELEASE_GCS_PATH var with the literal due to docker build. + # from push_release. + run: | + ESCAPED_RELEASE_GCS_PATH=$(printf '%s\n' "$RELEASE_GCS_PATH" | sed -e 's/[\/&]/\\&/g') + sed -i "s/\-p//g; s/push_release: build/push_release:/g; s/\"\$(RELEASE_GCS_PATH)\"/${ESCAPED_RELEASE_GCS_PATH}/g" Makefile.core.mk + - name: Tweak release-binary script + if: ${{ steps.check_already_built.outputs.should_build == '1' }} + # 1) Setup in-docker auth against gcloud + # 2) Remove unnecessary debug build + # 3) Remove unnecessary Wasm build and publish + # Note that /work is the mount destination of the current home of make command with BUILD_WITH_CONTAINER=1. + run: | + CREDENTIAL_FILE_NAME=$(echo ${GOOGLE_APPLICATION_CREDENTIALS} | awk -F/ '{print $NF}') + cp scripts/release-binary.sh scripts/release-binary.sh.tmp + echo gcloud auth activate-service-account --key-file="/work/${CREDENTIAL_FILE_NAME}" > scripts/release-binary.sh + cat scripts/release-binary.sh.tmp >> scripts/release-binary.sh + sed -i 's/release release\-symbol debug/release release\-symbol/' scripts/release-binary.sh + sed -i '/if \[\[ "${BUILD_ENVOY_BINARY_ONLY}" !/, +6d' scripts/release-binary.sh + sed -i -n '/Build and publish Wasm plugins/q;p' scripts/release-binary.sh + cat scripts/release-binary.sh + - name: Add FIPS flag + if: ${{ steps.check_already_built.outputs.should_build == '1' }} + run: echo "build --define boringssl=fips" >> .bazelrc + - name: Build and push + if: ${{ steps.check_already_built.outputs.should_build == '1' }} + env: + BUILD_ENVOY_BINARY_ONLY: 1 + BUILD_WITH_CONTAINER: 1 + run: make push_release + + create-images-arm64: + name: create-images-arm64 + runs-on: ["self-hosted", "arm64"] + env: + TARGETARCH: arm64 + DOCKER_ARCHITECTURES: linux/arm64 + HUB: ${{ secrets.CLOUDSMITH_HUB }} + needs: [build_arm_proxy] + # 'if' condition causes this job to run even if some of the dependent jobs + # have been skipped, e.g. `build_fips_proxy`. + # see https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#example-not-requiring-dependent-jobs-to-be-successful + if: "contains(github.event.ref, 'multiarch')" + + steps: + - name: Checkout to choosen tag + uses: actions/checkout@v2 + with: + ref: ${{ github.event.inputs.tag }} + fetch-depth: 0 + + - name: Login to CloudSmith + uses: docker/login-action@v1 + with: + registry: ${{ env.HUB }} + username: ${{ secrets.CLOUDSMITH_USER }} + password: ${{ secrets.CLOUDSMITH_API_KEY }} + + - name: Get Registry + id: get_registry + run: | + [[ ${HUB} == *.* ]] && REGISTRY=$(echo ${HUB} | cut -d/ -f1) + echo REGISTRY=${REGISTRY} + echo REGISTRY=${REGISTRY} >> $GITHUB_ENV + + - name: Get the tag + id: get_tag + run: echo ::set-output name=TAG::${GITHUB_REF/refs\/tags\/test-/} + shell: bash + + - name: Get minor version + id: get_minor_ver + run: echo ::set-output name=REL_BRANCH_VER::$(echo $TAG | grep -Eo '[0-9]+\.[0-9]+') + shell: bash + env: + TAG: ${{ github.event.inputs.tag }} + + - name: build and push images + run: bash ./tetrateci/create_istio_release.sh + env: + CLOUDSMITH_USER: ${{ secrets.CLOUDSMITH_USER }} + CLOUDSMITH_API_KEY: ${{ secrets.CLOUDSMITH_API_KEY }} + TAG: ${{ github.event.inputs.tag }} + REL_BRANCH_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + + create-images-amd64: + name: create-images-amd64 + runs-on: ['self-hosted','Linux','x64','c5.4xlarge'] needs: [build_fips_proxy] + # 'if' condition causes this job to run even if some of the dependent jobs + # have been skipped, e.g. `build_fips_proxy`. + # see https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#example-not-requiring-dependent-jobs-to-be-successful + if: ${{ !cancelled() && !failure() }} env: HUB: ${{ secrets.CLOUDSMITH_HUB }} @@ -119,6 +280,85 @@ jobs: username: ${{ secrets.CLOUDSMITH_USER }} password: ${{ secrets.CLOUDSMITH_API_KEY }} + - name: Get Registry + id: get_registry + run: | + [[ ${HUB} == *.* ]] && REGISTRY=$(echo ${HUB} | cut -d/ -f1) + echo REGISTRY=${REGISTRY} + echo REGISTRY=${REGISTRY} >> $GITHUB_ENV + + - name: Get minor version + id: get_minor_ver + run: echo ::set-output name=REL_BRANCH_VER::$(echo $TAG | grep -Eo '[0-9]+\.[0-9]+') + shell: bash + env: + TAG: ${{ github.event.inputs.tag }} + + - name: build and push images + run: bash ./tetrateci/create_istio_release.sh + env: + CLOUDSMITH_USER: ${{ secrets.CLOUDSMITH_USER }} + CLOUDSMITH_API_KEY: ${{ secrets.CLOUDSMITH_API_KEY }} + TAG: ${{ github.event.inputs.tag }} + REL_BRANCH_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} + Scan: + name: Scanner + runs-on: ubuntu-20.04 + needs: [create-images-amd64] + if: ${{ !cancelled() && !failure() }} + steps: + - name: Checkout code + uses: actions/checkout@v3 + - name: Get minor version + id: get_minor_ver + run: echo ::set-output name=REL_BRANCH_VER::$(echo $TAG | grep -Eo '[0-9]+\.[0-9]+') + shell: bash + env: + TAG: ${{ steps.get_tag.outputs.TAG }} + - name: List docker images + run: bash ./tetrateci/images.sh && pwd + env: + TAG: ${{ github.event.inputs.tag }} + - name: Scan images for CRITICAL vulnerabilities + uses: docker://ghcr.io/aquasecurity/trivy + with: + entrypoint: /bin/sh + args: -c "trivy image --clear-cache; for img in $(cat list.txt); do trivy image --exit-code 0 --severity HIGH,CRITICAL --no-progress --quiet --format sarif $img >> $img.sarif ;done " + - name: Upload Trivy scan results to GitHub Security tab + uses: github/codeql-action/upload-sarif@v1 + with: + sarif_file: containers.istio.tetratelabs.com + create-multiarch-images: + name: create-multiarch-images + runs-on: ubuntu-latest + needs: [create-images-amd64, create-images-arm64] + # 'if' condition causes this job to run even if some of the dependent jobs + # have been skipped, e.g. `build_fips_proxy`. + # see https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#example-not-requiring-dependent-jobs-to-be-successful + if: "contains(github.event.ref, 'multiarch')" + env: + HUB: ${{ secrets.CLOUDSMITH_HUB }} + steps: + - name: Checkout to choosen tag + uses: actions/checkout@v2 + with: + ref: ${{ github.event.inputs.tag }} + fetch-depth: 0 + + - name: Login to CloudSmith + uses: docker/login-action@v1 + with: + registry: ${{ env.HUB }} + username: ${{ secrets.CLOUDSMITH_USER }} + password: ${{ secrets.CLOUDSMITH_API_KEY }} + + - name: Get Registry + id: get_registry + run: | + [[ ${HUB} == *.* ]] && REGISTRY=$(echo ${HUB} | cut -d/ -f1) + echo REGISTRY=${REGISTRY} + echo REGISTRY=${REGISTRY} >> $GITHUB_ENV + - name: Get minor version id: get_minor_ver run: echo ::set-output name=REL_BRANCH_VER::$(echo $TAG | grep -Eo '[0-9]+\.[0-9]+') @@ -126,10 +366,10 @@ jobs: env: TAG: ${{ github.event.inputs.tag }} - - name: Run release builder + - name: build and push images + run: bash ./tetrateci/create_multiarch_images.sh env: CLOUDSMITH_USER: ${{ secrets.CLOUDSMITH_USER }} CLOUDSMITH_API_KEY: ${{ secrets.CLOUDSMITH_API_KEY }} TAG: ${{ github.event.inputs.tag }} REL_BRANCH_VER: ${{ steps.get_minor_ver.outputs.REL_BRANCH_VER }} - run: ./tetrateci/create_istio_release.sh diff --git a/.github/workflows/tools.yml b/.github/workflows/tools.yml index 2c24391c7137..48ca7b8a34e7 100644 --- a/.github/workflows/tools.yml +++ b/.github/workflows/tools.yml @@ -8,7 +8,7 @@ on: required: true jobs: - build_tools_docker_arm64: + build_tools_docker: name: build tools docker runs-on: ["self-hosted", "arm64"] env: diff --git a/.gitignore b/.gitignore deleted file mode 100644 index e69de29bb2d1..000000000000 diff --git a/tetrateci/1.12/test/skip.d/eks-arm64 b/tetrateci/1.12/test/skip.d/eks-arm64 new file mode 100644 index 000000000000..6a584a50939a --- /dev/null +++ b/tetrateci/1.12/test/skip.d/eks-arm64 @@ -0,0 +1,44 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. + +istio.io/istio/tests/integration/helm/upgrade=* + +istio.io/istio/tests/integration/pilot=* +istio.io/istio/tests/integration/pilot/endpointslice=* +istio.io/istio/tests/integration/pilot/revisions=TestMultiRevision + +istio.io/istio/tests/integration/security=* +istio.io/istio/tests/integration/security/sds_ingress=* +istio.io/istio/tests/integration/security/sds_tls_origination=TestSimpleTlsOrigination TestMutualTlsOrigination + +istio.io/istio/tests/integration/telemetry/stats/prometheus/nullvm=TestAccessLogs +istio.io/istio/tests/integration/telemetry/tracing/zipkin/clienttracing=TestClientTracing +istio.io/istio/tests/integration/telemetry/tracing/zipkin/servertracing=TestProxyTracing + +istio.io/istio/tests/integration/telemetry/stackdriver=* +istio.io/istio/tests/integration/telemetry/stackdriver/api=* +istio.io/istio/tests/integration/telemetry/stackdriver/vm=* + +istio.io/istio/tests/integration/telemetry/tracing/opencensusagent=* +istio.io/istio/tests/integration/telemetry/policy=* diff --git a/tetrateci/1.13/test/skip.d/eks b/tetrateci/1.13/test/skip.d/eks index 7282571932de..701835417d81 100644 --- a/tetrateci/1.13/test/skip.d/eks +++ b/tetrateci/1.13/test/skip.d/eks @@ -22,9 +22,9 @@ # # indicates that tests for the package `` should not be run at all. -istio.io/istio/tests/integration/helm/upgrade=* -istio.io/istio/tests/integration/pilot/revisions=* -istio.io/istio/tests/integration/security/sds_ingress/quic=* +istio.io/istio/tests/integration/helm/upgrade=TestRevisionTagsUpgradeFromPrevious +MinorRelease + istio.io/istio/tests/integration/pilot=TestIngress/status istio.io/istio/tests/integration/telemetry/tracing/zipkin/servertracing=TestProxyTracing diff --git a/tetrateci/1.13/test/skip.d/eks-arm64 b/tetrateci/1.13/test/skip.d/eks-arm64 new file mode 100644 index 000000000000..8b1b324a06b5 --- /dev/null +++ b/tetrateci/1.13/test/skip.d/eks-arm64 @@ -0,0 +1,43 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. + +istio.io/istio/tests/integration/helm/upgrade=* + +istio.io/istio/tests/integration/pilot=* +istio.io/istio/tests/integration/pilot/endpointslice=* +istio.io/istio/tests/integration/pilot/revisions=TestMultiRevision + +istio.io/istio/tests/integration/security=* +istio.io/istio/tests/integration/security/sds_ingress/quic=* +istio.io/istio/tests/integration/security/https_jwt=* +istio.io/istio/tests/integration/security/sds_ingress=* +istio.io/istio/tests/integration/security/sds_tls_origination=TestSimpleTlsOrigination TestMutualTlsOrigination + +istio.io/istio/tests/integration/telemetry/tracing/zipkin/clienttracing=TestClientTracing +istio.io/istio/tests/integration/telemetry/tracing/zipkin/servertracing=TestProxyTracing +istio.io/istio/tests/integration/telemetry/stackdriver=* +istio.io/istio/tests/integration/telemetry/stackdriver/api=* +istio.io/istio/tests/integration/telemetry/stackdriver/vm=* +istio.io/istio/tests/integration/telemetry/tracing/opencensusagent=* +istio.io/istio/tests/integration/telemetry/policy=* diff --git a/tetrateci/1.13/test/skip.d/gke b/tetrateci/1.13/test/skip.d/gke index c7a89d81e452..998342b10e03 100644 --- a/tetrateci/1.13/test/skip.d/gke +++ b/tetrateci/1.13/test/skip.d/gke @@ -20,14 +20,11 @@ # =* # ``` # -# Indicates that tests for the package `` should not be run at all. +# indicates that tests for the package `` should not be run at all. +istio.io/istio/tests/integration/helm/upgrade=TestRevisionTagsUpgradeFromPrevious +MinorRelease -istio.io/istio/tests/integration/helm=* -istio.io/istio/tests/integration/helm/upgrade=* -istio.io/istio/tests/integration/security=TestReachability/beta-mtls-off -istio.io/istio/tests/integration/security/sds_ingress/quic=* -istio.io/istio/tests/integration/security/mtls_first_party_jwt=* -istio.io/istio/tests/integration/security/https_jwt=TestJWTHTTPS/jwt-authn/a/to_b/valid-token-forward-remote-jwks istio.io/istio/tests/integration/pilot=TestGateway TestIngress TestDescribe TestTraffic + istio.io/istio/tests/integration/telemetry/stackdriver=* diff --git a/tetrateci/1.14/test/skip.d/eks b/tetrateci/1.14/test/skip.d/eks new file mode 100644 index 000000000000..701835417d81 --- /dev/null +++ b/tetrateci/1.14/test/skip.d/eks @@ -0,0 +1,33 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. + +istio.io/istio/tests/integration/helm/upgrade=TestRevisionTagsUpgradeFromPrevious +MinorRelease + +istio.io/istio/tests/integration/pilot=TestIngress/status + +istio.io/istio/tests/integration/telemetry/tracing/zipkin/servertracing=TestProxyTracing + +istio.io/istio/tests/integration/telemetry/stackdriver=* +istio.io/istio/tests/integration/telemetry/stackdriver/vm=* diff --git a/tetrateci/1.14/test/skip.d/eks-arm64 b/tetrateci/1.14/test/skip.d/eks-arm64 new file mode 100644 index 000000000000..087f725dac0c --- /dev/null +++ b/tetrateci/1.14/test/skip.d/eks-arm64 @@ -0,0 +1,27 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. + +istio.io/istio/tests/integration/pilot=TestGatewayConformance +istio.io/istio/tests/integration/helm/upgrade=* +istio.io/istio/tests/integration/security/sds_ingress/quic=* diff --git a/tetrateci/1.14/test/skip.d/gke b/tetrateci/1.14/test/skip.d/gke new file mode 100644 index 000000000000..cfd5067d50c9 --- /dev/null +++ b/tetrateci/1.14/test/skip.d/gke @@ -0,0 +1,37 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. + +istio.io/istio/tests/integration/helm/upgrade=TestRevisionTagsUpgradeFromPrevious +MinorRelease + +istio.io/istio/tests/integration/pilot=TestGateway TestIngress TestDescribe TestTraffic +istio.io/istio/tests/integration/helm=* +istio.io/istio/tests/integration/helm/upgrade=* +istio.io/istio/tests/integration/security=TestReachability/beta-mtls-off +istio.io/istio/tests/integration/security/sds_ingress/quic=* +istio.io/istio/tests/integration/security/mtls_first_party_jwt=* +istio.io/istio/tests/integration/security/https_jwt=TestJWTHTTPS/jwt-authn/a/to_b/valid-token-forward-remote-jwks +istio.io/istio/tests/integration/pilot=TestGateway TestIngress TestDescribe TestTraffic +istio.io/istio/tests/integration/telemetry/stackdriver=* +istio.io/istio/tests/integration/telemetry/stackdriver=* diff --git a/tetrateci/1.15/test/skip.d/eks b/tetrateci/1.15/test/skip.d/eks new file mode 100644 index 000000000000..756ef590e7f1 --- /dev/null +++ b/tetrateci/1.15/test/skip.d/eks @@ -0,0 +1,27 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. + +istio.io/istio/tests/integration/pilot=TestGatewayConformance TestIngress/status +istio.io/istio/tests/integration/helm/upgrade=* +istio.io/istio/tests/integration/security/sds_ingress/quic=* diff --git a/tetrateci/1.15/test/skip.d/eks-arm64 b/tetrateci/1.15/test/skip.d/eks-arm64 new file mode 100644 index 000000000000..087f725dac0c --- /dev/null +++ b/tetrateci/1.15/test/skip.d/eks-arm64 @@ -0,0 +1,27 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. + +istio.io/istio/tests/integration/pilot=TestGatewayConformance +istio.io/istio/tests/integration/helm/upgrade=* +istio.io/istio/tests/integration/security/sds_ingress/quic=* diff --git a/tetrateci/1.15/test/skip.d/gke b/tetrateci/1.15/test/skip.d/gke new file mode 100644 index 000000000000..66c8c4a2eb92 --- /dev/null +++ b/tetrateci/1.15/test/skip.d/gke @@ -0,0 +1,37 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. + +istio.io/istio/tests/integration/pilot=TestGateway TestIngress TestDescribe TestTraffic TestGatewayConformance TestTunnelingOutboundTraffic +istio.io/istio/tests/integration/pilot/revisioncmd=* +istio.io/istio/tests/integration/helm=* +istio.io/istio/tests/integration/helm/upgrade=* +istio.io/istio/tests/integration/security=TestReachability/beta-mtls-off TestAuthz_EgressGateway TestReachability/global_no_peer_authn/http +istio.io/istio/tests/integration/security/sds_ingress/quic=* +istio.io/istio/tests/integration/security/mtls_first_party_jwt=* +istio.io/istio/tests/integration/security/https_jwt=TestJWTHTTPS/jwt-authn/a/to_b/valid-token-forward-remote-jwks +istio.io/istio/tests/integration/security/filebased_tls_origination=TestEgressGatewayTls +istio.io/istio/tests/integration/telemetry/outboundtrafficpolicy=* +istio.io/istio/tests/integration/telemetry/stats/prometheus/nullvm=* +istio.io/istio/tests/integration/telemetry/stats/prometheus/wasm=* +istio.io/istio/tests/integration/telemetry/stackdriver=* diff --git a/tetrateci/1.16/test/skip.d/eks b/tetrateci/1.16/test/skip.d/eks new file mode 100644 index 000000000000..40ed004341a6 --- /dev/null +++ b/tetrateci/1.16/test/skip.d/eks @@ -0,0 +1,29 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. + +istio.io/istio/tests/integration/telemetry/stackdriver=* +istio.io/istio/tests/integration/telemetry/stackdriver/vm=* +istio.io/istio/tests/integration/pilot=TestGatewayConformance TestIngress/status +istio.io/istio/tests/integration/helm/upgrade=* +istio.io/istio/tests/integration/security/sds_ingress/quic=* diff --git a/tetrateci/1.16/test/skip.d/eks-arm64 b/tetrateci/1.16/test/skip.d/eks-arm64 new file mode 100644 index 000000000000..2df591d54737 --- /dev/null +++ b/tetrateci/1.16/test/skip.d/eks-arm64 @@ -0,0 +1,30 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. +istio.io/istio/tests/integration/operator=TestPostInstallControlPlaneVerification +istio.io/istio/tests/integration/pilot=TestGatewayConformance +istio.io/istio/tests/integration/helm/upgrade=* +istio.io/istio/tests/integration/security/sds_ingress/quic=* +istio.io/istio/tests/integration/telemetry/stackdriver=* +istio.io/istio/tests/integration/telemetry/stackdriver/vm=* +istio.io/istio/tests/integration/telemetry_envoyfilter_nullvm=TestDashboard diff --git a/tetrateci/1.16/test/skip.d/gke b/tetrateci/1.16/test/skip.d/gke new file mode 100644 index 000000000000..66c8c4a2eb92 --- /dev/null +++ b/tetrateci/1.16/test/skip.d/gke @@ -0,0 +1,37 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. + +istio.io/istio/tests/integration/pilot=TestGateway TestIngress TestDescribe TestTraffic TestGatewayConformance TestTunnelingOutboundTraffic +istio.io/istio/tests/integration/pilot/revisioncmd=* +istio.io/istio/tests/integration/helm=* +istio.io/istio/tests/integration/helm/upgrade=* +istio.io/istio/tests/integration/security=TestReachability/beta-mtls-off TestAuthz_EgressGateway TestReachability/global_no_peer_authn/http +istio.io/istio/tests/integration/security/sds_ingress/quic=* +istio.io/istio/tests/integration/security/mtls_first_party_jwt=* +istio.io/istio/tests/integration/security/https_jwt=TestJWTHTTPS/jwt-authn/a/to_b/valid-token-forward-remote-jwks +istio.io/istio/tests/integration/security/filebased_tls_origination=TestEgressGatewayTls +istio.io/istio/tests/integration/telemetry/outboundtrafficpolicy=* +istio.io/istio/tests/integration/telemetry/stats/prometheus/nullvm=* +istio.io/istio/tests/integration/telemetry/stats/prometheus/wasm=* +istio.io/istio/tests/integration/telemetry/stackdriver=* diff --git a/tetrateci/1.17/test/skip.d/eks b/tetrateci/1.17/test/skip.d/eks new file mode 100644 index 000000000000..d94ed36c1f77 --- /dev/null +++ b/tetrateci/1.17/test/skip.d/eks @@ -0,0 +1,32 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. +istio.io/istio/tests/integration/telemetry/stackdriver=* +istio.io/istio/tests/integration/telemetry/stackdriver/vm=* +istio.io/istio/tests/integration/telemetry/api=TestAccessLogsMode +istio.io/istio/tests/integration/pilot=TestGatewayConformance TestIngress/status +istio.io/istio/tests/integration/helm=TestDefaultInstall TestInstallWithFirstPartyJwt +istio.io/istio/tests/integration/operator=TestPostInstallControlPlaneVerification +istio.io/istio/tests/integration/security=TestReachability/global_no_peer_authn TestReachability/migration_tls_mutual TestReachability/migration_no_tls TestReachability/mtls_strict +istio.io/istio/tests/integration/helm/upgrade=* +istio.io/istio/tests/integration/security/sds_ingress/quic=* diff --git a/tetrateci/1.17/test/skip.d/eks-arm64 b/tetrateci/1.17/test/skip.d/eks-arm64 new file mode 100644 index 000000000000..eab40ca7e49e --- /dev/null +++ b/tetrateci/1.17/test/skip.d/eks-arm64 @@ -0,0 +1,31 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. +istio.io/istio/tests/integration/operator=TestPostInstallControlPlaneVerification +istio.io/istio/tests/integration/pilot=TestGatewayConformance +istio.io/istio/tests/integration/helm=TestDefaultInstall TestInstallWithFirstPartyJwt +istio.io/istio/tests/integration/helm/upgrade=* +istio.io/istio/tests/integration/security/sds_ingress/quic=* +istio.io/istio/tests/integration/telemetry/stackdriver=* +istio.io/istio/tests/integration/telemetry/stackdriver/vm=* +istio.io/istio/tests/integration/telemetry_envoyfilter_nullvm=TestDashboard diff --git a/tetrateci/1.17/test/skip.d/gke b/tetrateci/1.17/test/skip.d/gke new file mode 100644 index 000000000000..66c8c4a2eb92 --- /dev/null +++ b/tetrateci/1.17/test/skip.d/gke @@ -0,0 +1,37 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. + +istio.io/istio/tests/integration/pilot=TestGateway TestIngress TestDescribe TestTraffic TestGatewayConformance TestTunnelingOutboundTraffic +istio.io/istio/tests/integration/pilot/revisioncmd=* +istio.io/istio/tests/integration/helm=* +istio.io/istio/tests/integration/helm/upgrade=* +istio.io/istio/tests/integration/security=TestReachability/beta-mtls-off TestAuthz_EgressGateway TestReachability/global_no_peer_authn/http +istio.io/istio/tests/integration/security/sds_ingress/quic=* +istio.io/istio/tests/integration/security/mtls_first_party_jwt=* +istio.io/istio/tests/integration/security/https_jwt=TestJWTHTTPS/jwt-authn/a/to_b/valid-token-forward-remote-jwks +istio.io/istio/tests/integration/security/filebased_tls_origination=TestEgressGatewayTls +istio.io/istio/tests/integration/telemetry/outboundtrafficpolicy=* +istio.io/istio/tests/integration/telemetry/stats/prometheus/nullvm=* +istio.io/istio/tests/integration/telemetry/stats/prometheus/wasm=* +istio.io/istio/tests/integration/telemetry/stackdriver=* diff --git a/tetrateci/1.18/test/skip.d/eks b/tetrateci/1.18/test/skip.d/eks new file mode 100644 index 000000000000..2a654739e2c8 --- /dev/null +++ b/tetrateci/1.18/test/skip.d/eks @@ -0,0 +1,32 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. +istio.io/istio/tests/integration/telemetry/stackdriver=* +istio.io/istio/tests/integration/telemetry/stackdriver/vm=* +istio.io/istio/tests/integration/telemetry/api=TestAccessLogsMode +istio.io/istio/tests/integration/pilot=TestGatewayConformance TestIngress/status TestGateway +istio.io/istio/tests/integration/helm=TestDefaultInstall TestInstallWithFirstPartyJwt +istio.io/istio/tests/integration/operator=TestPostInstallControlPlaneVerification +istio.io/istio/tests/integration/security=TestReachability/global_no_peer_authn TestReachability/migration_tls_mutual TestReachability/migration_no_tls TestReachability/mtls_strict +istio.io/istio/tests/integration/helm/upgrade=* +istio.io/istio/tests/integration/security/sds_ingress/quic=* diff --git a/tetrateci/1.18/test/skip.d/eks-arm64 b/tetrateci/1.18/test/skip.d/eks-arm64 new file mode 100644 index 000000000000..eab40ca7e49e --- /dev/null +++ b/tetrateci/1.18/test/skip.d/eks-arm64 @@ -0,0 +1,31 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. +istio.io/istio/tests/integration/operator=TestPostInstallControlPlaneVerification +istio.io/istio/tests/integration/pilot=TestGatewayConformance +istio.io/istio/tests/integration/helm=TestDefaultInstall TestInstallWithFirstPartyJwt +istio.io/istio/tests/integration/helm/upgrade=* +istio.io/istio/tests/integration/security/sds_ingress/quic=* +istio.io/istio/tests/integration/telemetry/stackdriver=* +istio.io/istio/tests/integration/telemetry/stackdriver/vm=* +istio.io/istio/tests/integration/telemetry_envoyfilter_nullvm=TestDashboard diff --git a/tetrateci/1.18/test/skip.d/gke b/tetrateci/1.18/test/skip.d/gke new file mode 100644 index 000000000000..66c8c4a2eb92 --- /dev/null +++ b/tetrateci/1.18/test/skip.d/gke @@ -0,0 +1,37 @@ +# e2e tests to skip (until a long-term fix is found) +# +# Each line has format: +# +# ```text +# = ... +# ``` +# +# where +# 1. - is a name of a package with Istio e2e tests, e.g. +# `istio.io/istio/tests/integration/pilot` +# 2. - is a regexp that matches unit tests to skip, e.g. +# 'TestA', 'TestA|TestB|TestC', 'TestA/case-b', etc. +# Each `` value will be translated into the +# `--istio.test.skip` option of the Istio Test Framework. +# +# A special case, +# +# ```text +# =* +# ``` +# +# indicates that tests for the package `` should not be run at all. + +istio.io/istio/tests/integration/pilot=TestGateway TestIngress TestDescribe TestTraffic TestGatewayConformance TestTunnelingOutboundTraffic +istio.io/istio/tests/integration/pilot/revisioncmd=* +istio.io/istio/tests/integration/helm=* +istio.io/istio/tests/integration/helm/upgrade=* +istio.io/istio/tests/integration/security=TestReachability/beta-mtls-off TestAuthz_EgressGateway TestReachability/global_no_peer_authn/http +istio.io/istio/tests/integration/security/sds_ingress/quic=* +istio.io/istio/tests/integration/security/mtls_first_party_jwt=* +istio.io/istio/tests/integration/security/https_jwt=TestJWTHTTPS/jwt-authn/a/to_b/valid-token-forward-remote-jwks +istio.io/istio/tests/integration/security/filebased_tls_origination=TestEgressGatewayTls +istio.io/istio/tests/integration/telemetry/outboundtrafficpolicy=* +istio.io/istio/tests/integration/telemetry/stats/prometheus/nullvm=* +istio.io/istio/tests/integration/telemetry/stats/prometheus/wasm=* +istio.io/istio/tests/integration/telemetry/stackdriver=* diff --git a/tetrateci/ci_workflow.md b/tetrateci/ci_workflow.md new file mode 100644 index 000000000000..bbf288e9142b --- /dev/null +++ b/tetrateci/ci_workflow.md @@ -0,0 +1,26 @@ +## Github Workflows +### backport_commits.yml +Creates a backports PR for any commit made to the `tetrate-workflow` branch to any branch which conform those regexes +- `origin/tetrate-release-[[:digit:]]+.[[:digit:]]+$` +- `origin/tetratefips-release-[[:digit:]]+.[[:digit:]]+$` + +The script is adapted from [here](https://github.com/repo-sync/pull-request). The catch is the script only ports changes if there are on the `tetrateci` or `.github/workflows` folders. Merging with a commit or rebasing is not used to avoid merge conflicts. + +### sync_fork.yml +Runs every midnight and checks if there are any new tag on `istio/istio`, if no new tags are founds the action is done. On the other hand if there are new tags, the script loops through them creating corresponding `test-tetrate-x.y.z-v0` & `test-tetratefips-x.y.z-v0` tags and merging them with `tetrate-release-x.y` & `tetratefips-release-x.y` branches. + +Also pushes the archives from `istio/istio` release to the cloudsmith repo. + +### e2e_tests.yml +Runs if there are any tags pushed with `test-` prefix. Utilizes `istio/release-builder` to generate docker images which are to be used for subsequent testing. The one thing to keep in mind it, the same script is used for making the releases and we only differentiate that based on whether the `TEST` environment variable is defined or not. + +Subsequent 4 jobs runs the istio integration tests on applicable versions of eks, gke, aks and eksd, though some of them are disable for various reasons for now. All the platforms have corresponding create and cleanup scripts. The `version_check.py` has a matrix of istio versions vs k8s versions which determines which versions we need to get the istio release tested on. + +All the minor versions of istio have their own testing scripts, the reason being there are patches which need to be applied before testing so the tests dont fail. The tests being written for `kind` have some default assumptions which might not be applicable for all the platforms we test on. + +After the tests pass, the `test-` prefix is stripped off the current tag and the tree is tagged with the remaining, cutting a release with something similar to `tetrate-x.y.z-vn`. + +Then the release builder is again triggered to create the release images and archives with `tetrate-x.y.z-vn` tag but this time without defining the `TEST` environment variable. A fips compliant build is only triggered if the tag contains `fips` in it. The only difference between fips and non fips build is the `Go` we are using. The `create_istio_release.sh` script sets up the environment manually instead if using the docker image is due to some restrictions in the Github Actions, it becomes a bit hard to procure the logs and monitor the whole process. + +### make_release.yml +It is same as the last process of `e2e_tests.yml` but with a manual trigger. Changes made to any of them must be backported to the other one, since they more or less do the same thing. diff --git a/tetrateci/cleanup_eks_cluster.sh b/tetrateci/cleanup_eks_cluster.sh index cdf74b12ec11..1f2cfea68f80 100755 --- a/tetrateci/cleanup_eks_cluster.sh +++ b/tetrateci/cleanup_eks_cluster.sh @@ -6,7 +6,7 @@ set -o errexit set -o pipefail SHA8=$(git rev-parse --short $GITHUB_SHA) -SUFFIX=$(sed 's/\.//g' <<< $K8S_VERSION) +SUFFIX=$(sed 's/\.//g' <<< $K8S_VERSION.$TEST_ARCH) CLUSTER_NAME="test-istio-$SHA8-$SUFFIX" echo "Deleting eks cluster $CLUSTER_NAME" eksctl delete cluster --name $CLUSTER_NAME diff --git a/tetrateci/create_eks_cluster.sh b/tetrateci/create_eks_cluster.sh index 8a229054e08f..5cfe80b02584 100755 --- a/tetrateci/create_eks_cluster.sh +++ b/tetrateci/create_eks_cluster.sh @@ -24,8 +24,13 @@ then fi SHA8=$(git rev-parse --short $GITHUB_SHA) -SUFFIX=$(sed 's/\.//g' <<< $K8S_VERSION) +SUFFIX=$(sed 's/\.//g' <<< $K8S_VERSION.$TEST_ARCH) CLUSTER_NAME="test-istio-$SHA8-$SUFFIX" +NODE_TYPE="m6a.4xlarge" +if [[ "${TEST_ARCH}" = "arm64" ]]; then + NODE_TYPE="m6g.4xlarge" +fi echo "creating a eks cluster with \"$CLUSTER_NAME\" name..." -eksctl create cluster --name $CLUSTER_NAME --version $K8S_VERSION --nodes 3 --node-type m5.xlarge +eksctl create cluster --name $CLUSTER_NAME --version $K8S_VERSION --nodes 3 --node-type $NODE_TYPE --tags tetrate:owner=psb,tetrate:purpose=development,tetrate:team=eng:tid,tetrate:customer=internal,tetrate:lifespan=ongoing + diff --git a/tetrateci/create_istio_release.sh b/tetrateci/create_istio_release.sh index 3ce31fc446cb..afcca27c8f6b 100755 --- a/tetrateci/create_istio_release.sh +++ b/tetrateci/create_istio_release.sh @@ -6,25 +6,23 @@ set -x BASEDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd )" -## Set up apporiate go version -if [[ ${TAG} =~ "fips" ]]; then - echo "Set up FIPS compliant Golang" - source ${BASEDIR}/tetrateci/setup_boring_go.sh -else - echo "Set up Golang" - source ${BASEDIR}/tetrateci/setup_go.sh -fi +sudo rm -rf /usr/local/go + +source ${BASEDIR}/tetrateci/setup_go.sh + + ## Set up release-builder # BOM is needed for generating bill of materials, required by Istio since 1.13, https://github.com/istio/release-builder/pull/893 -go install sigs.k8s.io/bom/cmd/bom@v0.2.2 -cp /home/runner/go/bin/bom /usr/local/bin/ +# go install sigs.k8s.io/bom/cmd/bom@v0.2.2 +# sudo cp /home/runner/go/bin/bom /usr/local/bin/ sudo gem install fpm sudo apt-get install go-bindata -y export BRANCH=release-${REL_BRANCH_VER} cd .. +rm -rf release-builder git clone https://github.com/istio/release-builder --branch ${BRANCH} @@ -48,10 +46,21 @@ if [[ ${TAG} =~ "fips" ]]; then # Escape '/' PROXY_DISTROLESS_BASE_ESCAPED=$(sed 's/\//\\\//g' <<< ${PROXY_DISTROLESS_BASE}) sed -i "s/.*as distroless/${PROXY_DISTROLESS_BASE_ESCAPED}/" ${BASEDIR}/operator/docker/Dockerfile.operator + export ISTIO_ENVOY_BASE_URL=https://storage.googleapis.com/getistio-build/proxy-fips +fi + - export ISTIO_ENVOY_BASE_URL=https://storage.googleapis.com/getistio-build/proxy-fips +if [[ "$(uname -m)" = "aarch64" ]]; then + sed -i 's/gcr\.io\/istio-release/gcr\.io\/tetrate-istio-arm/' $(find ${BASEDIR} | grep Dockerfile) + sed -i 's/gcr\.io\/tetrate-istio-arm\/iptables@sha256:[0-9a-f]*/gcr\.io\/istio-release\/iptables@sha256:8efeb55ddf08f2f513d303b8f0ff42c9f08f355de2f4124e641d209d11a6af91/' ${BASEDIR}/pilot/docker/Dockerfile.proxyv2 + export ISTIO_ENVOY_BASE_URL=https://storage.googleapis.com/getistio-build/proxy-arm + export BASE_VERSION=1602e34d9524a2a312907aab276bcd7100da52df # 1.12 + fi + + + # HACK : default manifest from release builder is modified echo "Generating the manifests" # we are generating the different yamls for both the archive & docker image builds which are saved to release-builder folder @@ -65,6 +74,11 @@ echo "TEST flag is '${TEST:-}'" echo "Getting into release builder" cd release-builder + +if [[ "$(uname -m)" = "aarch64" ]]; then + sed -i 's/linux_amd64/linux_arm64/' pkg/model/model.go +fi + echo "Copying istio directory" cp -r ../istio . # export IMAGE_VERSION=$(curl https://raw.githubusercontent.com/istio/test-infra/master/prow/config/jobs/release-builder.yaml | grep "image: gcr.io" | head -n 1 | cut -d: -f3) @@ -79,9 +93,26 @@ if [[ ${TAG} =~ "fips" ]]; then text="if [[ "\${GOARCH}" == "amd64" ]]; then export CGO_ENABLED=1; else export CGO_ENABLED=0; fi" sed -i 's/export CGO_ENABLED=${CGO_ENABLED:-0}/'"$text"'/g' istio/common/scripts/gobuild.sh fi + +# Generalizing TAG variable exporting option to incorporate ARM build.We need amd64 and arm64 suffix in docker images to create multi-arch images.Not needed for tetrate and tetratefips build. +if [[ ${TAG} =~ "multiarch" ]]; then + if [[ "$(uname -m)" = "aarch64" ]]; then + export TAG="${TAG}-arm64" + else + export TAG="${TAG}-amd64" + fi +fi + +#install rpm-build package +sudo apt-get install rpm -y # Build Docker Images -mkdir /tmp/istio-release -go run main.go build --manifest manifest.docker.yaml +sudo rm -rf /tmp/istio-release && mkdir /tmp/istio-release + +if [[ ${TAG} =~ "fips" ]]; then + GOEXPERIMENT=boringcrypto go run main.go build --manifest manifest.docker.yaml +else + go run main.go build --manifest manifest.docker.yaml +fi # go run main.go validate --release /tmp/istio-release/out # seems like it fails if not all the targets are generated #loading pilot image manually since docker container create command is failing due to unavailbilty of pilot image locally @@ -95,15 +126,23 @@ echo "Images are built with: go $BUILD_GO_VERSION" [ $BUILD_GO_VERSION == go$GOLANG_VERSION ] || exit 1 -# fips go versions are like 1.14.12b5, extra checking to not miss anything -if [ ${TAG} =~ "fips" ]; then - [[ $BUILD_GO_VERSION =~ 1.[0-9]+.[0-9]+[a-z][0-9]$ ]] || exit 1 +# Check if binaries are compiled with boringcrypto +if [ ${TAG} =~ "fips" ]; then + CHECK_CRYPTO=$(go version pilot-bin| cut -f3 -d" ") + [[ $CHECK_CRYPTO == X:boringcrypto ]] || exit 1 fi go run main.go publish --release /tmp/istio-release/out --dockerhub $HUB echo "Cleaning up the istio source artificats...." sudo rm -rf /tmp/istio-release/sources/ +if [[ "$(uname -m)" = "x86_64" ]]; then + export TAG="${TAG%-amd64}" + ${BASEDIR}/tetrateci/gen_release_manifest.py ${BASEDIR}/../release-builder/example/manifest.yaml ${BASEDIR}/../release-builder/ +else + exit 0 +fi + # If RELEASE, Build Archives if [[ -z ${TEST:-} ]]; then echo "Building archives..." @@ -121,6 +160,7 @@ if [[ -z ${TEST:-} ]]; then go run main.go build --manifest manifest.archive.yaml python3 -m pip install --upgrade cloudsmith-cli --user + export PATH=$PATH:/home/runner/.local/bin PACKAGES=$(ls /tmp/istio-release/out/ | grep "istio") for package in $PACKAGES; do diff --git a/tetrateci/create_multiarch_images.sh b/tetrateci/create_multiarch_images.sh new file mode 100755 index 000000000000..e6e154c59b63 --- /dev/null +++ b/tetrateci/create_multiarch_images.sh @@ -0,0 +1,36 @@ +#!/usr/bin/env bash + +set -o errexit +set -o pipefail +# set -x + +BASEDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd )" + +IMAGES=(app +app_sidecar_centos_7 +app_sidecar_centos_8 +app_sidecar_debian_10 +app_sidecar_debian_9 +app_sidecar_ubuntu_bionic +app_sidecar_ubuntu_focal +app_sidecar_ubuntu_xenial +install-cni +istioctl +operator +pilot +proxyv2) + +IMAGE_SUFFIXES=("" "-debug" "-distroless") + +for image in "${IMAGES[@]}"; do + for suffix in "${IMAGE_SUFFIXES[@]}"; do + AMD64_IMAGE=$HUB/${image}:${TAG}-amd64${suffix} + ARM64_IMAGE=$HUB/${image}:${TAG}-arm64${suffix} + MULTIARCH_IMAGE=$HUB/${image}:${TAG}${suffix} + + if docker manifest inspect ${AMD64_IMAGE} &> /dev/null && docker manifest inspect ${ARM64_IMAGE} &> /dev/null; then + docker manifest create ${MULTIARCH_IMAGE} --amend ${AMD64_IMAGE} --amend ${ARM64_IMAGE} + docker manifest push ${MULTIARCH_IMAGE} + fi + done +done diff --git a/tetrateci/docs/release.md b/tetrateci/docs/release.md index 7fcb07e7302e..be590518c86b 100644 --- a/tetrateci/docs/release.md +++ b/tetrateci/docs/release.md @@ -1,73 +1,3 @@ # Release Process -## Make_release workflow. - -This workflow will create docker images for various istio components, istioctl binaries for all the OS distros and push them to cloud-smith.This workflow creates two types of builds based on the tag. - -1. tetratefips release - If the tag has fips keyword in the tag (x.xx.x-tetratefips-vx), it will create a tetrate fips build which will be compiled using boringgo with fips build of envoy-proxy. - -2. tetrate release - If the tag does not contain fips keyword, it will be build using native go with upstream envoy proxy. - - -## Create a Release using make_release workflow. - -This workflow needs to be run after e2e workflow i.e after created test docker image and running them through integration test-suite on aws and eks environment. Once the e2e test result is fine, this workflow can create the images and artifacts for istio build and push them to cloud-smith repository. - -1. Create a tag in x.xx.x-tetrate-v0 or x.xx.x-tetratefips-v0 on respective release branch depending on the requirement, the later will generate fips build of istio. - -2. This workflows has 2 Jobs, first one creats a fips compliant proxy depending on the tag eg(x.xx.x-tetratefips-vx), if tag does not contain fips, this job is skipped and the second job, release-builder-run, will create the build. - -3. (Optional) login to cloud-smith and check if the binaries and docker images are available. - docker images are stored in tetrate/getistio-containers repo and artifacts are saved in tetrate/getistio repo. - - ## Publish the build to TID website https://istio.tetratelabs.io/ - - 1. Clone getmesh repo https://github.com/tetratelabs/getmesh.git - - 2. Update site/manifest.json with the new release attributes like release version, eol , flavor etc. - -``` - { - "istio_minor_versions_eol_dates": { - "1.13": "2023-02-11", - "1.12": "2022-11-18", - "1.11": "2022-10-11", - "1.10": "2022-07-17" - }, - "istio_distributions": [ - { - "version": "1.13.2", - "flavor": "tetrate", - "flavor_version": 0, - "k8s_versions": [ - "1.20", - "1.21", - "1.22", - "1.23" - ], - "release_notes": [ - "https://istio.io/latest/news/releases/1.13.x/announcing-1.13.2/" - ], - "is_security_patch": false - }, - { - "version": "1.13.2", - "flavor": "tetratefips", - "flavor_version": 0, - "k8s_versions": [ - "1.20", - "1.21", - "1.22", - "1.23" - ], - "release_notes": [ - "https://istio.io/latest/news/releases/1.13.x/announcing-1.13.2/" - ], - "is_security_patch": false - }, -``` - - 3. Push the changes to remote branch, which will trigger CI which takes care of the docs build and run unit and e2e tests for new release and push to the website. - +TODO(psbrar99): explain the process. diff --git a/tetrateci/fips.md b/tetrateci/fips.md new file mode 100644 index 000000000000..d8558df5e10f --- /dev/null +++ b/tetrateci/fips.md @@ -0,0 +1,60 @@ +## Introduction + +Google's BoringCrypto [module][1] is used for [FIPS-compliant Istio builds][2]. BoringCrypto is a core module of the +BoringSSL library and has been tested by CMVP to be [FIPS validated][3]. Both the Istio control plane and data plane +are built with these modules. The quickest way to get started with FIPS Istio is to use the +[Tetrate Istio Distribution][4]. + +## FIPS Verification + +The easiest way to verify the Go version is with Docker. First, create the containers from the [CloudSmith][5] images. +```shell +HUB=containers.istio.tetratelabs.com +TAG=1.11.4-tetratefips-v0 +PILOT_CONTAINER_ID=$(docker create $HUB/pilot:$TAG) +PROXY_CONTAINER_ID=$(docker create $HUB/proxyv2:$TAG) +OPERATOR_CONTAINER_ID=$(docker create $HUB/operator:$TAG) +ISTIOCTL_CONTAINER_ID=$(docker create $HUB/istioctl:$TAG) +CNI_CONTAINER_ID=$(docker create $HUB/install-cni:$TAG) +``` + +Copy the binaries from the containers to your local machine. +```shell +docker cp $PILOT_CONTAINER_ID:/usr/local/bin/pilot-discovery pilot-discovery +docker cp $PROXY_CONTAINER_ID:/usr/local/bin/pilot-agent pilot-agent +docker cp $PROXY_CONTAINER_ID:/usr/local/bin/envoy envoy +docker cp $OPERATOR_CONTAINER_ID:/usr/local/bin/operator operator +docker cp $ISTIOCTL_CONTAINER_ID:/usr/local/bin/istioctl istioctl +docker cp $CNI_CONTAINER_ID:/usr/local/bin/install-cni install-cni +``` + +Verify the Go version used by the binaries. +```shell +go version pilot-discovery +go version pilot-agent +go version operator +go version istioctl +go version install-cni +``` + +- For Istio minor version <1.15 + The Go version should include `b` to indicate BoringSSL, `go1.16.9b7` +- For Istio minor version >1.15 + The Go version should indicate X:boringcrypto as cryptolibrary, `pilot-discovery: go1.19.1 X:boringcrypto` + + +Verify Envoy is using BoringSSL FIPS: +```shell +envoy --version | cut -f4 -d" " +``` + +The version should include `BoringSSL-FIPS`, for example: +```shell +ed148b62dfb0dc79adc8c8573ced4806883389c0/1.19.2-dev/Modified/RELEASE/BoringSSL-FIPS +``` + +[1]: https://go.googlesource.com/go/+/dev.boringcrypto/README.boringcrypto.md +[2]: https://go-boringcrypto.storage.googleapis.com +[3]: https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/3678 +[4]: https://istio.tetratelabs.io/ +[5]: https://cloudsmith.io/~tetrate/repos/getistio-containers/packages/ diff --git a/tetrateci/gen_iop.py b/tetrateci/gen_iop.py index 7710f886efaf..ab6364c51c54 100755 --- a/tetrateci/gen_iop.py +++ b/tetrateci/gen_iop.py @@ -6,6 +6,9 @@ with open(r'./tests/integration/iop-integration-test-defaults.yaml') as file : iop_config = yaml.load(file, Loader=yaml.FullLoader) - iop_config['spec']['values'].update(extra) + if "values" not in iop_config['spec']: + iop_config['spec'] = dict(values=extra) + else: + iop_config['spec']['values'].update(extra) f = open(r'./tetrateci/iop-gke-integration.yml', 'w') yaml.dump(iop_config, f) diff --git a/tetrateci/gen_release_manifest.py b/tetrateci/gen_release_manifest.py index e169c5379caf..421dfe1393d9 100755 --- a/tetrateci/gen_release_manifest.py +++ b/tetrateci/gen_release_manifest.py @@ -27,9 +27,14 @@ manifest["version"] = tag manifest["dependencies"]["istio"] = {"localpath" : "./istio"} manifest["dependencies"]["client-go"]["branch"] = branch - manifest["dependencies"]["gogo-genproto"]["branch"] = branch + del manifest["dependencies"]["proxy"]["auto"] + manifest["dependencies"]["proxy"]["branch"] = branch manifest["dependencies"]["tools"]["branch"] = branch - manifest["dependencies"]["envoy"]["git"] = "https://github.com/istio/envoy" + # genproto has been removed from 1.14 + # added check for "gogo-genproto" dependenciy if it present then assign branch + if "gogo-genproto" in manifest["dependencies"]: + manifest["dependencies"]["gogo-genproto"]["branch"] = branch + manifest["dependencies"]["envoy"]["git"] = "https://github.com/envoyproxy/envoy" manifest['outputs'] = ["docker"] f = open(os.path.join(destination_folder, "manifest.docker.yaml"), 'w') yaml.dump(manifest, f) @@ -37,5 +42,4 @@ manifest['outputs'] = ["archive"] print(manifest) f = open(os.path.join(destination_folder, "manifest.archive.yaml"), 'w') - yaml.dump(manifest, f) - + yaml.dump(manifest, f) \ No newline at end of file diff --git a/tetrateci/images.sh b/tetrateci/images.sh new file mode 100644 index 000000000000..d61f5ec7b8d4 --- /dev/null +++ b/tetrateci/images.sh @@ -0,0 +1,25 @@ +#!/usr/bin/env bash + +set -o errexit +set -o pipefail +# set -x + +BASEDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd )" + +mkdir containers.istio.tetratelabs.com + +IMAGES=(app +install-cni +istioctl +pilot +proxyv2) + + +IMAGE_SUFFIXES=("debug" "distroless") + +for image in "${IMAGES[@]}"; do + for suffix in "${IMAGE_SUFFIXES[@]}"; do + echo containers.istio.tetratelabs.com/${image}:${TAG}-${suffix} >> list.txt + cat list.txt + done +done diff --git a/tetrateci/manifest.yaml.in b/tetrateci/manifest.yaml.in index 12ed1514d53d..6d944f232bed 100644 --- a/tetrateci/manifest.yaml.in +++ b/tetrateci/manifest.yaml.in @@ -32,7 +32,7 @@ dependencies: git: https://github.com/istio/tools branch: $BRANCH envoy: - git: https://github.com/istio/envoy + git: https://github.com/envoyproxy/envoy auto: proxy_workspace dashboards: diff --git a/tetrateci/patches/common/enable-arm64.1.12.patch b/tetrateci/patches/common/enable-arm64.1.12.patch new file mode 100644 index 000000000000..d9e0e4845e18 --- /dev/null +++ b/tetrateci/patches/common/enable-arm64.1.12.patch @@ -0,0 +1,24 @@ +diff --git a/manifests/charts/gateways/istio-egress/values.yaml b/manifests/charts/gateways/istio-egress/values.yaml +index 5562ba5342..0308cbdd8c 100644 +--- a/manifests/charts/gateways/istio-egress/values.yaml ++++ b/manifests/charts/gateways/istio-egress/values.yaml +@@ -170,6 +170,7 @@ global: + # 3 - Most preferred + arch: + amd64: 2 ++ arm64: 2 + s390x: 2 + ppc64le: 2 + +diff --git a/manifests/charts/gateways/istio-ingress/values.yaml b/manifests/charts/gateways/istio-ingress/values.yaml +index 7be1815185..726aad6f0f 100644 +--- a/manifests/charts/gateways/istio-ingress/values.yaml ++++ b/manifests/charts/gateways/istio-ingress/values.yaml +@@ -183,6 +183,7 @@ global: + # 3 - Most preferred + arch: + amd64: 2 ++ arm64: 2 + s390x: 2 + ppc64le: 2 + diff --git a/tetrateci/patches/eks/eks-ingress.1.10.patch b/tetrateci/patches/eks/eks-ingress.1.10.patch deleted file mode 100644 index 5585df7a68fe..000000000000 --- a/tetrateci/patches/eks/eks-ingress.1.10.patch +++ /dev/null @@ -1,61 +0,0 @@ -diff --git a/pkg/test/framework/components/echo/kube/deployment.go b/pkg/test/framework/components/echo/kube/deployment.go -index 69bba6131c..3a0c1bb679 100644 ---- a/pkg/test/framework/components/echo/kube/deployment.go -+++ b/pkg/test/framework/components/echo/kube/deployment.go -@@ -758,6 +758,7 @@ spec: - // LoadBalancer may not be suppported and the command doesn't have NodePort fallback logic that the tests do - cmd = append(cmd, "--ingressIP", istiodAddr.IP.String()) - } -+ cmd = append(cmd, "--ingressIP", istiodAddr.IP.String()) - // make sure namespace controller has time to create root-cert ConfigMap - if err := retry.UntilSuccess(func() error { - _, _, err = istioCtl.Invoke(cmd) -diff --git a/pkg/test/framework/components/istio/util.go b/pkg/test/framework/components/istio/util.go -index e4e47c79fb..02106614fc 100644 ---- a/pkg/test/framework/components/istio/util.go -+++ b/pkg/test/framework/components/istio/util.go -@@ -97,6 +97,7 @@ func (i *operatorComponent) RemoteDiscoveryAddressFor(cluster cluster.Cluster) ( - } else { - addr = i.CustomIngressFor(primary, eastWestIngressServiceName, eastWestIngressIstioLabel).DiscoveryAddress() - } -+ - if addr.IP.String() == "" { - return net.TCPAddr{}, fmt.Errorf("failed to get ingress IP for %s", primary.Name()) - } -@@ -162,6 +163,17 @@ func getRemoteServiceAddress(s *kube.Settings, cluster cluster.Cluster, ns, labe - if ingr.IP == "" && ingr.Hostname == "" { - return nil, false, fmt.Errorf("service %s/%s is not available yet: no ingress", svc.Namespace, svc.Name) - } -+ -+ if ingr.Hostname != "" { -+ ip, err := net.LookupIP(ingr.Hostname) -+ if err != nil { -+ return nil, false, fmt.Errorf("service %s/%s is not available yet: no ingress", svc.Namespace, svc.Name) -+ } -+ if len(ip) > 0 { -+ ingr.IP = ip[0].String() -+ } -+ } -+ - if ingr.IP != "" { - return net.TCPAddr{IP: net.ParseIP(ingr.IP), Port: port}, true, nil - } -diff --git a/tests/integration/pilot/ingress_test.go b/tests/integration/pilot/ingress_test.go -index bb76db2fb6..643d3f25d1 100644 ---- a/tests/integration/pilot/ingress_test.go -+++ b/tests/integration/pilot/ingress_test.go -@@ -334,6 +334,14 @@ spec: - if hostIsIP { - got = ing.Status.LoadBalancer.Ingress[0].IP - } -+ -+ if ing.Status.LoadBalancer.Ingress[0].Hostname != "" { -+ ip, _ := net.LookupIP(ing.Status.LoadBalancer.Ingress[0].Hostname) -+ if len(ip) > 0 { -+ got = ip[0].String() -+ } -+ } -+ - if got != host { - return fmt.Errorf("unexpected ingress status, got %+v want %v", got, host) - } diff --git a/tetrateci/patches/eks/eks-ingress.1.11.patch b/tetrateci/patches/eks/eks-ingress.1.11.patch deleted file mode 100644 index 528c2e163c3a..000000000000 --- a/tetrateci/patches/eks/eks-ingress.1.11.patch +++ /dev/null @@ -1,61 +0,0 @@ -diff --git a/pkg/test/framework/components/echo/kube/deployment.go b/pkg/test/framework/components/echo/kube/deployment.go -index d369f095cd..1ac7e6ae66 100644 ---- a/pkg/test/framework/components/echo/kube/deployment.go -+++ b/pkg/test/framework/components/echo/kube/deployment.go -@@ -793,6 +793,7 @@ spec: - // LoadBalancer may not be suppported and the command doesn't have NodePort fallback logic that the tests do - cmd = append(cmd, "--ingressIP", istiodAddr.IP.String()) - } -+ cmd = append(cmd, "--ingressIP", istiodAddr.IP.String()) - if nsLabels, err := cfg.Namespace.Labels(); err != nil { - log.Warnf("failed fetching labels for %s; assuming no-revision (can cause failures): %v", cfg.Namespace.Name(), err) - } else if rev := nsLabels[label.IoIstioRev.Name]; rev != "" { -diff --git a/pkg/test/framework/components/istio/util.go b/pkg/test/framework/components/istio/util.go -index bee2ce22d7..fa36e65035 100644 ---- a/pkg/test/framework/components/istio/util.go -+++ b/pkg/test/framework/components/istio/util.go -@@ -97,6 +97,7 @@ func (i *operatorComponent) RemoteDiscoveryAddressFor(cluster cluster.Cluster) ( - } else { - addr = i.CustomIngressFor(primary, eastWestIngressServiceName, eastWestIngressIstioLabel).DiscoveryAddress() - } -+ - if addr.IP.String() == "" { - return net.TCPAddr{}, fmt.Errorf("failed to get ingress IP for %s", primary.Name()) - } -@@ -162,6 +163,17 @@ func getRemoteServiceAddress(s *kube.Settings, cluster cluster.Cluster, ns, labe - if ingr.IP == "" && ingr.Hostname == "" { - return nil, false, fmt.Errorf("service %s/%s is not available yet: no ingress", svc.Namespace, svc.Name) - } -+ -+ if ingr.Hostname != "" { -+ ip, err := net.LookupIP(ingr.Hostname) -+ if err != nil { -+ return nil, false, fmt.Errorf("service %s/%s is not available yet: no ingress", svc.Namespace, svc.Name) -+ } -+ if len(ip) > 0 { -+ ingr.IP = ip[0].String() -+ } -+ } -+ - if ingr.IP != "" { - return net.TCPAddr{IP: net.ParseIP(ingr.IP), Port: port}, true, nil - } -diff --git a/tests/integration/pilot/ingress_test.go b/tests/integration/pilot/ingress_test.go -index 2453b16fd3..2cbe717afc 100644 ---- a/tests/integration/pilot/ingress_test.go -+++ b/tests/integration/pilot/ingress_test.go -@@ -529,6 +529,14 @@ spec: - if hostIsIP { - got = ing.Status.LoadBalancer.Ingress[0].IP - } -+ -+ if ing.Status.LoadBalancer.Ingress[0].Hostname != "" { -+ ip, _ := net.LookupIP(ing.Status.LoadBalancer.Ingress[0].Hostname) -+ if len(ip) > 0 { -+ got = ip[0].String() -+ } -+ } -+ - if got != host { - return fmt.Errorf("unexpected ingress status, got %+v want %v", got, host) - } diff --git a/tetrateci/patches/eks/eks-ingress.1.13.patch b/tetrateci/patches/eks/eks-ingress.1.13.patch deleted file mode 100644 index 981a8ab9d183..000000000000 --- a/tetrateci/patches/eks/eks-ingress.1.13.patch +++ /dev/null @@ -1,61 +0,0 @@ -diff --git a/pkg/test/framework/components/echo/kube/deployment.go b/pkg/test/framework/components/echo/kube/deployment.go -index 374551f30b..f151c4f608 100644 ---- a/pkg/test/framework/components/echo/kube/deployment.go -+++ b/pkg/test/framework/components/echo/kube/deployment.go -@@ -814,6 +814,7 @@ spec: - // LoadBalancer may not be supported and the command doesn't have NodePort fallback logic that the tests do - cmd = append(cmd, "--ingressIP", istiodAddr.IP.String()) - } -+ cmd = append(cmd, "--ingressIP", istiodAddr.IP.String()) - if nsLabels, err := cfg.Namespace.Labels(); err != nil { - log.Warnf("failed fetching labels for %s; assuming no-revision (can cause failures): %v", cfg.Namespace.Name(), err) - } else if rev := nsLabels[label.IoIstioRev.Name]; rev != "" { -diff --git a/pkg/test/framework/components/istio/util.go b/pkg/test/framework/components/istio/util.go -index b411e61230..ee5a0d42e4 100644 ---- a/pkg/test/framework/components/istio/util.go -+++ b/pkg/test/framework/components/istio/util.go -@@ -97,6 +97,7 @@ func (i *operatorComponent) RemoteDiscoveryAddressFor(cluster cluster.Cluster) ( - } else { - addr = i.CustomIngressFor(primary, eastWestIngressServiceName, eastWestIngressIstioLabel).DiscoveryAddress() - } -+ - if addr.IP.String() == "" { - return net.TCPAddr{}, fmt.Errorf("failed to get ingress IP for %s", primary.Name()) - } -@@ -162,6 +163,17 @@ func getRemoteServiceAddress(s *kube.Settings, cluster cluster.Cluster, ns, labe - if ingr.IP == "" && ingr.Hostname == "" { - return nil, false, fmt.Errorf("service %s/%s is not available yet: no ingress", svc.Namespace, svc.Name) - } -+ -+ if ingr.Hostname != "" { -+ ip, err := net.LookupIP(ingr.Hostname) -+ if err != nil { -+ return nil, false, fmt.Errorf("service %s/%s is not available yet: no ingress", svc.Namespace, svc.Name) -+ } -+ if len(ip) > 0 { -+ ingr.IP = ip[0].String() -+ } -+ } -+ - if ingr.IP != "" { - return net.TCPAddr{IP: net.ParseIP(ingr.IP), Port: port}, true, nil - } -diff --git a/tests/integration/pilot/ingress_test.go b/tests/integration/pilot/ingress_test.go -index f4638b9a3c..78e5ba3701 100644 ---- a/tests/integration/pilot/ingress_test.go -+++ b/tests/integration/pilot/ingress_test.go -@@ -601,6 +601,14 @@ spec: - if hostIsIP { - got = ing.Status.LoadBalancer.Ingress[0].IP - } -+ -+ if ing.Status.LoadBalancer.Ingress[0].Hostname != "" { -+ ip, _ := net.LookupIP(ing.Status.LoadBalancer.Ingress[0].Hostname) -+ if len(ip) > 0 { -+ got = ip[0].String() -+ } -+ } -+ - if got != host { - return fmt.Errorf("unexpected ingress status, got %+v want %v", got, host) - } diff --git a/tetrateci/patches/eks/eks-ingress.1.7.patch b/tetrateci/patches/eks/eks-ingress.1.7.patch deleted file mode 100644 index f1269cc0fced..000000000000 --- a/tetrateci/patches/eks/eks-ingress.1.7.patch +++ /dev/null @@ -1,58 +0,0 @@ -diff --git a/pkg/test/framework/components/echo/kube/deployment.go b/pkg/test/framework/components/echo/kube/deployment.go -index eeb0f9bccf..e05af78053 100644 ---- a/pkg/test/framework/components/echo/kube/deployment.go -+++ b/pkg/test/framework/components/echo/kube/deployment.go -@@ -19,6 +19,7 @@ import ( - "net" - "strconv" - "text/template" -+ "time" - - "github.com/Masterminds/sprig" - -@@ -366,7 +367,7 @@ func generateYAMLWithSettings(cfg echo.Config, settings *image.Settings, - var err error - addr, err = istio.GetRemoteDiscoveryAddress("istio-system", cluster, s.Minikube) - return err -- }) -+ }, retry.Timeout(3*time.Minute), retry.Delay(5*time.Second)) - if err != nil { - return "", "", err - } -diff --git a/pkg/test/framework/components/ingress/kube.go b/pkg/test/framework/components/ingress/kube.go -index 0531e26cf9..e3360f1981 100644 ---- a/pkg/test/framework/components/ingress/kube.go -+++ b/pkg/test/framework/components/ingress/kube.go -@@ -115,6 +115,14 @@ func (c *kubeComponent) getAddressInner(ns string, port int) (interface{}, bool, - } - - if len(svc.Status.LoadBalancer.Ingress) == 0 || svc.Status.LoadBalancer.Ingress[0].IP == "" { -+ if svc.Status.LoadBalancer.Ingress[0].IP == "" && svc.Status.LoadBalancer.Ingress[0].Hostname != "" { -+ ip, err := net.LookupIP(svc.Status.LoadBalancer.Ingress[0].Hostname) -+ if err != nil { -+ return nil, false, err -+ } -+ return net.TCPAddr{IP: ip[0], Port: port}, true, nil -+ } -+ - return nil, false, fmt.Errorf("service ingress is not available yet: %s/%s", svc.Namespace, svc.Name) - } - -diff --git a/pkg/test/framework/components/istio/util.go b/pkg/test/framework/components/istio/util.go -index 6ff9f57d0b..cf72e0a238 100644 ---- a/pkg/test/framework/components/istio/util.go -+++ b/pkg/test/framework/components/istio/util.go -@@ -108,6 +108,13 @@ func GetRemoteDiscoveryAddress(namespace string, cluster resource.Cluster, useNo - - // If running in KinD, MetalLB must be installed to enable LoadBalancer resources - if len(svc.Status.LoadBalancer.Ingress) == 0 || svc.Status.LoadBalancer.Ingress[0].IP == "" { -+ if svc.Status.LoadBalancer.Ingress[0].IP == "" && svc.Status.LoadBalancer.Ingress[0].Hostname != "" { -+ ip, err := net.LookupIP(svc.Status.LoadBalancer.Ingress[0].Hostname) -+ if err != nil { -+ return net.TCPAddr{}, err -+ } -+ return net.TCPAddr{IP: ip[0], Port: discoveryPort}, nil -+ } - return net.TCPAddr{}, fmt.Errorf("service ingress is not available yet: %s/%s", svc.Namespace, svc.Name) - } - diff --git a/tetrateci/patches/eks/eks-ingress.1.8.patch b/tetrateci/patches/eks/eks-ingress.1.8.patch deleted file mode 100644 index 7200eda01ad3..000000000000 --- a/tetrateci/patches/eks/eks-ingress.1.8.patch +++ /dev/null @@ -1,62 +0,0 @@ -diff --git a/pkg/test/framework/components/echo/kube/instance.go b/pkg/test/framework/components/echo/kube/instance.go -index 4ec208e775..9975b62ba2 100644 ---- a/pkg/test/framework/components/echo/kube/instance.go -+++ b/pkg/test/framework/components/echo/kube/instance.go -@@ -238,10 +238,14 @@ spec: - if cfg.AutoRegisterVM { - cmd = append(cmd, "--autoregister") - } -+ - if !ctx.Environment().(*kube.Environment).Settings().LoadBalancerSupported { - // LoadBalancer may not be suppported and the command doesn't have NodePort fallback logic that the tests do - cmd = append(cmd, "--ingressIP", istiodAddr.IP.String()) - } -+ -+ cmd = append(cmd, "--ingressIP", istiodAddr.IP.String()) -+ - // make sure namespace controller has time to create root-cert ConfigMap - if err := retry.UntilSuccess(func() error { - _, _, err = istioCtl.Invoke(cmd) -diff --git a/pkg/test/framework/components/istio/util.go b/pkg/test/framework/components/istio/util.go -index 0852219fee..120508da1b 100644 ---- a/pkg/test/framework/components/istio/util.go -+++ b/pkg/test/framework/components/istio/util.go -@@ -153,6 +153,13 @@ func getRemoteServiceAddress(s *kube.Settings, cluster cluster.Cluster, ns, labe - } - - if len(svc.Status.LoadBalancer.Ingress) == 0 || svc.Status.LoadBalancer.Ingress[0].IP == "" { -+ if svc.Status.LoadBalancer.Ingress[0].IP == "" && svc.Status.LoadBalancer.Ingress[0].Hostname != "" { -+ ip, err := net.LookupIP(svc.Status.LoadBalancer.Ingress[0].Hostname) -+ if err != nil { -+ return nil, false, err -+ } -+ return net.TCPAddr{IP: ip[0], Port: port}, true, nil -+ } - return nil, false, fmt.Errorf("service %s is not available yet: %s/%s", svcName, svc.Namespace, svc.Name) - } - -diff --git a/tests/integration/pilot/ingress_test.go b/tests/integration/pilot/ingress_test.go -index ed81a5eb29..376ed8b20f 100644 ---- a/tests/integration/pilot/ingress_test.go -+++ b/tests/integration/pilot/ingress_test.go -@@ -18,6 +18,7 @@ package pilot - import ( - "context" - "fmt" -+ "net" - "testing" - "time" - -@@ -269,6 +270,12 @@ spec: - if err != nil { - return err - } -+ -+ if len(ing.Status.LoadBalancer.Ingress) == 1 { -+ ip, _ := net.LookupIP(ing.Status.LoadBalancer.Ingress[0].Hostname) -+ ing.Status.LoadBalancer.Ingress[0].IP = ip[0].String() -+ } -+ - if len(ing.Status.LoadBalancer.Ingress) != 1 || ing.Status.LoadBalancer.Ingress[0].IP != ip { - return fmt.Errorf("unexpected ingress status, got %+v want %v", ing.Status.LoadBalancer, ip) - } diff --git a/tetrateci/patches/eks/eks-ingress.1.9.patch b/tetrateci/patches/eks/eks-ingress.1.9.patch deleted file mode 100644 index 7a3cd767da32..000000000000 --- a/tetrateci/patches/eks/eks-ingress.1.9.patch +++ /dev/null @@ -1,58 +0,0 @@ -diff --git a/pkg/test/framework/components/echo/kube/deployment.go b/pkg/test/framework/components/echo/kube/deployment.go -index 9a0f7282cc..e9203dee2b 100644 ---- a/pkg/test/framework/components/echo/kube/deployment.go -+++ b/pkg/test/framework/components/echo/kube/deployment.go -@@ -735,6 +735,9 @@ spec: - // LoadBalancer may not be suppported and the command doesn't have NodePort fallback logic that the tests do - cmd = append(cmd, "--ingressIP", istiodAddr.IP.String()) - } -+ -+ cmd = append(cmd, "--ingressIP", istiodAddr.IP.String()) -+ - // make sure namespace controller has time to create root-cert ConfigMap - if err := retry.UntilSuccess(func() error { - _, _, err = istioCtl.Invoke(cmd) -diff --git a/pkg/test/framework/components/istio/util.go b/pkg/test/framework/components/istio/util.go -index d9de316906..abe6165113 100644 ---- a/pkg/test/framework/components/istio/util.go -+++ b/pkg/test/framework/components/istio/util.go -@@ -153,6 +153,14 @@ func getRemoteServiceAddress(s *kube.Settings, cluster cluster.Cluster, ns, labe - } - - if len(svc.Status.LoadBalancer.Ingress) == 0 || svc.Status.LoadBalancer.Ingress[0].IP == "" { -+ if svc.Status.LoadBalancer.Ingress[0].IP == "" && svc.Status.LoadBalancer.Ingress[0].Hostname != "" { -+ ip, err := net.LookupIP(svc.Status.LoadBalancer.Ingress[0].Hostname) -+ if err != nil { -+ return nil, false, err -+ } -+ return net.TCPAddr{IP: ip[0], Port: port}, true, nil -+ } -+ - return nil, false, fmt.Errorf("service %s is not available yet: %s/%s", svcName, svc.Namespace, svc.Name) - } - -diff --git a/tests/integration/pilot/ingress_test.go b/tests/integration/pilot/ingress_test.go -index 3e0ab9f184..874503fd6f 100644 ---- a/tests/integration/pilot/ingress_test.go -+++ b/tests/integration/pilot/ingress_test.go -@@ -19,6 +19,7 @@ import ( - "context" - "fmt" - "io/ioutil" -+ "net" - "path/filepath" - "testing" - "time" -@@ -278,6 +279,12 @@ spec: - if err != nil { - return err - } -+ -+ if len(ing.Status.LoadBalancer.Ingress) == 1 { -+ ip, _ := net.LookupIP(ing.Status.LoadBalancer.Ingress[0].Hostname) -+ ing.Status.LoadBalancer.Ingress[0].IP = ip[0].String() -+ } -+ - if len(ing.Status.LoadBalancer.Ingress) != 1 || ing.Status.LoadBalancer.Ingress[0].IP != ip { - return fmt.Errorf("unexpected ingress status, got %+v want %v", ing.Status.LoadBalancer, ip) - } diff --git a/tetrateci/patches/eks/eks_1.15.patch b/tetrateci/patches/eks/eks_1.15.patch new file mode 100644 index 000000000000..72cffcba51ed --- /dev/null +++ b/tetrateci/patches/eks/eks_1.15.patch @@ -0,0 +1,125 @@ +From a2e8b635a71ac66273edd6388ba883a0214b9cf8 Mon Sep 17 00:00:00 2001 +From: psbrar99 +Date: Thu, 3 Aug 2023 16:25:50 -0600 +Subject: [PATCH] 1.15 patch + +Signed-off-by: psbrar99 +--- + .../components/echo/kube/deployment.go | 1 + + .../echo/kube/templates/vm_deployment.yaml | 22 +++++++++---------- + pkg/test/framework/components/istio/util.go | 9 ++++++++ + tests/integration/pilot/ingress_test.go | 14 +++++++++++- + 4 files changed, 34 insertions(+), 12 deletions(-) + +diff --git a/pkg/test/framework/components/echo/kube/deployment.go b/pkg/test/framework/components/echo/kube/deployment.go +index dbad3eb0be..9a53ff95dd 100644 +--- a/pkg/test/framework/components/echo/kube/deployment.go ++++ b/pkg/test/framework/components/echo/kube/deployment.go +@@ -515,6 +515,7 @@ spec: + // LoadBalancer may not be supported and the command doesn't have NodePort fallback logic that the tests do + cmd = append(cmd, "--ingressIP", istiodAddr.IP.String()) + } ++ cmd = append(cmd, "--ingressIP", istiodAddr.IP.String()) + if rev := getIstioRevision(cfg.Namespace); len(rev) > 0 { + cmd = append(cmd, "--revision", rev) + } +diff --git a/pkg/test/framework/components/echo/kube/templates/vm_deployment.yaml b/pkg/test/framework/components/echo/kube/templates/vm_deployment.yaml +index 4343abcd5c..1c61d2c1a5 100644 +--- a/pkg/test/framework/components/echo/kube/templates/vm_deployment.yaml ++++ b/pkg/test/framework/components/echo/kube/templates/vm_deployment.yaml +@@ -85,16 +85,16 @@ spec: + + # since we're not overwriting /etc/hosts on k8s, verify that istiod hostname in /etc/hosts + # matches the value generated by istioctl +- echo "checking istio host" +- SYSTEM_HOST=$(cat /etc/hosts | grep istiod) +- ISTIOCTL_HOST=$(cat /var/run/secrets/istio/bootstrap/hosts | grep istiod) +- if [ "$(echo "$SYSTEM_HOST" | tr -d '[:space:]')" != "$(echo "$ISTIOCTL_HOST" | tr -d '[:space:]')" ]; then +- echo "istiod host in /etc/hosts does not match value generated by istioctl" +- echo "/etc/hosts: $SYSTEM_HOST" +- echo "/var/run/secrets/istio/bootstrap/hosts: $ISTIOCTL_HOST" +- exit 1 +- fi +- echo "istiod host ok" ++ # echo "checking istio host" ++ # SYSTEM_HOST=$(cat /etc/hosts | grep istiod) ++ # ISTIOCTL_HOST=$(cat /var/run/secrets/istio/bootstrap/hosts | grep istiod) ++ # if [ "$(echo "$SYSTEM_HOST" | tr -d '[:space:]')" != "$(echo "$ISTIOCTL_HOST" | tr -d '[:space:]')" ]; then ++ # echo "istiod host in /etc/hosts does not match value generated by istioctl" ++ # echo "/etc/hosts: $SYSTEM_HOST" ++ # echo "/var/run/secrets/istio/bootstrap/hosts: $ISTIOCTL_HOST" ++ # exit 1 ++ # fi ++ # echo "istiod host ok" + + # read certs from correct directory + sudo sh -c 'echo PROV_CERT=/var/run/secrets/istio >> /var/lib/istio/envoy/cluster.env' +@@ -173,4 +173,4 @@ spec: + name: {{ $value.Value }} + {{- end }} + {{- end }} +-{{- end}} ++{{- end}} +\ No newline at end of file +diff --git a/pkg/test/framework/components/istio/util.go b/pkg/test/framework/components/istio/util.go +index 18c5502e1d..60198b4f7d 100644 +--- a/pkg/test/framework/components/istio/util.go ++++ b/pkg/test/framework/components/istio/util.go +@@ -132,6 +132,15 @@ func getRemoteServiceAddress(s *kube.Settings, cluster cluster.Cluster, ns, labe + if ingr.IP == "" && ingr.Hostname == "" { + return nil, false, fmt.Errorf("service %s/%s is not available yet: no ingress", svc.Namespace, svc.Name) + } ++ if ingr.Hostname != "" { ++ ip, err := net.LookupIP(ingr.Hostname) ++ if err != nil { ++ return nil, false, fmt.Errorf("service %s/%s is not available yet: no ingress", svc.Namespace, svc.Name) ++ } ++ if len(ip) > 0 { ++ ingr.IP = ip[0].String() ++ } ++ } + if ingr.IP != "" { + return net.TCPAddr{IP: net.ParseIP(ingr.IP), Port: port}, true, nil + } +diff --git a/tests/integration/pilot/ingress_test.go b/tests/integration/pilot/ingress_test.go +index 217da4f89c..9bd5a9287f 100644 +--- a/tests/integration/pilot/ingress_test.go ++++ b/tests/integration/pilot/ingress_test.go +@@ -228,7 +228,7 @@ spec: + }) + t.NewSubTest("status").Run(func(t framework.TestContext) { + retry.UntilSuccessOrFail(t, func() error { +- gwc, err := t.Clusters().Kube().Default().GatewayAPI().GatewayV1beta1().GatewayClasses().Get(context.Background(), "istio", metav1.GetOptions{}) ++ gwc, err := t.Clusters().Kube().Default().GatewayAPI().GatewayV1alpha2().GatewayClasses().Get(context.Background(), "istio", metav1.GetOptions{}) + if err != nil { + return err + } +@@ -601,6 +601,12 @@ spec: + if hostIsIP { + got = ing.Status.LoadBalancer.Ingress[0].IP + } ++ if ing.Status.LoadBalancer.Ingress[0].Hostname != "" { ++ ip, _ := net.LookupIP(ing.Status.LoadBalancer.Ingress[0].Hostname) ++ if len(ip) > 0 { ++ got = ip[0].String() ++ } ++ } + if got != host { + return fmt.Errorf("unexpected ingress status, got %+v want %v", got, host) + } +@@ -617,6 +623,12 @@ spec: + if hostIsIP { + got = ing.Status.LoadBalancer.Ingress[0].IP + } ++ if ing.Status.LoadBalancer.Ingress[0].Hostname != "" { ++ ip, _ := net.LookupIP(ing.Status.LoadBalancer.Ingress[0].Hostname) ++ if len(ip) > 0 { ++ got = ip[0].String() ++ } ++ } + if got != host { + return fmt.Errorf("unexpected ingress status, got %+v want %v", got, host) + } +-- +2.30.1 (Apple Git-130) + diff --git a/tetrateci/patches/eks/eks_1.16.patch b/tetrateci/patches/eks/eks_1.16.patch new file mode 100644 index 000000000000..88ca540a7a44 --- /dev/null +++ b/tetrateci/patches/eks/eks_1.16.patch @@ -0,0 +1,124 @@ +From db64738688316472d1840c5f0e58fcdf6b7c51d9 Mon Sep 17 00:00:00 2001 +From: psbrar99 +Date: Thu, 3 Aug 2023 13:52:36 -0600 +Subject: [PATCH] 1.16 patch + +Signed-off-by: psbrar99 +--- + .../components/echo/kube/deployment.go | 1 + + .../echo/kube/templates/vm_deployment.yaml | 22 +++++++++---------- + pkg/test/framework/components/istio/util.go | 9 ++++++++ + tests/integration/pilot/ingress_test.go | 13 +++++++++++ + 4 files changed, 34 insertions(+), 11 deletions(-) + +diff --git a/pkg/test/framework/components/echo/kube/deployment.go b/pkg/test/framework/components/echo/kube/deployment.go +index 3eb41c570b..bd9b567b83 100644 +--- a/pkg/test/framework/components/echo/kube/deployment.go ++++ b/pkg/test/framework/components/echo/kube/deployment.go +@@ -521,6 +521,7 @@ spec: + if rev := getIstioRevision(cfg.Namespace); len(rev) > 0 { + cmd = append(cmd, "--revision", rev) + } ++ cmd = append(cmd, "--ingressIP", istiodAddr.Addr().String()) + // make sure namespace controller has time to create root-cert ConfigMap + if err := retry.UntilSuccess(func() error { + stdout, stderr, err := istioCtl.Invoke(cmd) +diff --git a/pkg/test/framework/components/echo/kube/templates/vm_deployment.yaml b/pkg/test/framework/components/echo/kube/templates/vm_deployment.yaml +index 4343abcd5c..1c61d2c1a5 100644 +--- a/pkg/test/framework/components/echo/kube/templates/vm_deployment.yaml ++++ b/pkg/test/framework/components/echo/kube/templates/vm_deployment.yaml +@@ -85,16 +85,16 @@ spec: + + # since we're not overwriting /etc/hosts on k8s, verify that istiod hostname in /etc/hosts + # matches the value generated by istioctl +- echo "checking istio host" +- SYSTEM_HOST=$(cat /etc/hosts | grep istiod) +- ISTIOCTL_HOST=$(cat /var/run/secrets/istio/bootstrap/hosts | grep istiod) +- if [ "$(echo "$SYSTEM_HOST" | tr -d '[:space:]')" != "$(echo "$ISTIOCTL_HOST" | tr -d '[:space:]')" ]; then +- echo "istiod host in /etc/hosts does not match value generated by istioctl" +- echo "/etc/hosts: $SYSTEM_HOST" +- echo "/var/run/secrets/istio/bootstrap/hosts: $ISTIOCTL_HOST" +- exit 1 +- fi +- echo "istiod host ok" ++ # echo "checking istio host" ++ # SYSTEM_HOST=$(cat /etc/hosts | grep istiod) ++ # ISTIOCTL_HOST=$(cat /var/run/secrets/istio/bootstrap/hosts | grep istiod) ++ # if [ "$(echo "$SYSTEM_HOST" | tr -d '[:space:]')" != "$(echo "$ISTIOCTL_HOST" | tr -d '[:space:]')" ]; then ++ # echo "istiod host in /etc/hosts does not match value generated by istioctl" ++ # echo "/etc/hosts: $SYSTEM_HOST" ++ # echo "/var/run/secrets/istio/bootstrap/hosts: $ISTIOCTL_HOST" ++ # exit 1 ++ # fi ++ # echo "istiod host ok" + + # read certs from correct directory + sudo sh -c 'echo PROV_CERT=/var/run/secrets/istio >> /var/lib/istio/envoy/cluster.env' +@@ -173,4 +173,4 @@ spec: + name: {{ $value.Value }} + {{- end }} + {{- end }} +-{{- end}} ++{{- end}} +\ No newline at end of file +diff --git a/pkg/test/framework/components/istio/util.go b/pkg/test/framework/components/istio/util.go +index 2600fe421c..2d01c756c1 100644 +--- a/pkg/test/framework/components/istio/util.go ++++ b/pkg/test/framework/components/istio/util.go +@@ -137,6 +137,15 @@ func getRemoteServiceAddress(s *kube.Settings, cluster cluster.Cluster, ns, labe + if ingr.IP == "" && ingr.Hostname == "" { + return nil, false, fmt.Errorf("service %s/%s is not available yet: no ingress", svc.Namespace, svc.Name) + } ++ if ingr.Hostname != "" { ++ ip, err := net.LookupIP(ingr.Hostname) ++ if err != nil { ++ return nil, false, fmt.Errorf("service %s/%s is not available yet: no ingress", svc.Namespace, svc.Name) ++ } ++ if len(ip) > 0 { ++ ingr.IP = ip[0].String() ++ } ++ } + if ingr.IP != "" { + ipaddr, err := netip.ParseAddr(ingr.IP) + if err != nil { +diff --git a/tests/integration/pilot/ingress_test.go b/tests/integration/pilot/ingress_test.go +index 258dcf8de0..d1cd0601be 100644 +--- a/tests/integration/pilot/ingress_test.go ++++ b/tests/integration/pilot/ingress_test.go +@@ -232,6 +232,7 @@ spec: + if err != nil { + return err + } ++ + if s := kstatus.GetCondition(gwc.Status.Conditions, string(k8s.GatewayClassConditionStatusAccepted)).Status; s != metav1.ConditionTrue { + return fmt.Errorf("expected status %q, got %q", metav1.ConditionTrue, s) + } +@@ -601,6 +602,12 @@ spec: + if hostIsIP { + got = ing.Status.LoadBalancer.Ingress[0].IP + } ++ if ing.Status.LoadBalancer.Ingress[0].Hostname != "" { ++ ip, _ := net.LookupIP(ing.Status.LoadBalancer.Ingress[0].Hostname) ++ if len(ip) > 0 { ++ got = ip[0].String() ++ } ++ } + if got != host { + return fmt.Errorf("unexpected ingress status, got %+v want %v", got, host) + } +@@ -617,6 +624,12 @@ spec: + if hostIsIP { + got = ing.Status.LoadBalancer.Ingress[0].IP + } ++ if ing.Status.LoadBalancer.Ingress[0].Hostname != "" { ++ ip, _ := net.LookupIP(ing.Status.LoadBalancer.Ingress[0].Hostname) ++ if len(ip) > 0 { ++ got = ip[0].String() ++ } ++ } + if got != host { + return fmt.Errorf("unexpected ingress status, got %+v want %v", got, host) + } +-- +2.30.1 (Apple Git-130) + diff --git a/tetrateci/patches/eks/eks_1.17.patch b/tetrateci/patches/eks/eks_1.17.patch new file mode 100644 index 000000000000..02aa5e33e918 --- /dev/null +++ b/tetrateci/patches/eks/eks_1.17.patch @@ -0,0 +1,124 @@ +From 5de4783fd311442f004213871142e75ec320c135 Mon Sep 17 00:00:00 2001 +From: psbrar99 +Date: Thu, 3 Aug 2023 15:51:42 -0600 +Subject: [PATCH] 1.17 patch + +Signed-off-by: psbrar99 +--- + .../components/echo/kube/deployment.go | 1 + + .../echo/kube/templates/vm_deployment.yaml | 22 +++++++++---------- + pkg/test/framework/components/istio/util.go | 9 ++++++++ + tests/integration/pilot/ingress_test.go | 13 +++++++++++ + 4 files changed, 34 insertions(+), 11 deletions(-) + +diff --git a/pkg/test/framework/components/echo/kube/deployment.go b/pkg/test/framework/components/echo/kube/deployment.go +index 7574ed58c4..6f418d7f17 100644 +--- a/pkg/test/framework/components/echo/kube/deployment.go ++++ b/pkg/test/framework/components/echo/kube/deployment.go +@@ -521,6 +521,7 @@ spec: + if rev := getIstioRevision(cfg.Namespace); len(rev) > 0 { + cmd = append(cmd, "--revision", rev) + } ++ cmd = append(cmd, "--ingressIP", istiodAddr.Addr().String()) + // make sure namespace controller has time to create root-cert ConfigMap + if err := retry.UntilSuccess(func() error { + stdout, stderr, err := istioCtl.Invoke(cmd) +diff --git a/pkg/test/framework/components/echo/kube/templates/vm_deployment.yaml b/pkg/test/framework/components/echo/kube/templates/vm_deployment.yaml +index 4343abcd5c..1c61d2c1a5 100644 +--- a/pkg/test/framework/components/echo/kube/templates/vm_deployment.yaml ++++ b/pkg/test/framework/components/echo/kube/templates/vm_deployment.yaml +@@ -85,16 +85,16 @@ spec: + + # since we're not overwriting /etc/hosts on k8s, verify that istiod hostname in /etc/hosts + # matches the value generated by istioctl +- echo "checking istio host" +- SYSTEM_HOST=$(cat /etc/hosts | grep istiod) +- ISTIOCTL_HOST=$(cat /var/run/secrets/istio/bootstrap/hosts | grep istiod) +- if [ "$(echo "$SYSTEM_HOST" | tr -d '[:space:]')" != "$(echo "$ISTIOCTL_HOST" | tr -d '[:space:]')" ]; then +- echo "istiod host in /etc/hosts does not match value generated by istioctl" +- echo "/etc/hosts: $SYSTEM_HOST" +- echo "/var/run/secrets/istio/bootstrap/hosts: $ISTIOCTL_HOST" +- exit 1 +- fi +- echo "istiod host ok" ++ # echo "checking istio host" ++ # SYSTEM_HOST=$(cat /etc/hosts | grep istiod) ++ # ISTIOCTL_HOST=$(cat /var/run/secrets/istio/bootstrap/hosts | grep istiod) ++ # if [ "$(echo "$SYSTEM_HOST" | tr -d '[:space:]')" != "$(echo "$ISTIOCTL_HOST" | tr -d '[:space:]')" ]; then ++ # echo "istiod host in /etc/hosts does not match value generated by istioctl" ++ # echo "/etc/hosts: $SYSTEM_HOST" ++ # echo "/var/run/secrets/istio/bootstrap/hosts: $ISTIOCTL_HOST" ++ # exit 1 ++ # fi ++ # echo "istiod host ok" + + # read certs from correct directory + sudo sh -c 'echo PROV_CERT=/var/run/secrets/istio >> /var/lib/istio/envoy/cluster.env' +@@ -173,4 +173,4 @@ spec: + name: {{ $value.Value }} + {{- end }} + {{- end }} +-{{- end}} ++{{- end}} +\ No newline at end of file +diff --git a/pkg/test/framework/components/istio/util.go b/pkg/test/framework/components/istio/util.go +index 2600fe421c..2d01c756c1 100644 +--- a/pkg/test/framework/components/istio/util.go ++++ b/pkg/test/framework/components/istio/util.go +@@ -137,6 +137,15 @@ func getRemoteServiceAddress(s *kube.Settings, cluster cluster.Cluster, ns, labe + if ingr.IP == "" && ingr.Hostname == "" { + return nil, false, fmt.Errorf("service %s/%s is not available yet: no ingress", svc.Namespace, svc.Name) + } ++ if ingr.Hostname != "" { ++ ip, err := net.LookupIP(ingr.Hostname) ++ if err != nil { ++ return nil, false, fmt.Errorf("service %s/%s is not available yet: no ingress", svc.Namespace, svc.Name) ++ } ++ if len(ip) > 0 { ++ ingr.IP = ip[0].String() ++ } ++ } + if ingr.IP != "" { + ipaddr, err := netip.ParseAddr(ingr.IP) + if err != nil { +diff --git a/tests/integration/pilot/ingress_test.go b/tests/integration/pilot/ingress_test.go +index 82766cfc3c..6e1ef41f9f 100644 +--- a/tests/integration/pilot/ingress_test.go ++++ b/tests/integration/pilot/ingress_test.go +@@ -232,6 +232,7 @@ spec: + if err != nil { + return err + } ++ + if s := kstatus.GetCondition(gwc.Status.Conditions, string(k8s.GatewayClassConditionStatusAccepted)).Status; s != metav1.ConditionTrue { + return fmt.Errorf("expected status %q, got %q", metav1.ConditionTrue, s) + } +@@ -601,6 +602,12 @@ spec: + if hostIsIP { + got = ing.Status.LoadBalancer.Ingress[0].IP + } ++ if ing.Status.LoadBalancer.Ingress[0].Hostname != "" { ++ ip, _ := net.LookupIP(ing.Status.LoadBalancer.Ingress[0].Hostname) ++ if len(ip) > 0 { ++ got = ip[0].String() ++ } ++ } + if got != host { + return fmt.Errorf("unexpected ingress status, got %+v want %v", got, host) + } +@@ -617,6 +624,12 @@ spec: + if hostIsIP { + got = ing.Status.LoadBalancer.Ingress[0].IP + } ++ if ing.Status.LoadBalancer.Ingress[0].Hostname != "" { ++ ip, _ := net.LookupIP(ing.Status.LoadBalancer.Ingress[0].Hostname) ++ if len(ip) > 0 { ++ got = ip[0].String() ++ } ++ } + if got != host { + return fmt.Errorf("unexpected ingress status, got %+v want %v", got, host) + } +-- +2.30.1 (Apple Git-130) + diff --git a/tetrateci/patches/eks/eks_1.18.patch b/tetrateci/patches/eks/eks_1.18.patch new file mode 100644 index 000000000000..b6f27d9abd00 --- /dev/null +++ b/tetrateci/patches/eks/eks_1.18.patch @@ -0,0 +1,123 @@ +From 76ba05e0d163dc99389959c9922fd51d89e41f53 Mon Sep 17 00:00:00 2001 +From: psbrar99 +Date: Wed, 2 Aug 2023 15:05:54 -0600 +Subject: [PATCH] patch + +Signed-off-by: psbrar99 +--- + .../components/echo/kube/deployment.go | 1 + + .../echo/kube/templates/vm_deployment.yaml | 22 +++++++++---------- + pkg/test/framework/components/istio/util.go | 9 ++++++++ + tests/integration/pilot/ingress_test.go | 12 +++++----- + 4 files changed, 28 insertions(+), 16 deletions(-) + +diff --git a/pkg/test/framework/components/echo/kube/deployment.go b/pkg/test/framework/components/echo/kube/deployment.go +index a0c723e0e8..0739d0b0bf 100644 +--- a/pkg/test/framework/components/echo/kube/deployment.go ++++ b/pkg/test/framework/components/echo/kube/deployment.go +@@ -522,6 +522,7 @@ spec: + if rev := getIstioRevision(cfg.Namespace); len(rev) > 0 { + cmd = append(cmd, "--revision", rev) + } ++ cmd = append(cmd, "--ingressIP", istiodAddr.Addr().String()) + // make sure namespace controller has time to create root-cert ConfigMap + if err := retry.UntilSuccess(func() error { + stdout, stderr, err := istioCtl.Invoke(cmd) +diff --git a/pkg/test/framework/components/echo/kube/templates/vm_deployment.yaml b/pkg/test/framework/components/echo/kube/templates/vm_deployment.yaml +index 4343abcd5c..1c61d2c1a5 100644 +--- a/pkg/test/framework/components/echo/kube/templates/vm_deployment.yaml ++++ b/pkg/test/framework/components/echo/kube/templates/vm_deployment.yaml +@@ -85,16 +85,16 @@ spec: + + # since we're not overwriting /etc/hosts on k8s, verify that istiod hostname in /etc/hosts + # matches the value generated by istioctl +- echo "checking istio host" +- SYSTEM_HOST=$(cat /etc/hosts | grep istiod) +- ISTIOCTL_HOST=$(cat /var/run/secrets/istio/bootstrap/hosts | grep istiod) +- if [ "$(echo "$SYSTEM_HOST" | tr -d '[:space:]')" != "$(echo "$ISTIOCTL_HOST" | tr -d '[:space:]')" ]; then +- echo "istiod host in /etc/hosts does not match value generated by istioctl" +- echo "/etc/hosts: $SYSTEM_HOST" +- echo "/var/run/secrets/istio/bootstrap/hosts: $ISTIOCTL_HOST" +- exit 1 +- fi +- echo "istiod host ok" ++ # echo "checking istio host" ++ # SYSTEM_HOST=$(cat /etc/hosts | grep istiod) ++ # ISTIOCTL_HOST=$(cat /var/run/secrets/istio/bootstrap/hosts | grep istiod) ++ # if [ "$(echo "$SYSTEM_HOST" | tr -d '[:space:]')" != "$(echo "$ISTIOCTL_HOST" | tr -d '[:space:]')" ]; then ++ # echo "istiod host in /etc/hosts does not match value generated by istioctl" ++ # echo "/etc/hosts: $SYSTEM_HOST" ++ # echo "/var/run/secrets/istio/bootstrap/hosts: $ISTIOCTL_HOST" ++ # exit 1 ++ # fi ++ # echo "istiod host ok" + + # read certs from correct directory + sudo sh -c 'echo PROV_CERT=/var/run/secrets/istio >> /var/lib/istio/envoy/cluster.env' +@@ -173,4 +173,4 @@ spec: + name: {{ $value.Value }} + {{- end }} + {{- end }} +-{{- end}} ++{{- end}} +\ No newline at end of file +diff --git a/pkg/test/framework/components/istio/util.go b/pkg/test/framework/components/istio/util.go +index 2600fe421c..2d01c756c1 100644 +--- a/pkg/test/framework/components/istio/util.go ++++ b/pkg/test/framework/components/istio/util.go +@@ -137,6 +137,15 @@ func getRemoteServiceAddress(s *kube.Settings, cluster cluster.Cluster, ns, labe + if ingr.IP == "" && ingr.Hostname == "" { + return nil, false, fmt.Errorf("service %s/%s is not available yet: no ingress", svc.Namespace, svc.Name) + } ++ if ingr.Hostname != "" { ++ ip, err := net.LookupIP(ingr.Hostname) ++ if err != nil { ++ return nil, false, fmt.Errorf("service %s/%s is not available yet: no ingress", svc.Namespace, svc.Name) ++ } ++ if len(ip) > 0 { ++ ingr.IP = ip[0].String() ++ } ++ } + if ingr.IP != "" { + ipaddr, err := netip.ParseAddr(ingr.IP) + if err != nil { +diff --git a/tests/integration/pilot/ingress_test.go b/tests/integration/pilot/ingress_test.go +index 4f158b96ef..c332e1630e 100644 +--- a/tests/integration/pilot/ingress_test.go ++++ b/tests/integration/pilot/ingress_test.go +@@ -332,6 +332,12 @@ spec: + if hostIsIP { + got = ing.Status.LoadBalancer.Ingress[0].IP + } ++ if ing.Status.LoadBalancer.Ingress[0].Hostname != "" { ++ ip, _ := net.LookupIP(ing.Status.LoadBalancer.Ingress[0].Hostname) ++ if len(ip) > 0 { ++ got = ip[0].String() ++ } ++ } + if got != host { + return fmt.Errorf("unexpected ingress status, got %+v want %v", got, host) + } +@@ -441,10 +447,6 @@ func TestCustomGateway(t *testing.T) { + NewTest(t). + Features("traffic.ingress.custom"). + Run(func(t framework.TestContext) { +- inject := false +- if t.Settings().Compatibility { +- inject = true +- } + injectLabel := `sidecar.istio.io/inject: "true"` + if t.Settings().Revisions.Default() != "" { + injectLabel = fmt.Sprintf(`istio.io/rev: "%v"`, t.Settings().Revisions.Default()) +@@ -458,7 +460,7 @@ func TestCustomGateway(t *testing.T) { + } + + t.NewSubTest("minimal").Run(func(t framework.TestContext) { +- gatewayNs := namespace.NewOrFail(t, t, namespace.Config{Prefix: "custom-gateway-minimal", Inject: inject}) ++ gatewayNs := namespace.NewOrFail(t, t, namespace.Config{Prefix: "custom-gateway-minimal"}) + _ = t.ConfigIstio().Eval(gatewayNs.Name(), templateParams, `apiVersion: v1 + kind: Service + metadata: +-- +2.30.1 (Apple Git-130) + diff --git a/tetrateci/setup_boring_go.sh b/tetrateci/setup_boring_go.sh deleted file mode 100755 index 785b77aafafa..000000000000 --- a/tetrateci/setup_boring_go.sh +++ /dev/null @@ -1,36 +0,0 @@ -#!/usr/bin/env bash -set -e -set -u - -if $(grep -q "1.7" <<< $TAG); then - export GOLANG_VERSION=1.14.12b4 -fi - -if $(grep -q "1.8" <<< $TAG || grep -q "1.9" <<< $TAG); then - export GOLANG_VERSION=1.15.8b5 -fi - -if $(grep -q "1.10" <<< $TAG); then - export GOLANG_VERSION=1.16.9b7 -fi - -if $(grep -q "1.11" <<< $TAG); then - export GOLANG_VERSION=1.17.6b7 -fi - -if $(grep -q "1.12" <<< $TAG) || [[ "${REL_BRANCH_VER:-${ISTIO_MINOR_VER}}" == "1.13" ]]; then - export GOLANG_VERSION=1.17.8b7 -fi - -url="https://go-boringcrypto.storage.googleapis.com/go$GOLANG_VERSION.linux-amd64.tar.gz" - -wget -q -O go.tgz "$url" - -sudo tar -C /usr/local -xzf go.tgz -rm go.tgz - -export GOROOT=/usr/local/go -export PATH="$GOROOT/bin:$PATH" - -echo "FIPS compliant Go installed" -go version diff --git a/tetrateci/setup_go.sh b/tetrateci/setup_go.sh index 2b08113c9060..1672b217161f 100755 --- a/tetrateci/setup_go.sh +++ b/tetrateci/setup_go.sh @@ -22,7 +22,27 @@ if $(grep -q "1.12" <<< $TAG) || [[ "${REL_BRANCH_VER:-${ISTIO_MINOR_VER}}" == " export GOLANG_VERSION=1.17.8 fi -url="https://golang.org/dl/go$GOLANG_VERSION.linux-amd64.tar.gz" +if $(grep -q "1.14" <<< $TAG); then + export GOLANG_VERSION=1.18.4 +fi + +if $(grep -q "1.15" <<< $TAG); then + export GOLANG_VERSION=1.19.2 +fi + +if [[ "${REL_BRANCH_VER:-${ISTIO_MINOR_VER}}" == "1.16" ]]; then + export GOLANG_VERSION=1.19.5 +fi + +if [[ "${REL_BRANCH_VER:-${ISTIO_MINOR_VER}}" == "1.17" ]]; then + export GOLANG_VERSION=1.20.2 +fi + +if [[ "${REL_BRANCH_VER:-${ISTIO_MINOR_VER}}" == "1.18" ]]; then + export GOLANG_VERSION=1.20.6 +fi + +url="https://golang.org/dl/go$GOLANG_VERSION.linux-$(dpkg --print-architecture).tar.gz" wget -q -O go.tgz "$url" diff --git a/tetrateci/test_1.11.sh b/tetrateci/test_1.11.sh index cf2c8e8200ca..6474f997b1bf 100644 --- a/tetrateci/test_1.11.sh +++ b/tetrateci/test_1.11.sh @@ -64,20 +64,6 @@ for pkg in $PACKAGES; do SKIP_TEST_FLAGS+=( "--istio.test.skip=${test}" ) done - go test \ - -test.v \ - -timeout 2h \ - -tags=integ \ - "${pkg}" \ - --istio.test.select=-postsubmit,-flaky \ - ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ - --istio.test.ci \ - --istio.test.hub=${HUB} \ - --istio.test.tag=${TAG}-distroless \ - --istio.test.pullpolicy=IfNotPresent \ - --istio.test.retries=1 \ - ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ - && \ go test \ -test.v \ -timeout 2h \ diff --git a/tetrateci/test_1.12.sh b/tetrateci/test_1.12.sh index 1c0210d16cb3..ad7d9a53294e 100755 --- a/tetrateci/test_1.12.sh +++ b/tetrateci/test_1.12.sh @@ -49,20 +49,6 @@ for pkg in $PACKAGES; do SKIP_TEST_FLAGS+=( "--istio.test.skip=${test}" ) done - go test \ - -test.v \ - -timeout 2h \ - -tags=integ \ - "${pkg}" \ - --istio.test.select=-postsubmit,-flaky \ - ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ - --istio.test.ci \ - --istio.test.hub=${HUB} \ - --istio.test.tag=${TAG}-distroless \ - --istio.test.pullpolicy=IfNotPresent \ - --istio.test.retries=1 \ - ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ - && \ go test \ -test.v \ -timeout 2h \ diff --git a/tetrateci/test_1.13.sh b/tetrateci/test_1.13.sh index 898b7cf02532..6474f997b1bf 100644 --- a/tetrateci/test_1.13.sh +++ b/tetrateci/test_1.13.sh @@ -31,11 +31,13 @@ if [[ "${CLUSTER}" == "gke" ]]; then COMMON_TEST_FLAGS+=( "-istio.test.kube.helm.iopFile=${SCRIPTDIR}/iop-gke-integration.yml" ) + echo "Applying GKE specific patches...." + git apply "${SCRIPTDIR}/patches/gke/chiron-gke.patch" fi if [[ "${CLUSTER}" == "eks" ]]; then echo "Applying Ingress patch for EKS...." - git apply "${SCRIPTDIR}/patches/eks/eks-ingress.1.13.patch" + git apply "${SCRIPTDIR}/patches/eks/eks-ingress.1.11.patch" fi PACKAGES=$(go list -tags=integ "${ROOTDIR}/tests/integration/...") @@ -62,20 +64,6 @@ for pkg in $PACKAGES; do SKIP_TEST_FLAGS+=( "--istio.test.skip=${test}" ) done - go test \ - -test.v \ - -timeout 2h \ - -tags=integ \ - "${pkg}" \ - --istio.test.select=-postsubmit,-flaky \ - ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ - --istio.test.ci \ - --istio.test.hub=${HUB} \ - --istio.test.tag=${TAG}-distroless \ - --istio.test.pullpolicy=IfNotPresent \ - --istio.test.retries=1 \ - ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ - && \ go test \ -test.v \ -timeout 2h \ diff --git a/tetrateci/test_1.14.sh b/tetrateci/test_1.14.sh new file mode 100644 index 000000000000..898b7cf02532 --- /dev/null +++ b/tetrateci/test_1.14.sh @@ -0,0 +1,107 @@ +#!/usr/bin/env bash +# +# Copyright (c) Tetrate, Inc 2022 All Rights Reserved. + +set -e +set -u +set -x + +SCRIPTDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ) +ROOTDIR=$( cd "${SCRIPTDIR}/.." && pwd ) + +"${SCRIPTDIR}/version_check.py" && exit + +# shellcheck disable=SC1091 +source "${SCRIPTDIR}/setup_go.sh" + +COMMON_TEST_FLAGS=() + +echo "Applying patches...." + +# Apply the same patches that were applies when building test images +"${SCRIPTDIR}/apply_e2e_build_patches.sh" + +git apply "${SCRIPTDIR}/patches/common/increase-dashboard-timeout.1.11.patch" + +if [[ "${CLUSTER}" == "gke" ]]; then + echo "Generating operator config for GKE" + + # Overlay CNI Parameters for GCP : https://github.com/tetratelabs/getistio/issues/76 + python3 -m pip install pyyaml --user && python3 "${SCRIPTDIR}/gen_iop.py" + + COMMON_TEST_FLAGS+=( "-istio.test.kube.helm.iopFile=${SCRIPTDIR}/iop-gke-integration.yml" ) + +fi + +if [[ "${CLUSTER}" == "eks" ]]; then + echo "Applying Ingress patch for EKS...." + git apply "${SCRIPTDIR}/patches/eks/eks-ingress.1.13.patch" +fi + +PACKAGES=$(go list -tags=integ "${ROOTDIR}/tests/integration/...") + +echo "Starting Testing" + +FAILED_PACKAGES=() + +for pkg in $PACKAGES; do + echo "========================================================TESTING ${pkg} ========================================================" + + SKIP_RULE=$( grep -F "${pkg}=" "${SCRIPTDIR}/${ISTIO_MINOR_VER}/test/skip.d/${CLUSTER}" 2>/dev/null || echo "" ) + SKIP_TESTS=$( echo -n "${SKIP_RULE#${pkg}=}" ) + + if [[ "${SKIP_TESTS}" == "*" ]]; then + echo "Skipping according to the rule: ${SKIP_RULE}" + continue + fi + + read -ra SKIP_TESTS_ARRAY <<< "${SKIP_TESTS}" + + SKIP_TEST_FLAGS=() + for test in ${SKIP_TESTS_ARRAY[@]+"${SKIP_TESTS_ARRAY[@]}"} ; do + SKIP_TEST_FLAGS+=( "--istio.test.skip=${test}" ) + done + + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.hub=${HUB} \ + --istio.test.tag=${TAG}-distroless \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + && \ + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + || \ + { FAILED_PACKAGES+=( "${pkg}" ) && echo "Test Failed: ${pkg}" ; } + + find /tmp -mindepth 1 -maxdepth 1 -type d -name '*istio*' -exec sudo rm -f -- {} \; +done + +echo "Testing Done" + +if [[ ${#FAILED_PACKAGES[@]} -gt 0 ]]; then + echo "" + echo "Some of the tests have failed :(" + echo "" + echo "Packages with failed tests:" + for pkg in "${FAILED_PACKAGES[@]}"; do + echo "- ${pkg}" + done + exit 1 +fi diff --git a/tetrateci/test_1.15.sh b/tetrateci/test_1.15.sh new file mode 100644 index 000000000000..69f20af588a9 --- /dev/null +++ b/tetrateci/test_1.15.sh @@ -0,0 +1,108 @@ +#!/usr/bin/env bash +# +# Copyright (c) Tetrate, Inc 2022 All Rights Reserved. + +set -e +set -u +set -x + +SCRIPTDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ) +ROOTDIR=$( cd "${SCRIPTDIR}/.." && pwd ) + +"${SCRIPTDIR}/version_check.py" && exit + +# shellcheck disable=SC1091 +source "${SCRIPTDIR}/setup_go.sh" + +COMMON_TEST_FLAGS=() + +echo "Applying patches...." + +# Apply the same patches that were applies when building test images +"${SCRIPTDIR}/apply_e2e_build_patches.sh" + +git apply "${SCRIPTDIR}/patches/common/increase-dashboard-timeout.1.11.patch" + +if [[ "${CLUSTER}" == "gke" ]]; then + echo "Generating operator config for GKE" + + # Overlay CNI Parameters for GCP : https://github.com/tetratelabs/getistio/issues/76 + python3 -m pip install pyyaml --user && python3 "${SCRIPTDIR}/gen_iop.py" + + COMMON_TEST_FLAGS+=( "-istio.test.kube.helm.iopFile=${SCRIPTDIR}/iop-gke-integration.yml" ) + +fi + +if [[ "${CLUSTER}" == "eks" ]]; then + echo "Applying patch for EKS...." + git apply --3way "${SCRIPTDIR}/patches/eks/eks.${ISTIO_MINOR_VER}.patch" +fi + + +PACKAGES=$(go list -tags=integ "${ROOTDIR}/tests/integration/...") + +echo "Starting Testing" + +FAILED_PACKAGES=() + +for pkg in $PACKAGES; do + echo "========================================================TESTING ${pkg} ========================================================" + + SKIP_RULE=$( grep -F "${pkg}=" "${SCRIPTDIR}/${ISTIO_MINOR_VER}/test/skip.d/${CLUSTER}" 2>/dev/null || echo "" ) + SKIP_TESTS=$( echo -n "${SKIP_RULE#${pkg}=}" ) + + if [[ "${SKIP_TESTS}" == "*" ]]; then + echo "Skipping according to the rule: ${SKIP_RULE}" + continue + fi + + read -ra SKIP_TESTS_ARRAY <<< "${SKIP_TESTS}" + + SKIP_TEST_FLAGS=() + for test in ${SKIP_TESTS_ARRAY[@]+"${SKIP_TESTS_ARRAY[@]}"} ; do + SKIP_TEST_FLAGS+=( "--istio.test.skip=${test}" ) + done + + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.hub=${HUB} \ + --istio.test.tag=${TAG}-distroless \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + && \ + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + || \ + { FAILED_PACKAGES+=( "${pkg}" ) && echo "Test Failed: ${pkg}" ; } + + find /tmp -mindepth 1 -maxdepth 1 -type d -name '*istio*' -exec sudo rm -f -- {} \; +done + +echo "Testing Done" + +if [[ ${#FAILED_PACKAGES[@]} -gt 0 ]]; then + echo "" + echo "Some of the tests have failed :(" + echo "" + echo "Packages with failed tests:" + for pkg in "${FAILED_PACKAGES[@]}"; do + echo "- ${pkg}" + done + exit 1 +fi diff --git a/tetrateci/test_1.15_arm.sh b/tetrateci/test_1.15_arm.sh new file mode 100644 index 000000000000..51457acc93ce --- /dev/null +++ b/tetrateci/test_1.15_arm.sh @@ -0,0 +1,111 @@ +#!/usr/bin/env bash +# +# Copyright (c) Tetrate, Inc 2022 All Rights Reserved. + +set -e +set -u +set -x + +SCRIPTDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ) +ROOTDIR=$( cd "${SCRIPTDIR}/.." && pwd ) + +"${SCRIPTDIR}/version_check.py" && exit + +# shellcheck disable=SC1091 +source "${SCRIPTDIR}/setup_go.sh" + +COMMON_TEST_FLAGS=() + +echo "Applying patches...." + +# Apply the same patches that were applies when building test images +"${SCRIPTDIR}/apply_e2e_build_patches.sh" + +git apply "${SCRIPTDIR}/patches/common/increase-dashboard-timeout.1.11.patch" + +if [[ "${CLUSTER}" == "gke" ]]; then + echo "Generating operator config for GKE" + + # Overlay CNI Parameters for GCP : https://github.com/tetratelabs/getistio/issues/76 + python3 -m pip install pyyaml --user && python3 "${SCRIPTDIR}/gen_iop.py" + + COMMON_TEST_FLAGS+=( "-istio.test.kube.helm.iopFile=${SCRIPTDIR}/iop-gke-integration.yml" ) + +fi + +if [[ "${CLUSTER}" == "eks" ]]; then + echo "Applying patch for EKS...." + git apply --3way "${SCRIPTDIR}/patches/eks/eks.${ISTIO_MINOR_VER}.patch" +fi + +#go test -test.v -timeout 2h -tags=integ istio.io/istio/tests/integration/security --istio.test.select=-postsubmit,-flaky --istio.test.ci --istio.test.hub=${HUB} --istio.test.tag=${TAG}-distroless --istio.test.pullpolicy=IfNotPresent --istio.test.retries=1 && go test -test.v -timeout 2h -tags=integ istio.io/istio/tests/integration/security --istio.test.select=-postsubmit,-flaky --istio.test.ci --istio.test.hub=${HUB} --istio.test.tag=${TAG} --istio.test.pullpolicy=IfNotPresent + +PACKAGES=$(go list -tags=integ "${ROOTDIR}/tests/integration/...") + +echo "Starting Testing" + +FAILED_PACKAGES=() + +for pkg in $PACKAGES; do + echo "========================================================TESTING ${pkg} ========================================================" + + SKIP_RULE=$( grep -F "${pkg}=" "${SCRIPTDIR}/${ISTIO_MINOR_VER}/test/skip.d/${CLUSTER}" 2>/dev/null || echo "" ) + SKIP_TESTS=$( echo -n "${SKIP_RULE#${pkg}=}" ) + + if [[ "${SKIP_TESTS}" == "*" ]]; then + echo "Skipping according to the rule: ${SKIP_RULE}" + continue + fi + + read -ra SKIP_TESTS_ARRAY <<< "${SKIP_TESTS}" + + SKIP_TEST_FLAGS=() + for test in ${SKIP_TESTS_ARRAY[@]+"${SKIP_TESTS_ARRAY[@]}"} ; do + SKIP_TEST_FLAGS+=( "--istio.test.skip=${test}" ) + done + + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.skipVM=true \ + --istio.test.hub=${HUB} \ + --istio.test.tag=${TAG}-distroless \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + && \ + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.skipVM=true \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + || \ + { FAILED_PACKAGES+=( "${pkg}" ) && echo "Test Failed: ${pkg}" ; } + + find /tmp -mindepth 1 -maxdepth 1 -type d -name '*istio*' -exec sudo rm -f -- {} \; +done + +echo "Testing Done" + +if [[ ${#FAILED_PACKAGES[@]} -gt 0 ]]; then + echo "" + echo "Some of the tests have failed :(" + echo "" + echo "Packages with failed tests:" + for pkg in "${FAILED_PACKAGES[@]}"; do + echo "- ${pkg}" + done + exit 1 +fi diff --git a/tetrateci/test_1.16.sh b/tetrateci/test_1.16.sh new file mode 100644 index 000000000000..087521b02d95 --- /dev/null +++ b/tetrateci/test_1.16.sh @@ -0,0 +1,107 @@ +#!/usr/bin/env bash +# +# Copyright (c) Tetrate, Inc 2022 All Rights Reserved. + +set -e +set -u +set -x + +SCRIPTDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ) +ROOTDIR=$( cd "${SCRIPTDIR}/.." && pwd ) + +"${SCRIPTDIR}/version_check.py" && exit + +# shellcheck disable=SC1091 +source "${SCRIPTDIR}/setup_go.sh" + +COMMON_TEST_FLAGS=() + +echo "Applying patches...." + +# Apply the same patches that were applies when building test images +"${SCRIPTDIR}/apply_e2e_build_patches.sh" + + +if [[ "${CLUSTER}" == "gke" ]]; then + echo "Generating operator config for GKE" + + # Overlay CNI Parameters for GCP : https://github.com/tetratelabs/getistio/issues/76 + python3 -m pip install pyyaml --user && python3 "${SCRIPTDIR}/gen_iop.py" + + COMMON_TEST_FLAGS+=( "-istio.test.kube.helm.iopFile=${SCRIPTDIR}/iop-gke-integration.yml" ) + +fi + +if [[ "${CLUSTER}" == "eks" ]]; then + echo "Applying patch for EKS...." + git apply --3way "${SCRIPTDIR}/patches/eks/eks.${ISTIO_MINOR_VER}.patch" +fi + + +PACKAGES=$(go list -tags=integ "${ROOTDIR}/tests/integration/...") + +echo "Starting Testing" + +FAILED_PACKAGES=() + +for pkg in $PACKAGES; do + echo "========================================================TESTING ${pkg} ========================================================" + + SKIP_RULE=$( grep -F "${pkg}=" "${SCRIPTDIR}/${ISTIO_MINOR_VER}/test/skip.d/${CLUSTER}" 2>/dev/null || echo "" ) + SKIP_TESTS=$( echo -n "${SKIP_RULE#${pkg}=}" ) + + if [[ "${SKIP_TESTS}" == "*" ]]; then + echo "Skipping according to the rule: ${SKIP_RULE}" + continue + fi + + read -ra SKIP_TESTS_ARRAY <<< "${SKIP_TESTS}" + + SKIP_TEST_FLAGS=() + for test in ${SKIP_TESTS_ARRAY[@]+"${SKIP_TESTS_ARRAY[@]}"} ; do + SKIP_TEST_FLAGS+=( "--istio.test.skip=${test}" ) + done + + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.hub=${HUB} \ + --istio.test.tag=${TAG}-distroless \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + && \ + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + || \ + { FAILED_PACKAGES+=( "${pkg}" ) && echo "Test Failed: ${pkg}" ; } + + find /tmp -mindepth 1 -maxdepth 1 -type d -name '*istio*' -exec sudo rm -f -- {} \; +done + +echo "Testing Done" + +if [[ ${#FAILED_PACKAGES[@]} -gt 0 ]]; then + echo "" + echo "Some of the tests have failed :(" + echo "" + echo "Packages with failed tests:" + for pkg in "${FAILED_PACKAGES[@]}"; do + echo "- ${pkg}" + done + exit 1 +fi diff --git a/tetrateci/test_1.16_arm.sh b/tetrateci/test_1.16_arm.sh new file mode 100644 index 000000000000..97c23342d529 --- /dev/null +++ b/tetrateci/test_1.16_arm.sh @@ -0,0 +1,111 @@ +#!/usr/bin/env bash +# +# Copyright (c) Tetrate, Inc 2022 All Rights Reserved. + +set -e +set -u +set -x + +SCRIPTDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ) +ROOTDIR=$( cd "${SCRIPTDIR}/.." && pwd ) + +"${SCRIPTDIR}/version_check.py" && exit + +# shellcheck disable=SC1091 +source "${SCRIPTDIR}/setup_go.sh" + +COMMON_TEST_FLAGS=() + +echo "Applying patches...." + +# Apply the same patches that were applies when building test images +"${SCRIPTDIR}/apply_e2e_build_patches.sh" + +#git apply "${SCRIPTDIR}/patches/common/increase-dashboard-timeout.1.11.patch" + +if [[ "${CLUSTER}" == "gke" ]]; then + echo "Generating operator config for GKE" + + # Overlay CNI Parameters for GCP : https://github.com/tetratelabs/getistio/issues/76 + python3 -m pip install pyyaml --user && python3 "${SCRIPTDIR}/gen_iop.py" + + COMMON_TEST_FLAGS+=( "-istio.test.kube.helm.iopFile=${SCRIPTDIR}/iop-gke-integration.yml" ) + +fi + +if [[ "${CLUSTER}" == "eks" ]]; then + echo "Applying patch for EKS...." + git apply --3way "${SCRIPTDIR}/patches/eks/eks.${ISTIO_MINOR_VER}.patch" +fi + +#go test -test.v -timeout 2h -tags=integ istio.io/istio/tests/integration/security --istio.test.select=-postsubmit,-flaky --istio.test.ci --istio.test.hub=${HUB} --istio.test.tag=${TAG}-distroless --istio.test.pullpolicy=IfNotPresent --istio.test.retries=1 && go test -test.v -timeout 2h -tags=integ istio.io/istio/tests/integration/security --istio.test.select=-postsubmit,-flaky --istio.test.ci --istio.test.hub=${HUB} --istio.test.tag=${TAG} --istio.test.pullpolicy=IfNotPresent + +PACKAGES=$(go list -tags=integ "${ROOTDIR}/tests/integration/...") + +echo "Starting Testing" + +FAILED_PACKAGES=() + +for pkg in $PACKAGES; do + echo "========================================================TESTING ${pkg} ========================================================" + + SKIP_RULE=$( grep -F "${pkg}=" "${SCRIPTDIR}/${ISTIO_MINOR_VER}/test/skip.d/${CLUSTER}" 2>/dev/null || echo "" ) + SKIP_TESTS=$( echo -n "${SKIP_RULE#${pkg}=}" ) + + if [[ "${SKIP_TESTS}" == "*" ]]; then + echo "Skipping according to the rule: ${SKIP_RULE}" + continue + fi + + read -ra SKIP_TESTS_ARRAY <<< "${SKIP_TESTS}" + + SKIP_TEST_FLAGS=() + for test in ${SKIP_TESTS_ARRAY[@]+"${SKIP_TESTS_ARRAY[@]}"} ; do + SKIP_TEST_FLAGS+=( "--istio.test.skip=${test}" ) + done + + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.skipVM=true \ + --istio.test.hub=${HUB} \ + --istio.test.tag=${TAG}-distroless \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + && \ + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.skipVM=true \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + || \ + { FAILED_PACKAGES+=( "${pkg}" ) && echo "Test Failed: ${pkg}" ; } + + find /tmp -mindepth 1 -maxdepth 1 -type d -name '*istio*' -exec sudo rm -f -- {} \; +done + +echo "Testing Done" + +if [[ ${#FAILED_PACKAGES[@]} -gt 0 ]]; then + echo "" + echo "Some of the tests have failed :(" + echo "" + echo "Packages with failed tests:" + for pkg in "${FAILED_PACKAGES[@]}"; do + echo "- ${pkg}" + done + exit 1 +fi diff --git a/tetrateci/test_1.17.sh b/tetrateci/test_1.17.sh new file mode 100644 index 000000000000..6526631843e3 --- /dev/null +++ b/tetrateci/test_1.17.sh @@ -0,0 +1,109 @@ +#!/usr/bin/env bash +# +# Copyright (c) Tetrate, Inc 2022 All Rights Reserved. + +set -e +set -u +set -x + +SCRIPTDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ) +ROOTDIR=$( cd "${SCRIPTDIR}/.." && pwd ) + +"${SCRIPTDIR}/version_check.py" && exit + +# shellcheck disable=SC1091 +source "${SCRIPTDIR}/setup_go.sh" + +COMMON_TEST_FLAGS=() + +echo "Applying patches...." + +# Apply the same patches that were applies when building test images +"${SCRIPTDIR}/apply_e2e_build_patches.sh" + + +if [[ "${CLUSTER}" == "gke" ]]; then + echo "Generating operator config for GKE" + + # Overlay CNI Parameters for GCP : https://github.com/tetratelabs/getistio/issues/76 + python3 -m pip install pyyaml --user && python3 "${SCRIPTDIR}/gen_iop.py" + + COMMON_TEST_FLAGS+=( "-istio.test.kube.helm.iopFile=${SCRIPTDIR}/iop-gke-integration.yml" ) + +fi + +if [[ "${CLUSTER}" == "eks" ]]; then + echo "Applying patch for EKS...." + git apply --3way "${SCRIPTDIR}/patches/eks/eks.${ISTIO_MINOR_VER}.patch" +fi + + +PACKAGES=$(go list -tags=integ "${ROOTDIR}/tests/integration/...") + +echo "Starting Testing" + +FAILED_PACKAGES=() + +for pkg in $PACKAGES; do + echo "========================================================TESTING ${pkg} ========================================================" + + SKIP_RULE=$( grep -F "${pkg}=" "${SCRIPTDIR}/${ISTIO_MINOR_VER}/test/skip.d/${CLUSTER}" 2>/dev/null || echo "" ) + SKIP_TESTS=$( echo -n "${SKIP_RULE#${pkg}=}" ) + + if [[ "${SKIP_TESTS}" == "*" ]]; then + echo "Skipping according to the rule: ${SKIP_RULE}" + continue + fi + + read -ra SKIP_TESTS_ARRAY <<< "${SKIP_TESTS}" + + SKIP_TEST_FLAGS=() + for test in ${SKIP_TESTS_ARRAY[@]+"${SKIP_TESTS_ARRAY[@]}"} ; do + SKIP_TEST_FLAGS+=( "--istio.test.skip=${test}" ) + done + + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.skipVM=true \ + --istio.test.hub=${HUB} \ + --istio.test.tag=${TAG}-distroless \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + && \ + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.skipVM=true \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + || \ + { FAILED_PACKAGES+=( "${pkg}" ) && echo "Test Failed: ${pkg}" ; } + + find /tmp -mindepth 1 -maxdepth 1 -type d -name '*istio*' -exec sudo rm -f -- {} \; +done + +echo "Testing Done" + +if [[ ${#FAILED_PACKAGES[@]} -gt 0 ]]; then + echo "" + echo "Some of the tests have failed :(" + echo "" + echo "Packages with failed tests:" + for pkg in "${FAILED_PACKAGES[@]}"; do + echo "- ${pkg}" + done + exit 1 +fi diff --git a/tetrateci/test_1.17_arm.sh b/tetrateci/test_1.17_arm.sh new file mode 100644 index 000000000000..97c23342d529 --- /dev/null +++ b/tetrateci/test_1.17_arm.sh @@ -0,0 +1,111 @@ +#!/usr/bin/env bash +# +# Copyright (c) Tetrate, Inc 2022 All Rights Reserved. + +set -e +set -u +set -x + +SCRIPTDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ) +ROOTDIR=$( cd "${SCRIPTDIR}/.." && pwd ) + +"${SCRIPTDIR}/version_check.py" && exit + +# shellcheck disable=SC1091 +source "${SCRIPTDIR}/setup_go.sh" + +COMMON_TEST_FLAGS=() + +echo "Applying patches...." + +# Apply the same patches that were applies when building test images +"${SCRIPTDIR}/apply_e2e_build_patches.sh" + +#git apply "${SCRIPTDIR}/patches/common/increase-dashboard-timeout.1.11.patch" + +if [[ "${CLUSTER}" == "gke" ]]; then + echo "Generating operator config for GKE" + + # Overlay CNI Parameters for GCP : https://github.com/tetratelabs/getistio/issues/76 + python3 -m pip install pyyaml --user && python3 "${SCRIPTDIR}/gen_iop.py" + + COMMON_TEST_FLAGS+=( "-istio.test.kube.helm.iopFile=${SCRIPTDIR}/iop-gke-integration.yml" ) + +fi + +if [[ "${CLUSTER}" == "eks" ]]; then + echo "Applying patch for EKS...." + git apply --3way "${SCRIPTDIR}/patches/eks/eks.${ISTIO_MINOR_VER}.patch" +fi + +#go test -test.v -timeout 2h -tags=integ istio.io/istio/tests/integration/security --istio.test.select=-postsubmit,-flaky --istio.test.ci --istio.test.hub=${HUB} --istio.test.tag=${TAG}-distroless --istio.test.pullpolicy=IfNotPresent --istio.test.retries=1 && go test -test.v -timeout 2h -tags=integ istio.io/istio/tests/integration/security --istio.test.select=-postsubmit,-flaky --istio.test.ci --istio.test.hub=${HUB} --istio.test.tag=${TAG} --istio.test.pullpolicy=IfNotPresent + +PACKAGES=$(go list -tags=integ "${ROOTDIR}/tests/integration/...") + +echo "Starting Testing" + +FAILED_PACKAGES=() + +for pkg in $PACKAGES; do + echo "========================================================TESTING ${pkg} ========================================================" + + SKIP_RULE=$( grep -F "${pkg}=" "${SCRIPTDIR}/${ISTIO_MINOR_VER}/test/skip.d/${CLUSTER}" 2>/dev/null || echo "" ) + SKIP_TESTS=$( echo -n "${SKIP_RULE#${pkg}=}" ) + + if [[ "${SKIP_TESTS}" == "*" ]]; then + echo "Skipping according to the rule: ${SKIP_RULE}" + continue + fi + + read -ra SKIP_TESTS_ARRAY <<< "${SKIP_TESTS}" + + SKIP_TEST_FLAGS=() + for test in ${SKIP_TESTS_ARRAY[@]+"${SKIP_TESTS_ARRAY[@]}"} ; do + SKIP_TEST_FLAGS+=( "--istio.test.skip=${test}" ) + done + + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.skipVM=true \ + --istio.test.hub=${HUB} \ + --istio.test.tag=${TAG}-distroless \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + && \ + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.skipVM=true \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + || \ + { FAILED_PACKAGES+=( "${pkg}" ) && echo "Test Failed: ${pkg}" ; } + + find /tmp -mindepth 1 -maxdepth 1 -type d -name '*istio*' -exec sudo rm -f -- {} \; +done + +echo "Testing Done" + +if [[ ${#FAILED_PACKAGES[@]} -gt 0 ]]; then + echo "" + echo "Some of the tests have failed :(" + echo "" + echo "Packages with failed tests:" + for pkg in "${FAILED_PACKAGES[@]}"; do + echo "- ${pkg}" + done + exit 1 +fi diff --git a/tetrateci/test_1.18.sh b/tetrateci/test_1.18.sh new file mode 100644 index 000000000000..45131e81c948 --- /dev/null +++ b/tetrateci/test_1.18.sh @@ -0,0 +1,109 @@ +#!/usr/bin/env bash +# +# Copyright (c) Tetrate, Inc 2022 All Rights Reserved. + +set -e +set -u +set -x + +SCRIPTDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ) +ROOTDIR=$( cd "${SCRIPTDIR}/.." && pwd ) + +"${SCRIPTDIR}/version_check.py" && exit + +# shellcheck disable=SC1091 +source "${SCRIPTDIR}/setup_go.sh" + +COMMON_TEST_FLAGS=() + +echo "Applying patches...." + +# Apply the same patches that were applies when building test images +"${SCRIPTDIR}/apply_e2e_build_patches.sh" + + +if [[ "${CLUSTER}" == "gke" ]]; then + echo "Generating operator config for GKE" + + # Overlay CNI Parameters for GCP : https://github.com/tetratelabs/getistio/issues/76 + python3 -m pip install pyyaml --user && python3 "${SCRIPTDIR}/gen_iop.py" + + COMMON_TEST_FLAGS+=( "-istio.test.kube.helm.iopFile=${SCRIPTDIR}/iop-gke-integration.yml" ) + +fi + +if [[ "${CLUSTER}" == "eks" ]]; then + echo "Applying patch for EKS...." + git apply --3way "${SCRIPTDIR}/patches/eks/eks.${ISTIO_MINOR_VER}.patch" +fi + + +PACKAGES=$(go list -tags=integ "${ROOTDIR}/tests/integration/...") + +echo "Starting Testing" + +FAILED_PACKAGES=() + +for pkg in $PACKAGES; do + echo "========================================================TESTING ${pkg} ========================================================" + + SKIP_RULE=$( grep -F "${pkg}=" "${SCRIPTDIR}/${ISTIO_MINOR_VER}/test/skip.d/${CLUSTER}" 2>/dev/null || echo "" ) + SKIP_TESTS=$( echo -n "${SKIP_RULE#${pkg}=}" ) + + if [[ "${SKIP_TESTS}" == "*" ]]; then + echo "Skipping according to the rule: ${SKIP_RULE}" + continue + fi + + read -ra SKIP_TESTS_ARRAY <<< "${SKIP_TESTS}" + + SKIP_TEST_FLAGS=() + for test in ${SKIP_TESTS_ARRAY[@]+"${SKIP_TESTS_ARRAY[@]}"} ; do + SKIP_TEST_FLAGS+=( "--istio.test.skip=${test}" ) + done + + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.skipVM=true \ + --istio.test.hub=${HUB} \ + --istio.test.tag=${TAG}-distroless \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + && \ + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.skipVM=true \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + || \ + { FAILED_PACKAGES+=( "${pkg}" ) && echo "Test Failed: ${pkg}" ; } + + find /tmp -mindepth 1 -maxdepth 1 -type d -name '*istio*' -exec sudo rm -f -- {} \; +done + +echo "Testing Done" + +if [[ ${#FAILED_PACKAGES[@]} -gt 0 ]]; then + echo "" + echo "Some of the tests have failed :(" + echo "" + echo "Packages with failed tests:" + for pkg in "${FAILED_PACKAGES[@]}"; do + echo "- ${pkg}" + done + exit 1 +fi \ No newline at end of file diff --git a/tetrateci/test_1.18_arm.sh b/tetrateci/test_1.18_arm.sh new file mode 100644 index 000000000000..4e41de5ce0aa --- /dev/null +++ b/tetrateci/test_1.18_arm.sh @@ -0,0 +1,111 @@ +#!/usr/bin/env bash +# +# Copyright (c) Tetrate, Inc 2022 All Rights Reserved. + +set -e +set -u +set -x + +SCRIPTDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ) +ROOTDIR=$( cd "${SCRIPTDIR}/.." && pwd ) + +"${SCRIPTDIR}/version_check.py" && exit + +# shellcheck disable=SC1091 +source "${SCRIPTDIR}/setup_go.sh" + +COMMON_TEST_FLAGS=() + +echo "Applying patches...." + +# Apply the same patches that were applies when building test images +"${SCRIPTDIR}/apply_e2e_build_patches.sh" + +#git apply "${SCRIPTDIR}/patches/common/increase-dashboard-timeout.1.11.patch" + +if [[ "${CLUSTER}" == "gke" ]]; then + echo "Generating operator config for GKE" + + # Overlay CNI Parameters for GCP : https://github.com/tetratelabs/getistio/issues/76 + python3 -m pip install pyyaml --user && python3 "${SCRIPTDIR}/gen_iop.py" + + COMMON_TEST_FLAGS+=( "-istio.test.kube.helm.iopFile=${SCRIPTDIR}/iop-gke-integration.yml" ) + +fi + +if [[ "${CLUSTER}" == "eks" ]]; then + echo "Applying patch for EKS...." + git apply --3way "${SCRIPTDIR}/patches/eks/eks.${ISTIO_MINOR_VER}.patch" +fi + +#go test -test.v -timeout 2h -tags=integ istio.io/istio/tests/integration/security --istio.test.select=-postsubmit,-flaky --istio.test.ci --istio.test.hub=${HUB} --istio.test.tag=${TAG}-distroless --istio.test.pullpolicy=IfNotPresent --istio.test.retries=1 && go test -test.v -timeout 2h -tags=integ istio.io/istio/tests/integration/security --istio.test.select=-postsubmit,-flaky --istio.test.ci --istio.test.hub=${HUB} --istio.test.tag=${TAG} --istio.test.pullpolicy=IfNotPresent + +PACKAGES=$(go list -tags=integ "${ROOTDIR}/tests/integration/...") + +echo "Starting Testing" + +FAILED_PACKAGES=() + +for pkg in $PACKAGES; do + echo "========================================================TESTING ${pkg} ========================================================" + + SKIP_RULE=$( grep -F "${pkg}=" "${SCRIPTDIR}/${ISTIO_MINOR_VER}/test/skip.d/${CLUSTER}" 2>/dev/null || echo "" ) + SKIP_TESTS=$( echo -n "${SKIP_RULE#${pkg}=}" ) + + if [[ "${SKIP_TESTS}" == "*" ]]; then + echo "Skipping according to the rule: ${SKIP_RULE}" + continue + fi + + read -ra SKIP_TESTS_ARRAY <<< "${SKIP_TESTS}" + + SKIP_TEST_FLAGS=() + for test in ${SKIP_TESTS_ARRAY[@]+"${SKIP_TESTS_ARRAY[@]}"} ; do + SKIP_TEST_FLAGS+=( "--istio.test.skip=${test}" ) + done + + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.skipVM=true \ + --istio.test.hub=${HUB} \ + --istio.test.tag=${TAG}-distroless \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + && \ + go test \ + -test.v \ + -timeout 2h \ + -tags=integ \ + "${pkg}" \ + --istio.test.select=-postsubmit,-flaky \ + ${SKIP_TEST_FLAGS[@]+"${SKIP_TEST_FLAGS[@]}"} \ + --istio.test.ci \ + --istio.test.skipVM=true \ + --istio.test.pullpolicy=IfNotPresent \ + --istio.test.retries=1 \ + ${COMMON_TEST_FLAGS[@]+"${COMMON_TEST_FLAGS[@]}"} \ + || \ + { FAILED_PACKAGES+=( "${pkg}" ) && echo "Test Failed: ${pkg}" ; } + + find /tmp -mindepth 1 -maxdepth 1 -type d -name '*istio*' -exec sudo rm -f -- {} \; +done + +echo "Testing Done" + +if [[ ${#FAILED_PACKAGES[@]} -gt 0 ]]; then + echo "" + echo "Some of the tests have failed :(" + echo "" + echo "Packages with failed tests:" + for pkg in "${FAILED_PACKAGES[@]}"; do + echo "- ${pkg}" + done + exit 1 +fi \ No newline at end of file diff --git a/tetrateci/version_check.py b/tetrateci/version_check.py index 3c84064b6b16..93ddd11e6f89 100755 --- a/tetrateci/version_check.py +++ b/tetrateci/version_check.py @@ -10,6 +10,11 @@ "1.11": {"1.18", "1.19", "1.20", "1.21", "1.22"}, "1.12": {"1.19", "1.20", "1.21", "1.22"}, # officially supported versions according to https://istio.io/latest/news/releases/1.12.x/announcing-1.12 "1.13": {"1.20", "1.21", "1.22", "1.23"}, # officially supported versions according to https://istio.io/latest/news/releases/1.13.x/announcing-1.13 + "1.14": {"1.20","1.21", "1.22", "1.23", "1.24"} , # officially supported versions according to https://istio.io/latest/news/releases/1.14.x/announcing-1.14 + "1.15": {"1.21","1.22", "1.23", "1.24", "1.25"} , # officially supported versions according to https://istio.io/latest/news/releases/1.15.x/announcing-1.15 + "1.16": {"1.22", "1.23", "1.24", "1.25"} , # officially supported versions according to https://istio.io/latest/news/releases/1.16.x/announcing-1.16 + "1.17": {"1.22", "1.23", "1.24", "1.25"} , # officially supported versions according to https://istio.io/latest/news/releases/1.17.x/announcing-1.17 + "1.18": {"1.24", "1.25", "1.26", "1.27"} , # officially supported versions according to https://istio.io/latest/news/releases/1.18.x/announcing-1.18 } istio_ver = os.environ.get("ISTIO_MINOR_VER")