Skip to content

Commit

Permalink
[TT-13723] Update to Go 1.23 (#6812)
Browse files Browse the repository at this point in the history
### **User description**
<details open>
<summary><a href="https://tyktech.atlassian.net/browse/TT-13723"
title="TT-13723" target="_blank">TT-13723</a></summary>
  <br />
  <table>
    <tr>
      <th>Summary</th>
      <td>Update to Go 1.23</td>
    </tr>
    <tr>
      <th>Type</th>
      <td>
<img alt="Story"
src="https://tyktech.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10315?size=medium"
/>
        Story
      </td>
    </tr>
    <tr>
      <th>Status</th>
      <td>In Dev</td>
    </tr>
    <tr>
      <th>Points</th>
      <td>N/A</td>
    </tr>
    <tr>
      <th>Labels</th>
      <td>-</td>
    </tr>
  </table>
</details>
<!--
  do not remove this marker as it will break jira-lint's functionality.
  added_by_jira_lint
-->

---

https://tyktech.atlassian.net/browse/TT-13723

It seems some tests detect goroutine leaks now. The detected goroutines
leaked have been listed in the ignores of a debug2.Record; both
goroutine leak tests detect goroutines in background reliably. Both are
flaky otherwise, this passes a -count=100 run, with and without -race.


___

### **PR Type**
Enhancement, Tests, Configuration changes


___

### **Description**
- Introduced `debug2.Record` to enhance goroutine state tracking and
comparison.
- Improved goroutine leak detection tests using `debug2.Record`.
- Added unit and benchmark tests for `debug2.Record`.
- Updated CI workflows to use Go 1.23.x.
- Simplified Dockerfile by switching to Go 1.23-bullseye base image and
optimizing build steps.
- Updated plugin compiler and release workflows to support Go 1.23.
- Enhanced Taskfile to dynamically use the Go version from `go.mod`.
- Bumped Go version in `go.mod` to 1.23.4.



___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Tests</strong></td><td><details><summary>2
files</summary><table>
<tr>
  <td>
    <details>
<summary><strong>gateway_test.go</strong><dd><code>Improved goroutine
leak detection in tests.</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

gateway/gateway_test.go

<li>Enhanced goroutine leak tests with <code>debug2.Record</code> for
better <br>reliability.<br> <li> Introduced <code>newRecord</code>
helper function to manage ignored goroutines.<br> <li> Updated
assertions to use <code>debug2.Record</code> for goroutine count
<br>validation.<br>


</details>


  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6812/files#diff-d34c7069ce5e81d45082b19eb3e869ee1a086e185dcd6630e75e3ed0d368b546">+37/-15</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>goroutine_test.go</strong><dd><code>Added unit and
benchmark tests for `debug2.Record`.</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></summary>
<hr>

internal/debug2/goroutine_test.go

<li>Added unit tests for <code>debug2.Record</code> to validate
goroutine tracking.<br> <li> Included benchmark tests for performance
evaluation.<br> <li> Verified goroutine cleanup using <code>Since</code>
method.<br>


</details>


  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6812/files#diff-763a0643c49a4bbfa70fbf12088045a74be7fc5f4789d09f14a9298d0b65c226">+103/-0</a>&nbsp;
</td>

</tr>

</table></details></td></tr><tr><td><strong>Enhancement</strong></td><td><details><summary>4
files</summary><table>
<tr>
  <td>
    <details>
<summary><strong>goroutine.go</strong><dd><code>Introduced
`debug2.Record` for goroutine state tracking.</code>&nbsp;
</dd></summary>
<hr>

internal/debug2/goroutine.go

<li>Added <code>debug2.Record</code> to capture and compare goroutine
states.<br> <li> Implemented methods for parsing, counting, and
filtering goroutines.<br> <li> Introduced functionality to ignore
specific goroutines during <br>comparison.<br>


</details>


  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6812/files#diff-c6edbd39849f1acdcae221ef5e8b6a5886d644ec719064cf051be79f8c9377f8">+124/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>Dockerfile</strong><dd><code>Simplified Dockerfile with
Go 1.23 base image.</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

Dockerfile

<li>Simplified Dockerfile by using Go 1.23-bullseye base image.<br> <li>
Removed redundant Python installation steps.<br> <li> Optimized build
process with caching for Go modules.<br>


</details>


  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6812/files#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557">+11/-51</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>Taskfile.yml</strong><dd><code>Enhanced Taskfile to
support dynamic Go version.</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

Taskfile.yml

- Added dynamic Go version argument for Docker builds.



</details>


  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6812/files#diff-cd2d359855d0301ce190f1ec3b4c572ea690c83747f6df61c9340720e3d2425e">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>go.mod</strong><dd><code>Bumped Go version in module to
1.23.4.</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></summary>
<hr>

go.mod

- Updated Go version requirement to 1.23.4.



</details>


  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6812/files#diff-33ef32bf6c23acb95f5902d7097b7a1d5128ca061167ec0716715b0b9eeaa5f6">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Configuration
changes</strong></td><td><details><summary>4 files</summary><table>
<tr>
  <td>
    <details>
<summary><strong>ci-tests.yml</strong><dd><code>Updated CI workflow to
use Go 1.23.x.</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

.github/workflows/ci-tests.yml

- Updated Go version in CI matrix to 1.23.x.



</details>


  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6812/files#diff-03609cb60b0c6e92fb771eb8787d6722b8c31ca4c03eabc788e147acd8c6fb43">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>plugin-compiler-build.yml</strong><dd><code>Updated
plugin compiler workflow for Go 1.23.</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

.github/workflows/plugin-compiler-build.yml

<li>Updated base image to use Go 1.23-bullseye.<br> <li> Fixed
<code>BASE_IMAGE</code> argument in Docker build steps.<br>


</details>


  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6812/files#diff-f3a95a900eb0ac23af6314e9cdea29fa16af0a9bcb61793a83a32ff13d4c4e79">+3/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>release.yml</strong><dd><code>Updated release workflow
to support Go 1.23.</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

.github/workflows/release.yml

<li>Updated Go version in release workflow to 1.23-bullseye.<br> <li>
Adjusted Docker build conditions for the new Go version.<br>


</details>


  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6812/files#diff-87db21a973eed4fef5f32b267aa60fcee5cbdf03c67fafdc2a9b553bb0b15f34">+11/-11</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>Dockerfile</strong><dd><code>Updated plugin compiler
Dockerfile for Go 1.23.</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

ci/images/plugin-compiler/Dockerfile

- Updated base image to Go 1.23-bullseye.



</details>


  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6812/files#diff-0ded1ed63ca128bd2d22721b0bc19dc85e440e4922164f465ac647917321971e">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr></tr></tbody></table>

___

> 💡 **PR-Agent usage**: Comment `/help "your question"` on any pull
request to receive relevant information

---------

Co-authored-by: Tit Petric <[email protected]>
  • Loading branch information
titpetric and Tit Petric authored Jan 9, 2025
1 parent 4304645 commit daf76a8
Show file tree
Hide file tree
Showing 14 changed files with 360 additions and 127 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/ci-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ jobs:
fail-fast: false
matrix:
redis-version: [7]
go-version: [1.22.x]
go-version: [1.23.x]

env:
REDIS_IMAGE: redis:${{ matrix.redis-version }}
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/plugin-compiler-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ on:
- "v*"

env:
GOLANG_CROSS: 1.22-bullseye
GOLANG_CROSS: 1.23-bullseye

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
Expand Down Expand Up @@ -77,7 +77,7 @@ jobs:
labels: ${{ steps.set-metadata.outputs.labels }}
tags: ${{ steps.set-metadata.outputs.tags }}
build-args: |
BASE-IMAGE=tykio/golang-cross:${{ env.GOLANG_CROSS }}
BASE_IMAGE=tykio/golang-cross:${{ env.GOLANG_CROSS }}
GITHUB_SHA=${{ github.sha }}
GITHUB_TAG=${{ github.ref_name }}
Expand Down Expand Up @@ -108,7 +108,7 @@ jobs:
labels: ${{ steps.set-metadata-ee.outputs.labels }}
tags: ${{ steps.set-metadata-ee.outputs.tags }}
build-args: |
BASE-IMAGE=tykio/golang-cross:${{ env.GOLANG_CROSS }}
BASE_IMAGE=tykio/golang-cross:${{ env.GOLANG_CROSS }}
GITHUB_SHA=${{ github.sha }}
GITHUB_TAG=${{ github.ref_name }}
BUILD_TAG=ee
22 changes: 11 additions & 11 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,9 @@ jobs:
fail-fast: false
matrix:
golang_cross:
- 1.22-bullseye
- 1.23-bullseye
include:
- golang_cross: 1.22-bullseye
- golang_cross: 1.23-bullseye
goreleaser: 'ci/goreleaser/goreleaser.yml'
cgo: 1
rpmvers: 'el/7 el/8 el/9 amazon/2 amazon/2023'
Expand Down Expand Up @@ -127,12 +127,12 @@ jobs:
mask-aws-account-id: false
- uses: aws-actions/amazon-ecr-login@v2
id: ecr
if: ${{ matrix.golang_cross == '1.22-bullseye' }}
if: ${{ matrix.golang_cross == '1.23-bullseye' }}
with:
mask-password: 'true'
- name: Docker metadata for CI
id: ci_metadata_
if: ${{ matrix.golang_cross == '1.22-bullseye' }}
if: ${{ matrix.golang_cross == '1.23-bullseye' }}
uses: docker/metadata-action@v5
with:
images: ${{ steps.ecr.outputs.registry }}/tyk
Expand All @@ -146,7 +146,7 @@ jobs:
type=semver,pattern={{major}}.{{minor}},prefix=v
type=semver,pattern={{version}},prefix=v
- name: push image to CI
if: ${{ matrix.golang_cross == '1.22-bullseye' }}
if: ${{ matrix.golang_cross == '1.23-bullseye' }}
uses: docker/build-push-action@v6
with:
context: "dist"
Expand All @@ -163,7 +163,7 @@ jobs:
EDITION=
- name: Docker metadata for CI ee
id: ci_metadata_ee
if: ${{ matrix.golang_cross == '1.22-bullseye' }}
if: ${{ matrix.golang_cross == '1.23-bullseye' }}
uses: docker/metadata-action@v5
with:
images: ${{ steps.ecr.outputs.registry }}/tyk-ee
Expand All @@ -177,7 +177,7 @@ jobs:
type=semver,pattern={{major}}.{{minor}},prefix=v
type=semver,pattern={{version}},prefix=v
- name: push image to CI ee
if: ${{ matrix.golang_cross == '1.22-bullseye' }}
if: ${{ matrix.golang_cross == '1.23-bullseye' }}
uses: docker/build-push-action@v6
with:
context: "dist"
Expand Down Expand Up @@ -207,7 +207,7 @@ jobs:
type=semver,pattern={{version}}
labels: "org.opencontainers.image.title=tyk-gateway (distroless) \norg.opencontainers.image.description=Tyk Open Source API Gateway written in Go, supporting REST, GraphQL, TCP and gRPC protocols\norg.opencontainers.image.vendor=tyk.io\norg.opencontainers.image.version=${{ github.ref_name }}\n"
- name: push image to prod
if: ${{ matrix.golang_cross == '1.22-bullseye' }}
if: ${{ matrix.golang_cross == '1.23-bullseye' }}
uses: docker/build-push-action@v6
with:
context: "dist"
Expand Down Expand Up @@ -236,7 +236,7 @@ jobs:
type=semver,pattern={{version}}
labels: "org.opencontainers.image.title=tyk-gateway Enterprise Edition (distroless) \norg.opencontainers.image.description=Tyk Open Source API Gateway written in Go, supporting REST, GraphQL, TCP and gRPC protocols\norg.opencontainers.image.vendor=tyk.io\norg.opencontainers.image.version=${{ github.ref_name }}\n"
- name: push image to prod ee
if: ${{ matrix.golang_cross == '1.22-bullseye' }}
if: ${{ matrix.golang_cross == '1.23-bullseye' }}
uses: docker/build-push-action@v6
with:
context: "dist"
Expand All @@ -253,7 +253,7 @@ jobs:
EDITION=-ee
- name: save deb
uses: actions/upload-artifact@v4
if: ${{ matrix.golang_cross == '1.22-bullseye' }}
if: ${{ matrix.golang_cross == '1.23-bullseye' }}
with:
name: deb
retention-days: 1
Expand All @@ -263,7 +263,7 @@ jobs:
!dist/*fips*.deb
- name: save rpm
uses: actions/upload-artifact@v4
if: ${{ matrix.golang_cross == '1.22-bullseye' }}
if: ${{ matrix.golang_cross == '1.23-bullseye' }}
with:
name: rpm
retention-days: 1
Expand Down
File renamed without changes.
2 changes: 1 addition & 1 deletion .taskfiles/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
version: "3"

includes:
deps: ./deps/Taskfile.yml
deps: ./deps.yml
services:
taskfile: ../docker/services/Taskfile.yml
dir: ../docker/services
Expand Down
62 changes: 11 additions & 51 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,68 +1,28 @@
FROM debian:bookworm AS assets

# This Dockerfile facilitates bleeding edge development docker image builds
# directly from source. To build a development image, run `make docker`.
# If you need to tweak the environment for testing, you can override the
# `GO_VERSION` and `PYTHON_VERSION` as docker build arguments.

ARG GO_VERSION=1.22.6
ARG PYTHON_VERSION=3.11.6

WORKDIR /assets

RUN apt update && apt install wget -y && \
wget -q https://dl.google.com/go/go${GO_VERSION}.linux-amd64.tar.gz && \
wget -q https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tar.xz

FROM debian:bookworm

ARG GO_VERSION=1.22.6
ARG PYTHON_VERSION=3.11.6

COPY --from=assets /assets/ /tmp/
WORKDIR /tmp

# Install Go

ENV PATH=$PATH:/usr/local/go/bin

RUN tar -C /usr/local -xzf go${GO_VERSION}.linux-amd64.tar.gz && \
go version
ARG GO_VERSION=1.23
FROM golang:${GO_VERSION}-bullseye

# Build essentials

RUN apt update && apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libsqlite3-dev libreadline-dev libffi-dev curl wget libbz2-dev -y

# Install $PYTHON_VERSION

## This just installs whatever is is bullseye, makes docker build (fast/small)-(er)
RUN apt install python3 -y

## This runs python code slower, but the process finishes quicker
# RUN tar -xf Python-${PYTHON_VERSION}.tar.xz && ls -la && \
# cd Python-${PYTHON_VERSION}/ && \
# ./configure --enable-shared && make build_all && \
# make altinstall && \
# ldconfig $PWD

## This runs python code faster, but is expensive to build and runs regression tests
# RUN tar -xf Python-${PYTHON_VERSION}.tar.xz && ls -la && \
# cd Python-${PYTHON_VERSION}/ && \
# ./configure --enable-shared --enable-optimizations && make -j 2 && \
# make altinstall && \
# ldconfig $PWD

# Clean up build assets
RUN find /tmp -type f -delete

# Build gateway
RUN mkdir /opt/tyk-gateway
WORKDIR /opt/tyk-gateway

ADD go.mod go.sum /opt/tyk-gateway/
RUN go mod download

RUN --mount=type=cache,mode=0755,target=/go/pkg/mod \
--mount=type=cache,mode=0755,target=/root/.cache/go-build \
go mod download

ADD . /opt/tyk-gateway

RUN make build
RUN --mount=type=cache,mode=0755,target=/go/pkg/mod \
--mount=type=cache,mode=0755,target=/root/.cache/go-build \
make build

COPY tyk.conf.example tyk.conf

Expand Down
4 changes: 2 additions & 2 deletions Taskfile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ version: "3"

includes:
test: .taskfiles/test.yml
deps: .taskfiles/deps/Taskfile.yml
deps: .taskfiles/deps.yml
hooks: .taskfiles/hooks.yml
opentelemetry:
taskfile: ci/tests/tracing/Taskfile.yml
Expand All @@ -25,7 +25,7 @@ tasks:
docker:
desc: "build Tyk gateway internal/tyk-gateway"
cmds:
- docker build --platform "linux/amd64" --rm -t internal/tyk-gateway .
- docker build --build-arg GO_VERSION="$(go mod edit -json | jq .Go -r)" --platform "linux/amd64" --rm -t internal/tyk-gateway .
sources:
- go.mod
- go.sum
Expand Down
2 changes: 1 addition & 1 deletion ci/images/plugin-compiler/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
ARG BASE_IMAGE=tykio/golang-cross:1.22-bullseye
ARG BASE_IMAGE=tykio/golang-cross:1.23-bullseye
FROM ${BASE_IMAGE}

LABEL description="Image for plugin development"
Expand Down
56 changes: 0 additions & 56 deletions gateway/gateway_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ import (
"net/http/httptest"
"net/url"
"os"
"runtime"
"strconv"
"strings"
"sync/atomic"
Expand Down Expand Up @@ -642,61 +641,6 @@ func TestListenPathTykPrefix(t *testing.T) {
})
}

func TestReloadGoroutineLeakWithTest(t *testing.T) {
test.Flaky(t)

before := runtime.NumGoroutine()

ts := StartTest(nil)
ts.Close()

time.Sleep(time.Second)

after := runtime.NumGoroutine()

if before < after {
t.Errorf("Goroutine leak, was: %d, after reload: %d", before, after)
}
}

func TestReloadGoroutineLeakWithCircuitBreaker(t *testing.T) {
ts := StartTest(nil)
t.Cleanup(ts.Close)

globalConf := ts.Gw.GetConfig()
globalConf.EnableJSVM = false
ts.Gw.SetConfig(globalConf)

specs := ts.Gw.BuildAndLoadAPI(func(spec *APISpec) {
spec.Proxy.ListenPath = "/"
UpdateAPIVersion(spec, "v1", func(version *apidef.VersionInfo) {
version.ExtendedPaths = apidef.ExtendedPathsSet{
CircuitBreaker: []apidef.CircuitBreakerMeta{
{
Path: "/",
Method: http.MethodGet,
ThresholdPercent: 0.5,
Samples: 5,
ReturnToServiceAfter: 10,
},
},
}
})
})

before := runtime.NumGoroutine()

ts.Gw.LoadAPI(specs...) // just doing globalGateway.DoReload() doesn't load anything as BuildAndLoadAPI cleans up folder with API specs

time.Sleep(100 * time.Millisecond)

after := runtime.NumGoroutine()

if before < after {
t.Errorf("Goroutine leak, was: %d, after reload: %d", before, after)
}
}

func listenProxyProto(ls net.Listener) error {
pl := &proxyproto.Listener{Listener: ls}
for {
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/TykTechnologies/tyk

go 1.22.6
go 1.23.4

require (
github.com/Jeffail/tunny v0.1.4
Expand Down
Loading

0 comments on commit daf76a8

Please sign in to comment.