Upgrade artefactual-sdps/temporal-activities #1011

jraddaoui · 2024-08-26T19:39:44Z

Upgrade github.com/artefactual-sdps/temporal-activities after the
re-org, which also upgrades google.golang.org/grpc and other deps.
Ignore grpc.DialContext and grpc.WithBlock deprecated warning.

codecov · 2024-08-26T19:43:02Z

Codecov Report

Attention: Patch coverage is 41.17647% with 20 lines in your changes missing coverage. Please review.

Project coverage is 53.06%. Comparing base (e829a58) to head (9b674ba).

Files	Patch %	Lines
cmd/enduro-am-worker/main.go	0.00%	10 Missing ⚠️
cmd/enduro-a3m-worker/main.go	0.00%	6 Missing ⚠️
internal/telemetry/traces.go	0.00%	2 Missing ⚠️
internal/a3m/client.go	0.00%	1 Missing ⚠️
internal/workflow/processing.go	93.33%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1011      +/-   ##
==========================================
- Coverage   53.19%   53.06%   -0.14%     
==========================================
  Files         102      101       -1     
  Lines        5856     5814      -42     
==========================================
- Hits         3115     3085      -30     
+ Misses       2483     2476       -7     
+ Partials      258      253       -5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

jraddaoui · 2024-08-26T23:31:47Z

Upgrading github.com/artefactual-sdps/temporal-activities, several dependencies also got updated:

$ go get github.com/artefactual-sdps/temporal-activities
go: upgraded github.com/artefactual-labs/bagit-gython v0.0.0-20240723112522-d8d3815a59e6 => v0.2.0
go: upgraded github.com/artefactual-sdps/temporal-activities v0.0.0-20240722184934-3e9ad5a6dcda => v0.0.0-20240821162351-47302711bc7b
go: upgraded github.com/aws/aws-sdk-go v1.50.36 => v1.55.5
go: upgraded github.com/aws/aws-sdk-go-v2 v1.26.1 => v1.30.3
go: upgraded github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.2 => v1.6.3
go: upgraded github.com/aws/aws-sdk-go-v2/config v1.27.11 => v1.27.27
go: upgraded github.com/aws/aws-sdk-go-v2/credentials v1.17.11 => v1.17.27
go: upgraded github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.1 => v1.16.11
go: upgraded github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.16.9 => v1.17.10
go: upgraded github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.5 => v1.3.15
go: upgraded github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.5 => v2.6.15
go: upgraded github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.5 => v1.3.15
go: upgraded github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.2 => v1.11.3
go: upgraded github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.3.7 => v1.3.17
go: upgraded github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.7 => v1.11.17
go: upgraded github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.17.5 => v1.17.15
go: upgraded github.com/aws/aws-sdk-go-v2/service/s3 v1.53.1 => v1.58.3
go: upgraded github.com/aws/aws-sdk-go-v2/service/sso v1.20.5 => v1.22.4
go: upgraded github.com/aws/aws-sdk-go-v2/service/ssooidc v1.23.4 => v1.26.4
go: upgraded github.com/aws/aws-sdk-go-v2/service/sts v1.28.6 => v1.30.3
go: upgraded github.com/aws/smithy-go v1.20.2 => v1.20.3
go: upgraded github.com/cespare/xxhash/v2 v2.2.0 => v2.3.0
go: upgraded github.com/go-logr/logr v1.4.1 => v1.4.2
go: upgraded github.com/go-sql-driver/mysql v1.8.0 => v1.8.1
go: upgraded github.com/googleapis/gax-go/v2 v2.12.2 => v2.13.0
go: upgraded go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.49.0 => v0.53.0
go: upgraded go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 => v0.53.0
go: upgraded go.opentelemetry.io/otel v1.24.0 => v1.28.0
go: upgraded go.opentelemetry.io/otel/metric v1.24.0 => v1.28.0
go: upgraded go.opentelemetry.io/otel/trace v1.24.0 => v1.28.0
go: upgraded gocloud.dev v0.37.0 => v0.39.0
go: upgraded golang.org/x/crypto v0.22.0 => v0.26.0
go: upgraded golang.org/x/mod v0.16.0 => v0.17.0
go: upgraded golang.org/x/net v0.24.0 => v0.28.0
go: upgraded golang.org/x/oauth2 v0.18.0 => v0.22.0
go: upgraded golang.org/x/sync v0.7.0 => v0.8.0
go: upgraded golang.org/x/sys v0.22.0 => v0.24.0
go: upgraded golang.org/x/text v0.14.0 => v0.17.0
go: upgraded golang.org/x/time v0.5.0 => v0.6.0
go: upgraded golang.org/x/tools v0.19.0 => v0.21.1-0.20240508182429-e35e4ccd0d2d
go: upgraded golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 => v0.0.0-20240716161551-93cc26a95ae9
go: upgraded google.golang.org/api v0.169.0 => v0.191.0
go: upgraded google.golang.org/genproto/googleapis/api v0.0.0-20240401170217-c3f982113cda => v0.0.0-20240812133136-8ffd90a71988
go: upgraded google.golang.org/genproto/googleapis/rpc v0.0.0-20240401170217-c3f982113cda => v0.0.0-20240812133136-8ffd90a71988
go: upgraded google.golang.org/grpc v1.63.2 => v1.65.0
go: upgraded google.golang.org/protobuf v1.33.0 => v1.34.2

That broke the build process:

#16 [build-enduro 1/1] RUN --mount=type=cache,target=/go/pkg/mod 	--mount=type=cache,target=/root/.cache/go-build 	go build 	-trimpath 	-ldflags="-X 'github.com/artefactual-sdps/enduro/internal/version.Long=0.1.0-te829a5822' -X 'github.com/artefactual-sdps/enduro/internal/version.Short=0.1.0' -X 'github.com/artefactual-sdps/enduro/internal/version.GitCommit=e829a5822546e514a749a09717aef11655fc2051-dirty'" 	-o /out/enduro 	./cmd/enduro
#16 sha256:00c06629b76450c9543845480c891e52ce505432519655093c1e06c65e14e8f8
#16 1.009 # go.opentelemetry.io/otel/sdk/trace
#16 1.009 /go/pkg/mod/go.opentelemetry.io/otel/[email protected]/trace/span.go:165:20: cannot use (*recordingSpan)(nil) (value of type *recordingSpan) as ReadWriteSpan value in variable declaration: *recordingSpan does not implement ReadWriteSpan (missing method AddLink)
#16 1.009 		have addLink("go.opentelemetry.io/otel/trace".Link)
#16 1.009 		want AddLink("go.opentelemetry.io/otel/trace".Link)
#16 1.009 /go/pkg/mod/go.opentelemetry.io/otel/[email protected]/trace/span.go:804:20: cannot use nonRecordingSpan{} (value of type nonRecordingSpan) as "go.opentelemetry.io/otel/trace".Span value in variable declaration: nonRecordingSpan does not implement "go.opentelemetry.io/otel/trace".Span (missing method AddLink)
#16 1.009 /go/pkg/mod/go.opentelemetry.io/otel/[email protected]/trace/tracer.go:50:21: impossible type assertion: p.(*recordingSpan)
#16 1.009 	*recordingSpan does not implement "go.opentelemetry.io/otel/trace".Span (missing method AddLink)
#16 1.009 		have addLink("go.opentelemetry.io/otel/trace".Link)
#16 1.009 		want AddLink("go.opentelemetry.io/otel/trace".Link)
#16 1.009 /go/pkg/mod/go.opentelemetry.io/otel/[email protected]/trace/tracer.go:120:10: cannot use tr.newNonRecordingSpan(sc) (value of type nonRecordingSpan) as "go.opentelemetry.io/otel/trace".Span value in return statement: nonRecordingSpan does not implement "go.opentelemetry.io/otel/trace".Span (missing method AddLink)
#16 1.009 /go/pkg/mod/go.opentelemetry.io/otel/[email protected]/trace/tracer.go:122:9: cannot use tr.newRecordingSpan(psc, sc, name, samplingResult, config) (value of type *recordingSpan) as "go.opentelemetry.io/otel/trace".Span value in return statement: *recordingSpan does not implement "go.opentelemetry.io/otel/trace".Span (missing method AddLink)
#16 1.009 		have addLink("go.opentelemetry.io/otel/trace".Link)
#16 1.009 		want AddLink("go.opentelemetry.io/otel/trace".Link)
#16 ERROR: executor failed running [/bin/sh -c go build 	-trimpath 	-ldflags="-X '${VERSION_PATH}.Long=${VERSION_LONG}' -X '${VERSION_PATH}.Short=${VERSION_SHORT}' -X '${VERSION_PATH}.GitCommit=${VERSION_GIT_HASH}'" 	-o /out/enduro 	./cmd/enduro]: exit code: 1

Which I could fix upgrading all the other go.opentelemetry.io dependencies:

$ go get go.opentelemetry.io/otel/sdk
go: upgraded go.opentelemetry.io/otel v1.28.0 => v1.29.0
go: upgraded go.opentelemetry.io/otel/metric v1.28.0 => v1.29.0
go: upgraded go.opentelemetry.io/otel/sdk v1.24.0 => v1.29.0
go: upgraded go.opentelemetry.io/otel/trace v1.28.0 => v1.29.0

$ go get go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
go: upgraded github.com/cenkalti/backoff/v4 v4.2.1 => v4.3.0
go: upgraded github.com/grpc-ecosystem/grpc-gateway/v2 v2.19.1 => v2.22.0
go: upgraded go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.24.0 => v1.29.0
go: upgraded go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.24.0 => v1.29.0
go: upgraded go.opentelemetry.io/proto/otlp v1.1.0 => v1.3.1
go: upgraded google.golang.org/genproto/googleapis/api v0.0.0-20240812133136-8ffd90a71988 => v0.0.0-20240822170219-fc7c04adadcd
go: upgraded google.golang.org/genproto/googleapis/rpc v0.0.0-20240812133136-8ffd90a71988 => v0.0.0-20240822170219-fc7c04adadcd

$ go get go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace
go: upgraded go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.49.0 => v0.54.0
go: upgraded go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.53.0 => v0.54.0

$ go get go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc
go: upgraded go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.53.0 => v0.54.0

Now, running golangci-lint raises a few deprecation warnings, related to the google.golang.org/grpc upgrade:

$ make lint
internal/telemetry/traces.go:36:15: SA1019: grpc.DialContext is deprecated: use NewClient instead.  Will be supported throughout 1.x. (staticcheck)
	conn, err := grpc.DialContext(
	             ^
internal/telemetry/traces.go:40:3: SA1019: grpc.WithBlock is deprecated: this DialOption is not supported by NewClient. Will be supported throughout 1.x. (staticcheck)
		grpc.WithBlock(),
		^
internal/a3m/client.go:21:15: SA1019: grpc.DialContext is deprecated: use NewClient instead.  Will be supported throughout 1.x. (staticcheck)
	conn, err := grpc.DialContext(
	             ^
make: *** [Makefile:179: lint] Error 1

I have changed the grpc.DialContext calls to use grpc.NewClient instead, but that doesn't accept a context or the WithBlock option. Some notes about those deprecation can be found in:

jraddaoui · 2024-08-27T00:08:18Z

internal/a3m/client.go

 	c := &Client{}

-	conn, err := grpc.DialContext(
-		ctx,
+	conn, err := grpc.NewClient(


Would this be enough? Should we check the connection? Reading the links in the previous update it seems unnecessary. And, should we close the connection after all?

@jraddaoui it looks like defer conn.Close() is expected:
https://github.com/grpc/grpc-go/blob/master/examples/helloworld/greeter_client/main.go#L49

But other than that I think your change above is sufficient.

jraddaoui · 2024-08-27T00:08:37Z

internal/telemetry/traces.go

+	conn, err := grpc.NewClient(
 		cfg.Traces.Address,
 		grpc.WithTransportCredentials(insecure.NewCredentials()),
-		grpc.WithBlock(),
 	)


It sounds to me like grpc.WithBlock() is unnecessary based on https://github.com/grpc/grpc-go/blob/master/Documentation/anti-patterns.md#especially-bad-using-deprecated-dialoptions. It sounds like any following RPCs will wait for the connection before proceeding.

mcantelon · 2024-08-27T17:56:36Z

Nice! I'm in the process of trying it out! Did a go get -u ./... and go mod tidy and seemed to upgrade the go modules okay.

jraddaoui · 2024-08-27T18:05:16Z

Check https://enduro.readthedocs.io/dev-manual/deps/, @sevein suggested to avoid that.

mcantelon

Looks good to me (and upgraded/ran with no issue)!

jraddaoui · 2024-08-27T18:44:58Z

Thanks Mike! For the google.golang.org/grpc upgrade, I think I'll ignore the deprecation warning for now and create a new issue where we can discuss the changes better.

Upgrade `github.com/artefactual-sdps/temporal-activities` after the re-org, which also upgrades `google.golang.org/grpc` and other deps. Ignore `grpc.DialContext` and `grpc.WithBlock` deprecated warning. [skip-codecov]

[skip-codecov]

jraddaoui self-assigned this Aug 26, 2024

jraddaoui force-pushed the dev/update-temporal-activities branch from 89d12f7 to a929608 Compare August 26, 2024 23:30

jraddaoui changed the title ~~Bump Go version and upgrade temporal-activities~~ Upgrade artefactual-sdps/temporal-activities Aug 26, 2024

jraddaoui requested review from mcantelon and sevein August 26, 2024 23:32

jraddaoui force-pushed the dev/update-temporal-activities branch 2 times, most recently from a9ec983 to e9856be Compare August 26, 2024 23:52

jraddaoui commented Aug 27, 2024

View reviewed changes

mcantelon approved these changes Aug 27, 2024

View reviewed changes

jraddaoui force-pushed the dev/update-temporal-activities branch 2 times, most recently from 9b674ba to d54fcc4 Compare August 27, 2024 19:13

jraddaoui mentioned this pull request Aug 27, 2024

Problem: grpc.DialContext and grpc.WithBlock functions are deprecated #1012

Open

jraddaoui added 2 commits August 27, 2024 21:29

Upgrade artefactual-sdps/temporal-activities

3a873ab

Upgrade `github.com/artefactual-sdps/temporal-activities` after the re-org, which also upgrades `google.golang.org/grpc` and other deps. Ignore `grpc.DialContext` and `grpc.WithBlock` deprecated warning. [skip-codecov]

Use temporal-activities/archivezip

5ec88fe

[skip-codecov]

jraddaoui force-pushed the dev/update-temporal-activities branch from d54fcc4 to 5ec88fe Compare August 27, 2024 19:29

jraddaoui merged commit 5ec88fe into main Aug 27, 2024
13 checks passed

jraddaoui deleted the dev/update-temporal-activities branch August 27, 2024 19:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade artefactual-sdps/temporal-activities #1011

Upgrade artefactual-sdps/temporal-activities #1011

jraddaoui commented Aug 26, 2024 •

edited

Loading

codecov bot commented Aug 26, 2024 •

edited

Loading

jraddaoui commented Aug 26, 2024

jraddaoui Aug 27, 2024

djjuhasz Sep 4, 2024

jraddaoui Aug 27, 2024

djjuhasz Sep 4, 2024

mcantelon commented Aug 27, 2024

jraddaoui commented Aug 27, 2024

mcantelon left a comment

jraddaoui commented Aug 27, 2024

Upgrade artefactual-sdps/temporal-activities #1011

Upgrade artefactual-sdps/temporal-activities #1011

Conversation

jraddaoui commented Aug 26, 2024 • edited Loading

codecov bot commented Aug 26, 2024 • edited Loading

Codecov Report

jraddaoui commented Aug 26, 2024

jraddaoui Aug 27, 2024

Choose a reason for hiding this comment

djjuhasz Sep 4, 2024

Choose a reason for hiding this comment

jraddaoui Aug 27, 2024

Choose a reason for hiding this comment

djjuhasz Sep 4, 2024

Choose a reason for hiding this comment

mcantelon commented Aug 27, 2024

jraddaoui commented Aug 27, 2024

mcantelon left a comment

Choose a reason for hiding this comment

jraddaoui commented Aug 27, 2024

jraddaoui commented Aug 26, 2024 •

edited

Loading

codecov bot commented Aug 26, 2024 •

edited

Loading