roachtest: decommission/nodes=4/duration=1h0m0s failed

[cockroach-teamcity]

roachtest.decommission/nodes=4/duration=1h0m0s failed with artifacts on release-24.1 @ d1b7d6f20b0a9021dcaca312af8d794fa6392737:

VMs preempted during the test run: teamcity-18775136-1739082850-04-n4cpu4-0001 (us-east1-d)

**Other Failures:**
(test_runner.go:1968).func1: monitorForPreemptedVMs: Preempted VMs detected: [{projects/cockroach-ephemeral/zones/us-east1-d/instances/teamcity-18775136-1739082850-04-n4cpu4-0001 2025-02-09 07:12:52.736798 +0000 UTC}]
(test_runner.go:1257).runTest: test timed out (3h0m0s)
test artifacts and logs in: /artifacts/decommission/nodes=4/duration=1h0m0s/run_1

Parameters:

• arch=amd64
• cloud=gce
• coverageBuild=false
• cpu=4
• encrypted=false
• runtimeAssertionsBuild=false
• ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

• roachtest: decommission/nodes=4/duration=1h0m0s failed #139628 roachtest: decommission/nodes=4/duration=1h0m0s failed [C-bug C-test-failure O-roachtest O-robot T-kv branch-master]

/cc @cockroachdb/kv-triage

This test on roachdash | Improve this report!

Jira issue: CRDB-47374

[tbg]

Preempted VMs detected

@herkolategan shouldn't this have gotten routed to T-testeng? Maybe not on 24.1?

[herkolategan]

@tbg Yes, but I think we may need to backport #139075 @DarrylWong, thoughts?

[DarrylWong]

Yup, looks like this hit the hanging goroutine case where a timeout overrode the preemption. Will backport.

[herkolategan]

Thanks!