runtime: mysterious crashes and non-determinism with range over func

[seehuhn]

Go version

go version go1.23.0 darwin/arm64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/voss/Library/Caches/go-build'
GOENV='/Users/voss/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/voss/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/voss/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.23.0'
GODEBUG=''
GOTELEMETRY='on'
GOTELEMETRYDIR='/Users/voss/Library/Application Support/go/telemetry'
GCCGO='gccgo'
GOARM64='v8.0'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='1'
GOMOD='/Users/voss/project/pdf-issues/006-go-iter/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/var/folders/hn/tn2k45vx3pz4v24pbmf4zl040000gn/T/go-build3862330642=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

I tried to use the new range over func loops. Here is the code:

https://go.dev/play/p/rYiGIM4RaVe

This is a pared down version of a much longer program. The code here does no longer make much sense, but has no external dependencies and still shows the problem.

What did you see happen?

There are several problems with this code:

• On the playground, at https://go.dev/play/p/rYiGIM4RaVe, the code crashes with the message fatal error: concurrent map read and map write. Note that there is no explicit concurrency in this code. The crash goes away if the print statement on line 80 is enabled.
• On my own system (an M2 MacBook Air), the same code crashes with the following error message:

  unexpected fault address 0xf48d0c3a929e0
  fatal error: fault
  [signal SIGSEGV: segmentation violation code=0x2 addr=0xf48d0c3a929e0 pc=0x104d58a50]
  

  I attach the full stacktrace, in case it is useful: stacktrace.txt. When I enable the print statement on line 80 the problem goes away.
• While trying to debug the problem, similar code (which I didn't keep, sorry!) sometimes triggered the panic on line 95 with impossible key values for the map bestM.

What did you expect to see?

The code should not crash, and the absence/presence of the print statement should not change the behaviour of the program.

[gabyhelp]

Related Issues and Documentation

• sync: concurrent map iteration and map write on Map error #24112 (closed)
• cmd/compile: ICE compiling rangefunc for < 1.23 in combination with Cgo #67497 (closed)
• runtime?: "concurrent map read and write" crashes no longer put the 2 offending goroutines at the top of the output #63172 (closed)
• maps: segmentation violation in maps.Clone #69110 (closed)
• cmd/compile: Go1.22RC1 compiler crash when using GOEXPERIMENT=rangefunc #64930 (closed)
• cmd/compile: seg fault with generics, channels and closures #51355 (closed)
• golang/io: `io.Copy` will return an error that contain unexpected fault address when passing `io.MultiWriter` on darwin #60378 (closed)
• fmt: stack overflow with self-referenced map #53237 (closed)
• cmd/compile: SIGSEGV with 'go build' #50968 (closed)
• Heap corruption crash on darwin/arm64 #49865 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[prattmic]

Thanks for the report and reproducer!

This looks like some sort of memory corruption of the map. My initial thought is to wonder if the map is stack allocated and the coroutine handling is corrupting the stack/use-after-free.

cc @golang/runtime @dr2chase

[randall77]

Possibly related to #69434

[cherrymui]

This looks like an escape analysis bug? m := make(map[byte][]string) is stack allocated (confirmed with compiler's -m flag), whereas it is assigned to bestM which is a captured variable in the outer frame.

[ianlancetaylor]

@gopherbot Please open a backport to 1.23.

This appears to be a compiler bug related to the range-over-function code. The compiler is generating bad code--a variable that escapes is stack-allocated. There is no workaround (other than avoiding range-over-func).

[gopherbot]

Backport issue(s) opened: #69511 (for 1.23).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

[gopherbot]

Change https://go.dev/cl/614096 mentions this issue: cmd/compile: fix wrong esacpe analysis for rangefunc