wasi stdio is blocking

[shynome]

Describe the bug

goroutines is not running

more details: https://github.com/shynome/wazero-yamux-stdio-test

add log at yamux/session.go#L636, the log in goroutines is not running in wasm

	if flags&flagSYN == flagSYN {
		log.Println("handle ping")
		go func() {
			log.Println("handle ping and send")
			hdr := header(make([]byte, headerSize))
			hdr.encode(typePing, flagACK, 0, pingID)
			if err := s.sendNoWait(hdr); err != nil {
				s.logger.Printf("[WARN] yamux: failed to send ping reply: %v", err)
			}
		}()
		return nil
	}

To Reproduce

git clone --recurse-submodules https://github.com/shynome/wazero-yamux-stdio-test.git
cd wazero-yamux-stdio-test
# it works by system call
go run . -sys
# it is not work by wasi call
go run .

#  wasip1 wasm call by wasmer also have the same problem, it seem golang wasm problem
go run . -wasmer

Expected behavior

work the same as system call

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the relevant information):

• Go version: 1.20.2
• wazero Version: v1.2.1
• Host architecture: debian12 amd64
• Runtime mode: interpreter

Additional context

maybe this is a golang wasm problem, but I can't verify

[shynome]

Maybe golang wasip1 wasm need some additional config, I have a try with go_js_wasm_exec, that is working

gojsExec := fmt.Sprintf("%s/misc/wasm/go_js_wasm_exec", runtime.GOROOT())
cmd := exec.Command("go", "run", "-exec", gojsExec, "./w")
cmd.Env = append(os.Environ(), "GOOS=js", "GOARCH=wasm")
cmd.Stdin = cmdIn
cmd.Stdout = cmdOut
cmd.Stderr = os.Stderr
try.To(cmd.Start())
defer cmd.Wait()

[achille-roussel]

Hello @shynome, thanks for reporting!

I believe this might be due to the stdio streams being in blocking mode.

Could you try this patch and report whether it helps address the isssue?
https://go-review.googlesource.com/c/go/+/498196 (you will need to recompile Go from sources, see https://go.dev/doc/contribute for details).

[shynome]

Hello @shynome, thanks for reporting!

I believe this might be due to the stdio streams being in blocking mode.

Could you try this patch and report whether it helps address the isssue? https://go-review.googlesource.com/c/go/+/498196 (you will need to recompile Go from sources, see https://go.dev/doc/contribute for details).

thanks your reply! but this patch is not working, still has the problem hang up

[achille-roussel]

I have tried your example and I do think this is a case that would be solved by having the stdio streams set to non-blocking.

If I use the wasirun utility from wasi-go, it supports configuring non-blocking stdio, and the example you provided seems to behave as expected:

$ go run . -wasirun
2023/06/21 11:08:57 ping
2023/06/21 11:08:57 read
2023/06/21 11:08:57 write [0 2 0 1 0 0 0 0 0 0 0 0]
2023/06/21 11:08:57 wrote 12 <nil>
2023/06/21 18:08:57 wasm read
2023/06/21 18:08:57 wasm readed 12 <nil> [0 2 0 1 0 0 0 0 0 0 0 0]
2023/06/21 18:08:57 wasm read
2023/06/21 18:08:57 wasm write [0 2 0 2 0 0 0 0 0 0 0 0]
2023/06/21 18:08:57 wasm wrote 12 <nil>
2023/06/21 11:08:57 readed 12 <nil> [0 2 0 2 0 0 0 0 0 0 0 0]
2023/06/21 11:08:57 read
2023/06/21 11:08:57 pong
2023/06/21 11:08:57 readed 0 io: read/write on closed pipe []
2023/06/21 18:08:57 wasm readed 0 EOF []

This is the diff I used on your program:

$ git diff
diff --git a/main.go b/main.go
index 6baf1e4..ed77085 100644
--- a/main.go
+++ b/main.go
@@ -20,6 +20,8 @@ import (
 func main() {
        var callSystemExec bool
        flag.BoolVar(&callSystemExec, "sys", false, "use system exec")
+       var useWasirun bool
+       flag.BoolVar(&useWasirun, "wasirun", false, "use wasirun call")
        var useWasmer bool
        flag.BoolVar(&useWasmer, "wasmer", false, "use wasm3 call")
        var useGoJS bool
@@ -41,6 +43,19 @@ func main() {
                cmd.Stderr = os.Stderr
                try.To(cmd.Start())
                defer cmd.Wait()
+       } else if useWasirun {
+               // build wasip1
+               build := exec.Command("gotip", "build", "-o", "w.wasm", "./w")
+               build.Env = append(os.Environ(), "GOOS=wasip1", "GOARCH=wasm")
+               build.Stderr = os.Stderr
+               try.To(build.Run())
+
+               cmd := exec.Command("wasirun", "--non-blocking-stdio", "--", "w.wasm")
+               cmd.Stdin = cmdIn
+               cmd.Stdout = cmdOut
+               cmd.Stderr = os.Stderr
+               try.To(cmd.Start())
+               defer cmd.Wait()
        } else if useWasmer {
                // build wasip1
                build := exec.Command("gotip", "build", "-o", "w.wasm", "./w")

Now if I remove --non-blocking-stdio, the code blocks and doesn't handle the pings.

I also tried the patch that I pointed out to you and it seems to also address the issue, this is the diff I used:

$ git diff
diff --git a/main.go b/main.go
index 6baf1e4..e835246 100644
--- a/main.go
+++ b/main.go
@@ -20,6 +20,8 @@ import (
 func main() {
        var callSystemExec bool
        flag.BoolVar(&callSystemExec, "sys", false, "use system exec")
+       var useWasirun bool
+       flag.BoolVar(&useWasirun, "wasirun", false, "use wasirun call")
        var useWasmer bool
        flag.BoolVar(&useWasmer, "wasmer", false, "use wasm3 call")
        var useGoJS bool
@@ -41,6 +43,19 @@ func main() {
                cmd.Stderr = os.Stderr
                try.To(cmd.Start())
                defer cmd.Wait()
+       } else if useWasirun {
+               // build wasip1
+               build := exec.Command("../../../go.googlesource.com/go/bin/go", "build", "-o", "w.wasm", "./w")
+               build.Env = append(os.Environ(), "GOROOT=../../../go.googlesource.com/go", "GOOS=wasip1", "GOARCH=wasm")
+               build.Stderr = os.Stderr
+               try.To(build.Run())
+
+               cmd := exec.Command("wasirun", "--", "w.wasm")
+               cmd.Stdin = cmdIn
+               cmd.Stdout = cmdOut
+               cmd.Stderr = os.Stderr
+               try.To(cmd.Start())
+               defer cmd.Wait()
        } else if useWasmer {
                // build wasip1
                build := exec.Command("gotip", "build", "-o", "w.wasm", "./w")

Let me know if you have any trouble reproducing the findings!

[shynome]

ohhhh, it is working! Thanks for your help.

It seems wazero also need a --non-blocking-stdio option or default enable the option?

or leave it to third-party library like wasi-go?

[evacchi]

nonblocking stdio is definitely something I should circle back to 😬 #1500
EDIT: oh meanwhile, by all means, do use wasi-go if that suits your needs!

[chriso]

Related: #1538

[evacchi]

I worked at #1538 with #1542 but that didn't fix this 🤔

[evacchi]

so I think this actually works on main, but you need to start wazero the same way you started wasirun, i.e. as a process:

		cmd := exec.Command("wazero", "run", "w.wasm")
		cmd.Stdin = cmdIn
		cmd.Stdout = cmdOut
		cmd.Stderr = os.Stderr
		try.To(cmd.Start())
		defer cmd.Wait()

the reason is that if you instantiate wazero as a library and you pass in the pipe, then we cannot treat it as an OS file descriptor (to us it will be an opaque io.Reader or io.Writer), and thus we cannot set nonblocking mode.

Let me know if this works.

[shynome]

it is working!
why it is not working, beacuse it run gojs default, when I change it to wasi_snapshot_preview1 and set stdio as *os.File it is working now

	cmdIn, cmdWriter := try.To2(os.Pipe())
	cmdReader, cmdOut := try.To2(os.Pipe())
...
	mc := wazero.NewModuleConfig().
		WithArgs("wazero").
		WithRandSource(rand.Reader).
		WithSysNanosleep().
		WithSysNanotime().
		WithSysWalltime().
		WithStdin(cmdIn).
		WithStdout(cmdOut).
		WithStderr(os.Stderr)

Thanks for all people work!

[evacchi]

awesome! ah yes, the trick is you're using os.Pipe() instead of an io.Pipe() :)