Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v0.2.0 Websocket hangs forever #69

Open
rsukhodolskyi opened this issue May 26, 2023 · 2 comments
Open

v0.2.0 Websocket hangs forever #69

rsukhodolskyi opened this issue May 26, 2023 · 2 comments

Comments

@rsukhodolskyi
Copy link

I assume network poller waits for ack message from client which dropped connection what leads to ws stream to be frozen. This bug is only reproducible on v0.2.0 previous version i've used didn't have this issue: v0.0.0-20200429184054-15c2290dcb37

goroutine 716698 [IO wait, 44 minutes]:
internal/poll.runtime_pollWait(0x7f9463665528, 0x77)
	/usr/local/go/src/runtime/netpoll.go:305 +0x89
internal/poll.(*pollDesc).wait(0xc01058b200?, 0xc02dcfc00a?, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32
internal/poll.(*pollDesc).waitWrite(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:93
internal/poll.(*FD).Write(0xc01058b200, {0xc02dcfc00a, 0xbfc, 0xff6})
	/usr/local/go/src/internal/poll/fd_unix.go:391 +0x2c5
net.(*netFD).Write(0xc01058b200, {0xc02dcfc00a?, 0xc0135d8e70?, 0x0?})
	/usr/local/go/src/net/fd_posix.go:96 +0x29
net.(*conn).Write(0xc022fabec0, {0xc02dcfc00a?, 0x1a387c0?, 0x0?})
	/usr/local/go/src/net/net.go:195 +0x45
github.com/gorilla/websocket.(*Conn).write(0xc011896420, 0x1, {0xc0135d8fd0?, 0x7ea729?, 0x0?}, {0xc02dcfc00a, 0xbfc, 0xff6}, {0x0, 0x0, ...})
	/go/pkg/mod/github.com/gorilla/[email protected]/conn.go:392 +0x1a4
github.com/gorilla/websocket.(*messageWriter).flushFrame(0xc00f33e7b0, 0x1, {0x0?, 0x0?, 0xc0073ad780?})
	/go/pkg/mod/github.com/gorilla/[email protected]/conn.go:621 +0x405
github.com/gorilla/websocket.(*messageWriter).Close(0xc0135d9018?)
	/go/pkg/mod/github.com/gorilla/[email protected]/conn.go:731 +0x45
github.com/gorilla/websocket.(*Conn).WriteJSON(0x7de638?, {0x1a387c0, 0xc00f34cb20})
	/go/pkg/mod/github.com/gorilla/[email protected]/json.go:29 +0xdf
github.com/sourcegraph/jsonrpc2/websocket.ObjectStream.WriteObject({0x7f9460fd7758?}, {0x1a387c0?, 0xc00f34cb20?})
	/go/pkg/mod/github.com/sourcegraph/[email protected]/websocket/stream.go:25 +0x25

Moreover, because of the shared lock in Close function c.sending.Lock() it's also impossible to drop such client connection:

goroutine 716799 [semacquire, 43 minutes]:
sync.runtime_SemacquireMutex(0xa?, 0x0?, 0x24?)
	/usr/local/go/src/runtime/sema.go:77 +0x25
sync.(*Mutex).lockSlow(0xc026d8a310)
	/usr/local/go/src/sync/mutex.go:171 +0x165
sync.(*Mutex).Lock(...)
	/usr/local/go/src/sync/mutex.go:90
github.com/sourcegraph/jsonrpc2.(*Conn).close(0xc026d8a2d0, {0x0?, 0x0})
	/go/pkg/mod/github.com/sourcegraph/[email protected]/conn.go:169 +0x7d
github.com/sourcegraph/jsonrpc2.(*Conn).Close(...)
	/go/pkg/mod/github.com/sourcegraph/[email protected]/conn.go:69
@keegancsmith
Copy link
Member

keegancsmith commented Jun 5, 2023

I assume network poller waits for ack message from client which dropped connection what leads to ws stream to be frozen.

I think you are right, but then this depends on how you configure the code here. IE jsonrpc2 just takes in whatever you give it as a ws.Conn. You need to make it error out when things go wrong. I'm not really sure why this would be different between the two hashes without diving deeper. A reproduction would help a lot here.

Moreover, because of the shared lock in Close function c.sending.Lock() it's also impossible to drop such client connection:

Ouch, nice find. This is indeed a bug since 15c2290. Will take a look.

@keegancsmith
Copy link
Member

@rsukhodolskyi latest commit fixes Close not blocking. Do you have a reproduction to help track down the dropped acks?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants