Thread creation overhead due to cgo calls

[NeetishPathak]

The connections in Spacemonkey openssl wrapper makes use of cgo calls and this is an issue because it leads to massive number of threads for simultaneous ssl connections. cgo marks all C code as syscall causing scheduler to allocate a new thread if needed. Can this be addressed ?

[zeebo]

The library tries to do a good job about only doing short cgo calls: they should all return quickly.

Do you have a short example demonstrating the problem?

[pvoicu]

Indeed the cgo calls are short because they are not waiting for IO, however internally functions like SSL_do_handshake, SSL_read, SSL_write are making a lot of mutex lock()/unlock() calls. I put a trace in https://github.com/spacemonkeygo/openssl/blob/master/init_posix.go, I see hundreds of calls. This is source of contention and for higher load (2000 handshakes per second on my dev machine) I see a spike in the number of threads created by the runtime and a performance degradation.
Do you have a suggestion for the contention caused by the openssl mutexes? Should we compile openssl with no threads support since all the calls from spacemonkeygo into openssl are protected by mutexes anyway?

[slucx]

I have the same problem.

[slucx]

I think it is limit of go GPM