How to make sure we run the spacemonkey with non blocking BIO

[varunitgithub]

cgo marks all C calls as syscall causing scheduler to allocate a new thread everytime. This leads to scaling issues of applications, Is it possible to make the BIO as non blocking in which case most of the thread lock time will be highly optimized.

[zeebo]

It doesn't allocate a new thread every time: it manages a pool and will only allocate a thread if the pool is empty.

I believe it is already the case that all of the calls into C are not blocking. Can you provide an example of a blocking C call?

[pvoicu]

The potential blocking is caused by the contention on the map holding callback pointer parameters. https://github.com/spacemonkeygo/openssl/blob/master/bio.go#L54 . I believe spacemonkeygo uses the mapping defined in mapping.go to help in case the garbage collector moves the GO pointer, so the conversion from C int to the GO pointer is still valid.
This map protected by one global mutex is source of contention, and it matters for high load. I propose replacing with sync.Map or maybe some type of lock-free hash map. This is a simple change.
A more complex change would be to rewrite the BIO callbacks in C (from GO - go_read_bio_read, go_write_bio_ctrl, go_write_bio_write). Benefits are:

• context switch from C to GO is avoided. The saving is not only performance but avoidance of LockOSThread() at https://github.com/golang/go/blob/master/src/runtime/cgocall.go#L179
• mapping.go is not needed anymore