example: fast, multi-threaded, non-blocking, port and host reuse, thread-safe, epoll server

[enghitalo]

Benchmark

v wipe-cache && v -prod -skip-unused crun .

Running 30s test @ http://localhost:3000/
  16 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.47ms    1.19ms  22.48ms   78.40%
    Req/Sec    23.05k     2.02k   53.61k    81.71%
  11017868 requests in 30.10s, 0.97GB read
Requests/sec: 366036.54
Transfer/sec:     33.16MB

with pontencial to be faster

_{Huly®: V_0.6-21532}

[spytheman]

Please use something like examples/vanilla_http_server/, not examples/server/ .

Just server is too generic, and it does not make it clear, that it is an http server.

[spytheman]

Also add the new folder name in the efolders list in cmd/tools/vbuild-examples.v , so that v build-examples will know that it compiles as a project, and not each .v file in there, as a separate program (which is the reason for the current CI failures).

[enghitalo]

What we are missing to a 500k+ Requests/sec in

wrk -H 'Connection: keep-alive' --connections 512 --threads 8 --duration 5s --timeout 1 http://localhost:3000/

Unfortunately, I don't have much time to research and investigate bottlenecks. So, having the community's help would be amazing!!

[enghitalo]

wrk -H 'Connection: keep-alive' --connections 512 --threads 8 --duration 60s --timeout 1 http://localhost:3000/ 

Running 1m test @ http://localhost:3000/
  8 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.42ms    1.12ms  13.48ms   78.10%
    Req/Sec    47.73k     1.74k   51.86k    75.52%
  22792466 requests in 1.00m, 2.02GB read
Requests/sec: 379530.05
Transfer/sec:     34.39MB

[enghitalo]

NOTE: Removing all lockers and runnig with -gc none We have 425k+ Requests/sec. But running valgrind --tool=helgrind ./vanilla_http_server we will get a lot off errors what means that Garbage Collector is not working well with multi-threading. Then some freak workarounds like

(*server).lock_flag.lock()
mut readed_request_buffer := []u8{cap: bytes_read}
(*server).lock_flag.unlock()
defer {
	unsafe {
		(*server).lock_flag.lock()
		readed_request_buffer.free()
		(*server).lock_flag.unlock()
	}
}
unsafe {
	readed_request_buffer.push_many(&request_buffer[0], bytes_read)
}

Needs to be made :|

[enghitalo]

I'm very busy this week. I'll try to complete this task as soon as possible.

[JalonSolov]

It makes a lot more sense to use manual memory management for something low-level like this, where you want the best possible throughput.

[spytheman]

I am more concerned about it not compiling on windows.

Examples should either compile everywhere (ideally), or be skipped on the platforms, where they are known to fail in cmd/tools/vbuild-examples.v, otherwise the CI will be broken.

[enghitalo]

I am more concerned about it not compiling on windows.

Examples should either compile everywhere (ideally), or be skipped on the platforms, where they are known to fail in cmd/tools/vbuild-examples.v, otherwise the CI will be broken.

I agree with you. It will be compiled in all OSs