-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[log] Refactor and measure performance using benchmark #90
Commits on Sep 6, 2018
-
[log] Move survey of log lib into survey folder
- will have a design folder next to it, so the doc folder is no longer flat
Configuration menu - View commit details
-
Copy full SHA for 2a6b47c - Browse repository at this point
Copy the full SHA 2a6b47cView commit details -
- basically covered background, what was the goals, good thing about current implementation, bad thing etc.
Configuration menu - View commit details
-
Copy full SHA for fde590d - Browse repository at this point
Copy the full SHA fde590dView commit details
Commits on Sep 22, 2018
-
[httputil] Add unix domain sock transport #93
- get `{"message":"page not found"}`
Configuration menu - View commit details
-
Copy full SHA for 2c06217 - Browse repository at this point
Copy the full SHA 2c06217View commit details -
[httputil] Unix Transport can proxy docker sock
- just trim the path and keep the first /
Configuration menu - View commit details
-
Copy full SHA for 10e5c92 - Browse repository at this point
Copy the full SHA 10e5c92View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2205cab - Browse repository at this point
Copy the full SHA 2205cabView commit details -
[util] Fix #92 add dump val as (pretty) json yaml
- disable server handler check in unix server listen test, it failed CI because when close the server/remove socket file it seems the server didn't shut down gracefully ... https://travis-ci.org/dyweb/gommon/jobs/431772857
Configuration menu - View commit details
-
Copy full SHA for b360fe0 - Browse repository at this point
Copy the full SHA b360fe0View commit details -
[log][doc] Add implementation order
- first do benchmark to get a basic understanding of other logging libraries performance and what feature is lacking is gommon so we can modify the interface - then handle fields, structured logging makes log parsing much more easier, and is more generic since parser don't need to know the exact context, i.e. http access log etc. - finally handle the tree of logger problem, along with the skip runtime caller, it's essential for clean logging in testing and util functions
Configuration menu - View commit details
-
Copy full SHA for 33da251 - Browse repository at this point
Copy the full SHA 33da251View commit details -
[log] Init benchmark, json format no fields #88
go test -bench=. goos: linux goarch: amd64 pkg: github.com/dyweb/gommon/log/_benchmarks BenchmarkWithoutFieldsJSON/gommon-8 20000000 96.9 ns/op BenchmarkWithoutFieldsJSON/Zap-8 10000000 176 ns/op BenchmarkWithoutFieldsJSON/Zap.Sugar-8 5000000 257 ns/op BenchmarkWithoutFieldsJSON/zerorlog-8 20000000 118 ns/op PASS ok github.com/dyweb/gommon/log/_benchmarks 8.031s
Configuration menu - View commit details
-
Copy full SHA for c0d47a3 - Browse repository at this point
Copy the full SHA c0d47a3View commit details
Commits on Nov 18, 2018
-
[log] Add mem allocation to benchmark #88
go test -bench=. -benchmem goos: linux goarch: amd64 pkg: github.com/dyweb/gommon/log/_benchmarks BenchmarkWithoutFieldsJSON/gommon-8 10000000 126 ns/op 496 B/op 4 allocs/op BenchmarkWithoutFieldsJSON/Zap-8 10000000 205 ns/op 0 B/op 0 allocs/op BenchmarkWithoutFieldsJSON/Zap.Sugar-8 3000000 353 ns/op 128 B/op 2 allocs/op BenchmarkWithoutFieldsJSON/zerorlog-8 10000000 131 ns/op 0 B/op 0 allocs/op PASS ok github.com/dyweb/gommon/log/_benchmarks 6.617s
Configuration menu - View commit details
-
Copy full SHA for 7fc0d70 - Browse repository at this point
Copy the full SHA 7fc0d70View commit details -
[log] Accept []byte in formatHead
- previously make is called inside formatHead so it will allocate on heap for sure, after moving the make to handler and increase the hard coded prefix ... some allocation can be avoided, at least we know the direction for optimization now (either use a pool or use a fixed length make, if it's large enough it can handle most log messages on stack)
Configuration menu - View commit details
-
Copy full SHA for 58e9dc6 - Browse repository at this point
Copy the full SHA 58e9dc6View commit details
Commits on Nov 19, 2018
-
[log][bench] Add logrus and apex/bench to benchmark
BenchmarkWithoutFieldsText/gommon-8 30000000 129 ns/op 272 B/op 3 allocs/op BenchmarkWithoutFieldsText/std-8 10000000 401 ns/op 128 B/op 2 allocs/op BenchmarkWithoutFieldsText/zap-8 20000000 273 ns/op 72 B/op 3 allocs/op BenchmarkWithoutFieldsText/zap.sugar-8 20000000 309 ns/op 200 B/op 5 allocs/op BenchmarkWithoutFieldsText/zerolog-8 3000000 1395 ns/op 2011 B/op 36 allocs/op BenchmarkWithoutFieldsText/apex-8 3000000 1551 ns/op 320 B/op 15 allocs/op BenchmarkWithoutFieldsText/logrus-8 1000000 3209 ns/op 769 B/op 15 allocs/op BenchmarkWithoutFieldsJSON/gommon-8 50000000 79.6 ns/op 288 B/op 3 allocs/op BenchmarkWithoutFieldsJSON/zap-8 20000000 179 ns/op 0 B/op 0 allocs/op BenchmarkWithoutFieldsJSON/zap.sugar-8 20000000 259 ns/op 128 B/op 2 allocs/op BenchmarkWithoutFieldsJSON/zerolog-8 50000000 113 ns/op 0 B/op 0 allocs/op BenchmarkWithoutFieldsJSON/apex-8 2000000 1920 ns/op 512 B/op 10 allocs/op BenchmarkWithoutFieldsJSON/logrus-8 2000000 2990 ns/op 1218 B/op 22 allocs/op
Configuration menu - View commit details
-
Copy full SHA for 806e8e9 - Browse repository at this point
Copy the full SHA 806e8e9View commit details
Commits on Nov 21, 2018
-
[log][doc] Zap is encoding context rightaway when add
- originally I though it merge the fields when output, but actually when you call logger.With(f1, f2), it encode f1, f2 right away into the underlying, it makes adding context cheap but couple context with the core (similar to handler in gommon/log) so that each core actually has state (handler in gommon/log are stateless, for now), exchange space for time - a drawback of this if a key is used two times, you end up having two fields instead of 1, zerolog has the same problem, and they mentioned in README that json allow duplicated key and json encoding/decoding library will handle it
Configuration menu - View commit details
-
Copy full SHA for 27f6107 - Browse repository at this point
Copy the full SHA 27f6107View commit details
Commits on Nov 24, 2018
-
[log][bench] Add disabled level and klog (glog)
- when use `interface{}` as parameter, it will cause allocation, I know it is caued by `interface{}` based on the `16B` size but don't quite get why it escape ...
Configuration menu - View commit details
-
Copy full SHA for 1e01659 - Browse repository at this point
Copy the full SHA 1e01659View commit details -
[log][doc] Add example of other log libraries
- json format with context (logger w/ fields) - logrus and apex are actually entry w/ context where entry hold a pointer to the logger it should use ...
Configuration menu - View commit details
-
Copy full SHA for 773794c - Browse repository at this point
Copy the full SHA 773794cView commit details -
[log] Return struct instead of pointer in factory
- was planning to change identity to a struct, but since it could be nil and it is not used most of the time, decided to change it back to pointer - [ ] TODO: updated gommon generator template, so all code that use struct logger factory need update
Configuration menu - View commit details
-
Copy full SHA for a4fa6b1 - Browse repository at this point
Copy the full SHA a4fa6b1View commit details
Commits on Nov 25, 2018
-
[log][handler] Support contex in handler interface
- the 'context` is actually just fields attached to logger - there aren't many tests so we didn't break any test ....
Configuration menu - View commit details
-
Copy full SHA for 04bd691 - Browse repository at this point
Copy the full SHA 04bd691View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6496dca - Browse repository at this point
Copy the full SHA 6496dcaView commit details -
[log][handler] Reduce interface methods 6 -> 1
- previously was not using empty value for optional parameters and endup having 6 functions, this made handler implementation error prone due to copy and paste also made handler func impossible - now there is just one log method in Handler call HandleLog
Configuration menu - View commit details
-
Copy full SHA for 19539b3 - Browse repository at this point
Copy the full SHA 19539b3View commit details -
[log][bench] Add InfoF to benchmark
- it has less allocation than `Info` because it accepts string instead of `interface{}`
Configuration menu - View commit details
-
Copy full SHA for beff00e - Browse repository at this point
Copy the full SHA beff00eView commit details -
[log][level] Add Print as alias for Info level
- this should makes a gommon/log drop in replacement for some library that requires a standard library like interface `Print` and `Printf` - [ ] TODO: I don't want to add `Println` because `\n` does not make sense for most handlers, json, text w/ fields etc. there will be a `\n` for most most handlers, not after the message but after the fields
Configuration menu - View commit details
-
Copy full SHA for ba8d224 - Browse repository at this point
Copy the full SHA ba8d224View commit details
Commits on Nov 26, 2018
-
[log][json] Support escape in json string
- just copy and paste from standard library and remove check on html and jsonp, looked at zap and zerolog, they are basically the same ...
Configuration menu - View commit details
-
Copy full SHA for affaa35 - Browse repository at this point
Copy the full SHA affaa35View commit details -
Configuration menu - View commit details
-
Copy full SHA for b9725a1 - Browse repository at this point
Copy the full SHA b9725a1View commit details -
[log][bench] io.Writer cause byte slice to escape
- at first I thought allocate a fixed size on stack would remove the allocation, but I was wrong, because the slice is passed to io.Writer and a call on interface is marked as `parameter to indirect call` and so even a fixed size slice can escape ...
Configuration menu - View commit details
-
Copy full SHA for 53a65fe - Browse repository at this point
Copy the full SHA 53a65feView commit details -
[log][handler] Use magic number to reduce alloc
- since we can't avoid the alloc due to call on io.Writer, we just need to make sure to allocate a slice large enough for message and fields so we don't need to allocate during the middle of append - [ ] TODO: these are just magic numbers, which will waste resource/won't work when the size of fields key/value are too small/too big ... - [ ] TODO: no context with fields json still have two alloc ```` BenchmarkWithContextFieldsJSON/gommon-8 30000000 117 ns/op 352 B/op 3 allocs/op BenchmarkWithContextFieldsJSON/gommon.F-8 50000000 74.0 ns/op 224 B/op 1 allocs/op BenchmarkNoContextWithFieldsJSON/gommon.F-8 50000000 92.2 ns/op 352 B/op 2 allocs/op ````
Configuration menu - View commit details
-
Copy full SHA for 276d177 - Browse repository at this point
Copy the full SHA 276d177View commit details -
[log][bench] fields seems to be causing alloc
- at first I suspect it's because varadic arg but it seems not ... - could be the `Fields` type
Configuration menu - View commit details
-
Copy full SHA for 7fc9d48 - Browse repository at this point
Copy the full SHA 7fc9d48View commit details -
[log][bench] Interface caused alloc on fields
- this happens to zap as well but didn't happen to zerolog because latter don't have any usage of interface methods when attaching fields - this only works because zerolog only support json, so it can just encode things right away and don't need handler interface, the drawback is for all the other format, it has to decode the bytes into map[string]interface{} and then print them out ...
Configuration menu - View commit details
-
Copy full SHA for d2217b8 - Browse repository at this point
Copy the full SHA d2217b8View commit details -
[util] Add dump json to testutil #92
- actually they are alredy there in pretty.go under same package ...
Configuration menu - View commit details
-
Copy full SHA for a51fd3f - Browse repository at this point
Copy the full SHA a51fd3fView commit details -
Configuration menu - View commit details
-
Copy full SHA for e880246 - Browse repository at this point
Copy the full SHA e880246View commit details
Commits on Dec 6, 2018
-
[log][doc] Add design for tree of logger
- in v1 we followed logrus and add a filter for filtering log - in v2 we added parent and children but it will cause gc problem
Configuration menu - View commit details
-
Copy full SHA for 701080d - Browse repository at this point
Copy the full SHA 701080dView commit details
Commits on Dec 8, 2018
-
Configuration menu - View commit details
-
Copy full SHA for 13979fc - Browse repository at this point
Copy the full SHA 13979fcView commit details -
[log] Add log registry to keep tree #33 #78
- registry will make the tree explicit, logger has to manually register themselves to registry and registry have to manually register to other registry, short lived logger should not register them self - registry also have identity and is like folder while logger is like file (log structured file system XD) - the map for children logger will be removed in following commit, and generator logic will change as well
Configuration menu - View commit details
-
Copy full SHA for 9f39ed1 - Browse repository at this point
Copy the full SHA 9f39ed1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5e27c56 - Browse repository at this point
Copy the full SHA 5e27c56View commit details -
[log] Remove map of child loggers #33 #78
- now only registry is used to keep the relationship between loggers across package and projects
Configuration menu - View commit details
-
Copy full SHA for 33a3df1 - Browse repository at this point
Copy the full SHA 33a3df1View commit details -
Configuration menu - View commit details
-
Copy full SHA for b8957fa - Browse repository at this point
Copy the full SHA b8957faView commit details
Commits on Dec 9, 2018
-
[log] Add Copy for create func & method logger
- add WalkRegistry - remove deprecated logger_tree logic
Configuration menu - View commit details
-
Copy full SHA for e7d7305 - Browse repository at this point
Copy the full SHA e7d7305View commit details -
[log][bench] Add caller increased 3 times
- for single message w/ 1 allocation, increased from 60ns -> 236 - remove some outdated doc on log package, most of them are still very outdated though ...
Configuration menu - View commit details
-
Copy full SHA for 6d2de9b - Browse repository at this point
Copy the full SHA 6d2de9bView commit details -
Configuration menu - View commit details
-
Copy full SHA for f1a6678 - Browse repository at this point
Copy the full SHA f1a6678View commit details