Skip to content

Commit

Permalink
Benchmark: pointing to wrong test files and updated with new benchmar…
Browse files Browse the repository at this point in the history 
…ks (#139)

* Rules Match function in grule rule engine

* Rules Match function in grule rule engine

* Fetch all matching rules for a fact order by salience

* Fetch all matching rules for a fact order by salience

* Fetch all matching rules for a fact order by salience

* Updated with review comments

* Updated with review comments

* Updated with review comments

* Test Grule rule engine follows logical operator precedence

* Test Grule rule engine follows logical operator precedence

* Benchmark details in main page

* benchmark-params-wrongly-set

* Updated the benchmarks

* Updated the benchmark results

Co-authored-by: Ferdinand Neman <[email protected]>
  • Loading branch information
@jinagamvasubabu @newm4n
jinagamvasubabu and newm4n authored Nov 19, 2020
1 parent 2a720dd commit 0bd490f
Show file tree
Hide file tree
Showing 4 changed files with 91 additions and 66 deletions.
14 changes: 7 additions & 7 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -113,20 +113,20 @@ Since business rules are actually treated as data. Adjusting the rule according
Grule's Documentation now viewable in ViewDocs. [http://hyperjumptech.viewdocs.io](http://hyperjumptech.viewdocs.io/grule-rule-engine)

### Benchmark
`Loading 100 and 1000 rules into KnowledgeBase`:
* To load `100` rules into knowledgeBase it took `17523 ns/op` (took the highest value) that is equal to `~0.017523ms` and (`5071 B/op`) `0.005071MB` memory per operation
`Loading rules into KnowledgeBase`:

* To load `1000` rules into knowledgeBase it took `21222 ns/op` (took the highest value) that is equal to `~0.021222ms` and (`5071 B/op`) `0.005071MB` memory per operation
* To load `100` rules into knowledgeBase it took `99342047 ns/op` (took the highest value) that is equal to `~99.342047ms` and (`49295906 B/op`) `~49.295906MB` memory per operation

`Executing a fact against 100 and 1000 rules`:
* To load `1000` rules into knowledgeBase it took `933617752 ns/op` (took the highest value) that is equal to `~933.617752ms` and (`488126636 B/op`) `~488.126636` memory per operation

* To execute a fact against 100 rules, Grule Engine took `~703 ns/op` (took the highest value as base) that is hardly `~0.000703ms` and (`512 B/op`) `0.000512MB` memory per operation which is pretty fast.
`Executing rules against a fact`:

* To execute a fact against 1000 rules, Grule Engine took `~587 ns/op` (took the highest value as base) that is hardly `~0.000587ms` and (`512 B/op`) `0.000512MB` memory per operation which is also pretty fast.
* To execute a fact against 100 rules, Grule Engine took `~9697 ns/op` (took the highest value as base) that is hardly `~0.009697ms` and `3957 B/op` which is pretty fast.

You can read the [detail report here](docs/Benchmarking_en.md)
* To execute a fact against 1000 rules, Grule Engine took `~568959 ns/op` (took the highest value as base) that is hardly `~0.568959ms` and `293710 B/op` which is also pretty fast.


You can read the [detail report here](docs/Benchmarking_en.md)


# Tasks and Help Wanted
Expand Down
131 changes: 78 additions & 53 deletions docs/Benchmarking_en.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,76 +26,101 @@ goarch: amd64
Number of Cores - 6
Ram - 16 GB 2400 MHz DDR4
pkg: github.com/hyperjumptech/grule-rule-engine/examples/benchmark

Benchmark_Grule_Load_Rules/100_rules-12 60 20968700 ns/op 8871574 B/op 216554 allocs/op
Benchmark_Grule_Load_Rules/100_rules#01-12 60 20800060 ns/op 8871255 B/op 216552 allocs/op
Benchmark_Grule_Load_Rules/100_rules#02-12 60 21284699 ns/op 8871410 B/op 216553 allocs/op
Benchmark_Grule_Load_Rules/100_rules#03-12 61 20414968 ns/op 8871317 B/op 216552 allocs/op
Benchmark_Grule_Load_Rules/100_rules#04-12 58 20618596 ns/op 8871612 B/op 216554 allocs/op
Benchmark_Grule_Load_Rules/100_rules#05-12 60 21217303 ns/op 8871294 B/op 216552 allocs/op
Benchmark_Grule_Load_Rules/100_rules#06-12 67 21312189 ns/op 8871592 B/op 216554 allocs/op
Benchmark_Grule_Load_Rules/100_rules#07-12 61 20592475 ns/op 8871213 B/op 216552 allocs/op
Benchmark_Grule_Load_Rules/100_rules#08-12 60 22628754 ns/op 8871388 B/op 216553 allocs/op
Benchmark_Grule_Load_Rules/100_rules#09-12 68 21192157 ns/op 8871223 B/op 216552 allocs/op
Benchmark_Grule_Load_Rules/100_rules#10-12 60 21242572 ns/op 8871226 B/op 216552 allocs/op

Benchmark_Grule_Load_Rules/1000_rules-12 6 209761389 ns/op 88641262 B/op 2141287 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#01-12 6 204268674 ns/op 88644670 B/op 2141304 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#02-12 6 210895687 ns/op 88639476 B/op 2141278 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#03-12 6 214102248 ns/op 88642209 B/op 2141293 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#04-12 5 268977045 ns/op 88639793 B/op 2141279 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#05-12 5 211837045 ns/op 88641822 B/op 2141289 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#06-12 6 221863753 ns/op 88642209 B/op 2141293 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#07-12 6 223676073 ns/op 88643585 B/op 2141299 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#08-12 6 224317362 ns/op 88643070 B/op 2141297 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#09-12 5 241930711 ns/op 88641422 B/op 2141289 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#10-12 4 261857403 ns/op 88637592 B/op 2141269 allocs/op

```

Result
--
To load `100` rules into knowledgeBase it took `22628754 ns/op` (took the highest value) that is equal to `22.628754ms` and (`8871388 B/op`) `8.8MB` memory per operation

### Test1 - Loading Rules into KnowledgeBase
Result:
To load `1000` rules into knowledgeBase it took `261857403 ns/op` (took the highest value) that is equal to `~261ms` and `88MB` memory per operation

### Test2 - Executing a fact against rules
Command to run:
```go
Load 100 and 1000 Rules into Grule rule Engine
Benchmark_Grule_Load_Rules/100_rules-12 67137 17387 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/100_rules#01-12 67485 17447 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/100_rules#02-12 67332 17408 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/100_rules#03-12 67992 17436 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/100_rules#04-12 68170 17420 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/100_rules#05-12 67777 17645 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/100_rules#06-12 65100 17431 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/100_rules#07-12 67396 17396 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/100_rules#08-12 68132 17458 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/100_rules#09-12 67881 17399 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/100_rules#10-12 67216 17523 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/1000_rules-12 66828 17823 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#01-12 69122 17581 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#02-12 67815 17425 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#03-12 67405 19681 ns/op 5070 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#04-12 48511 21222 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#05-12 67779 18999 ns/op 5070 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#06-12 56694 17691 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#07-12 69086 17641 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#08-12 51638 19401 ns/op 5070 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#09-12 58940 20498 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#10-12 67411 19487 ns/op 5071 B/op 118 allocs/op
Benchmark_Grule_Load_Rules/100_rules-12 12 96674568 ns/op 49297966 B/op 731119 allocs/op
Benchmark_Grule_Load_Rules/100_rules#01-12 12 97915910 ns/op 49293839 B/op 731103 allocs/op
Benchmark_Grule_Load_Rules/100_rules#02-12 12 97716674 ns/op 49293398 B/op 731129 allocs/op
Benchmark_Grule_Load_Rules/100_rules#03-12 12 97227219 ns/op 49299542 B/op 731145 allocs/op
Benchmark_Grule_Load_Rules/100_rules#04-12 12 99342047 ns/op 49295906 B/op 731131 allocs/op
Benchmark_Grule_Load_Rules/100_rules#05-12 12 98636912 ns/op 49297570 B/op 731228 allocs/op
Benchmark_Grule_Load_Rules/100_rules#06-12 12 98414282 ns/op 49297168 B/op 731122 allocs/op
Benchmark_Grule_Load_Rules/100_rules#07-12 12 97733003 ns/op 49299440 B/op 731184 allocs/op
Benchmark_Grule_Load_Rules/100_rules#08-12 12 98122635 ns/op 49297690 B/op 731132 allocs/op
Benchmark_Grule_Load_Rules/100_rules#09-12 12 98451525 ns/op 49292262 B/op 731055 allocs/op

Benchmark_Grule_Load_Rules/1000_rules-12 2 933617752 ns/op 488126636 B/op 7239752 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#01-12 2 926896605 ns/op 488120920 B/op 7239869 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#02-12 2 928509980 ns/op 488118076 B/op 7239757 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#03-12 2 926093793 ns/op 488119492 B/op 7239927 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#04-12 2 924214904 ns/op 488154840 B/op 7240215 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#05-12 2 928009912 ns/op 488078180 B/op 7239902 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#06-12 2 925822584 ns/op 488082700 B/op 7239303 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#07-12 2 923116273 ns/op 488088032 B/op 7239301 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#08-12 2 924545950 ns/op 488103888 B/op 7240207 allocs/op
Benchmark_Grule_Load_Rules/1000_rules#09-12 2 930476936 ns/op 488166652 B/op 7240389 allocs/op


```

To load `100` rules into knowledgeBase it took `17523 ns/op` (took the highest value) that is equal to `~0.017523ms` and (`5071 B/op`) `0.005071MB` memory per operation
To load `100` rules into knowledgeBase it took `99342047 ns/op` (took the highest value) that is equal to `~99.342047ms` and (`49295906 B/op`) `~49.295906MB` memory per operation

To load `1000` rules into knowledgeBase it took `21222 ns/op` (took the highest value) that is equal to `~0.021222ms` and (`5071 B/op`) `0.005071MB` memory per operation
To load `1000` rules into knowledgeBase it took `933617752 ns/op` (took the highest value) that is equal to `~933.617752ms` and (`488126636 B/op`) `~488.126636` memory per operation

### Test2 - Executing a fact against rules
Result:
```go
Executing a fact against 100 and 1000 rules
Benchmark_Grule_Execution_Engine/100_rules-12 2055945 574 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#01-12 2048078 570 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#02-12 2086953 572 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#03-12 2094231 571 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#04-12 2078065 576 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#05-12 2028356 642 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#06-12 2002248 628 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#07-12 1850121 703 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#08-12 1761343 585 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#09-12 2080953 594 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#10-12 2082880 573 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules-12 2082183 575 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#01-12 2098585 568 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#02-12 2090640 570 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#03-12 2109938 587 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#04-12 2045216 576 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#05-12 2092534 575 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#06-12 1994415 579 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#07-12 2098788 599 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#08-12 2092808 573 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#09-12 2085716 609 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#10-12 1864302 576 ns/op 512 B/op 9 allocs/op
Benchmark_Grule_Execution_Engine/100_rules-12 140134 8175 ns/op 3939 B/op 59 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#01-12 140442 8240 ns/op 3939 B/op 59 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#02-12 141249 8151 ns/op 3937 B/op 59 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#03-12 142011 8191 ns/op 3935 B/op 59 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#04-12 137010 8226 ns/op 3947 B/op 59 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#05-12 122870 9112 ns/op 3989 B/op 59 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#06-12 133470 9697 ns/op 3957 B/op 59 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#07-12 135206 8210 ns/op 3952 B/op 59 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#08-12 139328 8213 ns/op 3941 B/op 59 allocs/op
Benchmark_Grule_Execution_Engine/100_rules#09-12 136437 8287 ns/op 3949 B/op 59 allocs/op

Benchmark_Grule_Execution_Engine/1000_rules-12 1912 525881 ns/op 273244 B/op 3843 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#01-12 2014 508415 ns/op 260310 B/op 3651 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#02-12 1770 568959 ns/op 293710 B/op 4147 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#03-12 1984 513188 ns/op 263958 B/op 3706 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#04-12 1771 566971 ns/op 293550 B/op 4145 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#05-12 1858 541169 ns/op 280695 B/op 3954 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#06-12 1896 530956 ns/op 275395 B/op 3875 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#07-12 1939 522682 ns/op 269694 B/op 3790 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#08-12 1851 545408 ns/op 281652 B/op 3968 allocs/op
Benchmark_Grule_Execution_Engine/1000_rules#09-12 1844 543697 ns/op 282657 B/op 3983 allocs/op



```

To execute a fact against 100 rules, Grule Engine took `~703 ns/op` (took the highest value as base) that is hardly `~0.000703ms` and (`512 B/op`) `0.000512MB` memory per operation which is pretty fast.
To execute a fact against 100 rules, Grule Engine took `~9697 ns/op` (took the highest value as base) that is hardly `~0.009697ms` and `3957 B/op` which is pretty fast.

To execute a fact against 1000 rules, Grule Engine took `~587 ns/op` (took the highest value as base) that is hardly `~0.000587ms` and (`512 B/op`) `0.000512MB` memory per operation which is also pretty fast.
To execute a fact against 1000 rules, Grule Engine took `~568959 ns/op` (took the highest value as base) that is hardly `~0.568959ms` and `293710 B/op` which is also pretty fast.


6 changes: 3 additions & 3 deletions examples/benchmark/ExecRules_benchmark_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ func Benchmark_Grule_Execution_Engine(b *testing.B) {
{"1000 rules", load1000RulesIntoKnowledgebase},
}
for _, rule := range rules {
for k := 0.; k <= 10; k++ {
for k := 0; k < 10; k++ {
b.Run(fmt.Sprintf("%s", rule.name), func(b *testing.B) {
rule.fun()
for i := 0; i < b.N; i++ {
Expand All @@ -52,7 +52,7 @@ func Benchmark_Grule_Execution_Engine(b *testing.B) {
}

func load100RulesIntoKnowledgebase() {
input, _ := ioutil.ReadFile("100_complicated_rules.grl")
input, _ := ioutil.ReadFile("100_rules.grl")
rules := string(input)
fact := &RideFact{
Distance: 6000,
Expand All @@ -68,7 +68,7 @@ func load100RulesIntoKnowledgebase() {
}

func load1000RulesIntoKnowledgebase() {
input, _ := ioutil.ReadFile("1000_complicated_rules.grl")
input, _ := ioutil.ReadFile("1000_rules.grl")
rules := string(input)
fact := &RideFact{
Distance: 6000,
Expand Down
6 changes: 3 additions & 3 deletions examples/benchmark/LoadRules_benchmark_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ func Benchmark_Grule_Load_Rules(b *testing.B) {
{"1000 rules", load1000RulesIntoKnowledgeBase},
}
for _, rule := range rules {
for k := 0.; k <= 10; k++ {
for k := 0; k < 10; k++ {
b.Run(fmt.Sprintf("%s", rule.name), func(b *testing.B) {
for i := 0; i < b.N; i++ {
rule.fun()
Expand All @@ -42,7 +42,7 @@ func Benchmark_Grule_Load_Rules(b *testing.B) {
}

func load100RulesIntoKnowledgeBase() {
input, _ := ioutil.ReadFile("100_complicated_rules.grl")
input, _ := ioutil.ReadFile("100_rules.grl")
rules := string(input)
fact := &RideFact{
Distance: 6000,
Expand All @@ -58,7 +58,7 @@ func load100RulesIntoKnowledgeBase() {
}

func load1000RulesIntoKnowledgeBase() {
input, _ := ioutil.ReadFile("1000_complicated_rules.grl")
input, _ := ioutil.ReadFile("1000_rules.grl")
rules := string(input)
fact := &RideFact{
Distance: 6000,
Expand Down

0 comments on commit 0bd490f

Please sign in to comment.