Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimise sanitize with string builder #2918

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions filters/builtin/stripquery.go
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,8 @@ func validHeaderFieldByte(b byte) bool {
}

// make sure we don't generate invalid headers
func sanitize(input string) string {
// temporary public function to benchmark it
Copy link
Member

@AlexanderYastrebov AlexanderYastrebov Feb 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no need to export it as benchmarks sits in the same package.
Usually we introduce benchmark as a first commit and include baseline benchmark results in the commit message.
Then change implementation, re-run benchmark and compare results to the baseline via https://pkg.go.dev/golang.org/x/perf/cmd/benchstat

To get a statistically meaningful benchmark results it should be run several times, usually we run 10.
See this PR as an example of performance-related change #2870

I've also created a small script to automate benchmarking two commits (its important that benchmark function is added in a separate commit to establish the baseline), see golang/go#63233 (comment)

func Sanitize(input string) string {
toAscii := strconv.QuoteToASCII(input)
var b bytes.Buffer
for _, i := range toAscii {
Expand All @@ -62,6 +63,18 @@ func sanitize(input string) string {
return b.String()
}

// temporary public function to benchmark it
func NewSanitize(input string) string {
toAscii := strconv.QuoteToASCII(input)
var s strings.Builder
for _, i := range toAscii {
if validHeaderFieldByte(byte(i)) {
s.WriteRune(i)
}
}
return s.String()
Comment on lines +68 to +75
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To get ultimate performance we can use byte version https://pkg.go.dev/strconv#AppendQuoteToASCII here and then replace invalid bytes with - instead of allocating another slice.

}

// Strips the query parameters and optionally preserves them in the X-Query-Param-xyz headers.
func (f *stripQuery) Request(ctx filters.FilterContext) {
r := ctx.Request()
Expand All @@ -85,7 +98,7 @@ func (f *stripQuery) Request(ctx filters.FilterContext) {
if r.Header == nil {
r.Header = http.Header{}
}
r.Header.Add(fmt.Sprintf("X-Query-Param-%s", sanitize(k)), v)
r.Header.Add(fmt.Sprintf("X-Query-Param-%s", Sanitize(k)), v)
}
}

Expand Down
39 changes: 38 additions & 1 deletion filters/builtin/stripquery_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,13 @@
package builtin

import (
"github.com/zalando/skipper/filters/filtertest"
"fmt"
"net/http"
"net/url"
"strings"
"testing"

"github.com/zalando/skipper/filters/filtertest"
)

func TestCreateStripQueryFilter(t *testing.T) {
Expand Down Expand Up @@ -91,3 +93,38 @@ func TestPreserveQuery(t *testing.T) {
}
}
}

func BenchmarkStripQuery(b *testing.B) {
var table = []struct {
url string
}{
{url: "http://example.org/foo?bar=baz"},
{url: "http://example.org/foo?query=cXVlcnkgewogIG5hbWUKfQo%253D&variables=&apiEndpoint=https%253A%252F%252Fwww.zalando.de%252Fapi%252Fgraphql%252Fpreview&frontendType=web&zalandoFeature=manual"},
{url: "http://example.org/foo?馬鹿=123"},
{url: "http://example.org/foo?a%20b=123"},
}

for i, v := range table {
url, _ := url.ParseRequestURI(v.url)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is redundant, test cases should contain query string instead of url

q := url.Query()
b.Run(fmt.Sprintf("[old sanitize] url %d", i+1), func(b *testing.B) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to add b.ReportAllocs() instead of -benchmem flag

for i := 0; i < b.N; i++ {
for k := range q {
Sanitize(k)
}
}
})
}

for i, v := range table {
url, _ := url.ParseRequestURI(v.url)
q := url.Query()
b.Run(fmt.Sprintf("[new sanitize] url %d", i + 1), func(b *testing.B) {
for i := 0; i < b.N; i++ {
for k := range q {
NewSanitize(k)
}
}
})
}
Comment on lines +119 to +129
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no need to have a second benchmark, see comment above.

}
14 changes: 14 additions & 0 deletions results.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
`go test -benchmem -run=^$ -bench ^BenchmarkStripQuery$ -benchtime=10s github.com/zalando/skipper/filters/builtin`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add results to the commit message as described in the comment above.


```
BenchmarkStripQuery/[old_sanitize]_url_1-8 85732707 118.6 ns/op 80 B/op 4 allocs/op
BenchmarkStripQuery/[old_sanitize]_url_2-8 14885370 808.2 ns/op 472 B/op 15 allocs/op
BenchmarkStripQuery/[old_sanitize]_url_3-8 73297474 162.9 ns/op 120 B/op 4 allocs/op
BenchmarkStripQuery/[old_sanitize]_url_4-8 100000000 114.1 ns/op 80 B/op 4 allocs/op


BenchmarkStripQuery/[new_sanitize]_url_1-8 100000000 103.5 ns/op 24 B/op 3 allocs/op
BenchmarkStripQuery/[new_sanitize]_url_2-8 15854868 778.5 ns/op 192 B/op 14 allocs/op
BenchmarkStripQuery/[new_sanitize]_url_3-8 71689101 166.3 ns/op 64 B/op 4 allocs/op
BenchmarkStripQuery/[new_sanitize]_url_4-8 100000000 102.0 ns/op 24 B/op 3 allocs/op
```
Loading