Version upgrade (#6)

* Bump golang.org/x/crypto from 0.14.0 to 0.17.0 Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.14.0 to 0.17.0. - [Commits](golang/crypto@v0.14.0...v0.17.0) --- updated-dependencies: - dependency-name: golang.org/x/crypto dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> * add ParentEndpoint to Backend config structure and info to the backend logs. Signed-off-by: David Hontecillas <[email protected]> * Do not abort the loading of handler plugins when one of them fails to load. Adjust log levels. Signed-off-by: Daniel Ortiz <[email protected]> * Fix typo Fix typo in "Response" word. Signed-off-by: Javier Vázquez Romera <[email protected]> * Add the encoding to the HTTPResponseError so, when returning an error from the backend, the content-type matches. This change only applies when using "return_error_code" or "return_error_details". Signed-off-by: Daniel Ortiz <[email protected]> * Fix tests and add mux support. Signed-off-by: Daniel Ortiz <[email protected]> * Remove mux support since we can't write headers after the first write. Signed-off-by: Daniel Ortiz <[email protected]> * refactor: autofix issues in 1 file Resolved issues in transport/http/client/status_test.go with DeepSource Autofix * Remove unused code. Signed-off-by: Daniel Ortiz <[email protected]> * add parent endpoint method to backend logs to better indetify the parent endpoint Signed-off-by: David Hontecillas <[email protected]> * Pass the application context to the request and response modifiers. Also extend the structs passed to the modifier functions so it can retrieve the execution Context. Reponse modifers now have access to the Request. Updated examples to show the new features. Signed-off-by: Daniel Ortiz <[email protected]> * refactor: autofix issues in 1 file Resolved issues in proxy/plugin/modifier_test.go with DeepSource Autofix * do not export a function returning an unexported type Signed-off-by: kpacha <[email protected]> * set the timeouts for the internal endpoint and backend(s) Signed-off-by: kpacha <[email protected]> * build the final string directly Signed-off-by: kpacha <[email protected]> * remove unrequired request clone Signed-off-by: kpacha <[email protected]> * fix log messages Signed-off-by: kpacha <[email protected]> * simplify the timeout assignation Signed-off-by: kpacha <[email protected]> * feat: add overwrite functionality for invalid Pattern Signed-off-by: Emin Aktas <[email protected]> * Bump google.golang.org/protobuf from 1.30.0 to 1.33.0 Bumps google.golang.org/protobuf from 1.30.0 to 1.33.0. --- updated-dependencies: - dependency-name: google.golang.org/protobuf dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> * requests should be cloned at the split/merge and concurrent mw only Signed-off-by: kpacha <[email protected]> * do not use log.Fatal Signed-off-by: kpacha <[email protected]> * set the max header bytes param configurable Signed-off-by: kpacha <[email protected]> * fix the hash of the new config test Signed-off-by: kpacha <[email protected]> * clone the body if there are one or more unsafe backends Signed-off-by: kpacha <[email protected]> * Bump golang.org/x/net from 0.17.0 to 0.23.0 Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0. - [Commits](golang/net@v0.17.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * add config field to serve tls using multiple key pairs Signed-off-by: David Hontecillas <[email protected]> * use list of certificates for serving TLS Signed-off-by: David Hontecillas <[email protected]> * test serving multiple certificates Signed-off-by: David Hontecillas <[email protected]> * fix style issues in certs tests Signed-off-by: David Hontecillas <[email protected]> * add config option to tweak the DNS cache TTL Signed-off-by: David Hontecillas <[email protected]> * get go version from runtime Signed-off-by: David Hontecillas <[email protected]> * Correct typo in plugin registration log Signed-off-by: Albert Lombarte <[email protected]> * h2c disabled by default --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: David Hontecillas <[email protected]> Signed-off-by: Daniel Ortiz <[email protected]> Signed-off-by: Javier Vázquez Romera <[email protected]> Signed-off-by: kpacha <[email protected]> Signed-off-by: Emin Aktas <[email protected]> Signed-off-by: Albert Lombarte <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: David Hontecillas <[email protected]> Co-authored-by: Daniel Ortiz <[email protected]> Co-authored-by: Javier Vázquez Romera <[email protected]> Co-authored-by: A. Lombarte <[email protected]> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> Co-authored-by: Daniel López <[email protected]> Co-authored-by: Emin Aktas <[email protected]> Co-authored-by: Albert Lombarte <[email protected]>
Groww-OSS · Nov 6, 2024 · 0b75ada · 0b75ada
1 parent d550b5f
commit 0b75ada
Show file tree

Hide file tree

Showing 36 changed files with 682 additions and 134 deletions.
diff --git a/async/asyncagent.go b/async/asyncagent.go
@@ -1,7 +1,6 @@
 // SPDX-License-Identifier: Apache-2.0
 
 /*
-
  */
 package async
 
@@ -66,8 +65,13 @@ func (a AgentStarter) Start(
 
 		logger.Debug(fmt.Sprintf("[SERVICE: AsyncAgent][%s] Starting the async agent", agent.Name))
 
+		for i := range agent.Backend {
+			agent.Backend[i].Timeout = agent.Consumer.Timeout
+		}
+
 		endpoint := &config.EndpointConfig{
 			Endpoint:    agent.Name,
+			Timeout:     agent.Consumer.Timeout,
 			Backend:     agent.Backend,
 			ExtraConfig: agent.ExtraConfig,
 		}

diff --git a/config/config.go b/config/config.go
@@ -89,6 +89,12 @@ type ServiceConfig struct {
 	// after reading the headers and the Handler can decide what
 	// is considered too slow for the body.
 	ReadHeaderTimeout time.Duration `mapstructure:"read_header_timeout"`
+	// MaxHeaderBytes controls the maximum number of bytes the
+	// server will read parsing the request header's keys and
+	// values, including the request line. It does not limit the
+	// size of the request body.
+	// If zero, DefaultMaxHeaderBytes (1MB) is used.
+	MaxHeaderBytes int `mapstructure:"max_header_bytes"`
 
 	// DisableKeepAlives, if true, prevents re-use of TCP connections
 	// between different HTTP requests.
@@ -179,6 +185,9 @@ type ServiceConfig struct {
 	// ClientTLS is used to configure the http default transport
 	// with TLS parameters
 	ClientTLS *ClientTLS `mapstructure:"client_tls"`
+
+	// DNSCacheTTL is the duration of the cached data for the DNS lookups
+	DNSCacheTTL time.Duration `mapstructure:"dns_cache_ttl"`
 }
 
 // AsyncAgent defines the configuration of a single subscriber/consumer to be initialized
@@ -280,6 +289,15 @@ type Backend struct {
 	HeadersToPass []string `mapstructure:"input_headers"`
 	// QueryStringsToPass has the list of query string params to be sent to the backend
 	QueryStringsToPass []string `mapstructure:"input_query_strings"`
+
+	// ParentEndpoint is to be filled by the parent endpoint with its pattern enpoint
+	// so logs and other instrumentation can output better info (thus, it is not loaded
+	// with `mapstructure` or `json` tags).
+	ParentEndpoint string `json:"-" mapstructure:"-"`
+	// ParentEndpointMethod is to be filled by the parent endpoint with its enpoint method
+	// so logs and other instrumentation can output better info (thus, it is not loaded
+	// with `mapstructure` or `json` tags).
+	ParentEndpointMethod string `json:"-" mapstructure:"-"`
 }
 
 // Plugin contains the config required by the plugin module
@@ -288,19 +306,26 @@ type Plugin struct {
 	Pattern string `mapstructure:"pattern"`
 }
 
+// TLSKeyPair contains a pair of public and private keys
+type TLSKeyPair struct {
+	PublicKey  string `mapstructure:"public_key"`
+	PrivateKey string `mapstructure:"private_key"`
+}
+
 // TLS defines the configuration params for enabling TLS (HTTPS & HTTP/2) at the router layer
 type TLS struct {
-	IsDisabled               bool     `mapstructure:"disabled"`
-	PublicKey                string   `mapstructure:"public_key"`
-	PrivateKey               string   `mapstructure:"private_key"`
-	CaCerts                  []string `mapstructure:"ca_certs"`
-	MinVersion               string   `mapstructure:"min_version"`
-	MaxVersion               string   `mapstructure:"max_version"`
-	CurvePreferences         []uint16 `mapstructure:"curve_preferences"`
-	PreferServerCipherSuites bool     `mapstructure:"prefer_server_cipher_suites"`
-	CipherSuites             []uint16 `mapstructure:"cipher_suites"`
-	EnableMTLS               bool     `mapstructure:"enable_mtls"`
-	DisableSystemCaPool      bool     `mapstructure:"disable_system_ca_pool"`
+	IsDisabled               bool         `mapstructure:"disabled"`
+	PublicKey                string       `mapstructure:"public_key"`
+	PrivateKey               string       `mapstructure:"private_key"`
+	CaCerts                  []string     `mapstructure:"ca_certs"`
+	MinVersion               string       `mapstructure:"min_version"`
+	MaxVersion               string       `mapstructure:"max_version"`
+	CurvePreferences         []uint16     `mapstructure:"curve_preferences"`
+	PreferServerCipherSuites bool         `mapstructure:"prefer_server_cipher_suites"`
+	CipherSuites             []uint16     `mapstructure:"cipher_suites"`
+	EnableMTLS               bool         `mapstructure:"enable_mtls"`
+	DisableSystemCaPool      bool         `mapstructure:"disable_system_ca_pool"`
+	Keys                     []TLSKeyPair `mapstructure:"keys"`
 }
 
 // ClientTLS defines the configuration params for an HTTP Client
@@ -497,6 +522,9 @@ func (s *ServiceConfig) initEndpoints() error {
 		e.ExtraConfig.sanitize()
 
 		for j, b := range e.Backend {
+			// we "tell" the backend which is his parent endpoint
+			b.ParentEndpoint = e.Endpoint
+			b.ParentEndpointMethod = e.Method
 			if err := s.initBackendDefaults(i, j); err != nil {
 				return err
 			}
@@ -787,6 +815,11 @@ func SetSequentialParamsPattern(pattern string) error {
 	return nil
 }
 
+// SetInvalidPattern sets the invalidPattern variable to the provided value.
+func SetInvalidPattern(pattern string) {
+	invalidPattern = pattern
+}
+
 func validateAddress(address string) bool {
 	ip := net.ParseIP(address)
 	return ip != nil

diff --git a/config/config_test.go b/config/config_test.go
@@ -211,7 +211,7 @@ func TestConfig_init(t *testing.T) {
 		t.Error(err.Error())
 	}
 
-	if hash != "GdZTJtCn9ZHj3iBR1ZxmZL65HjbTCU8HhbDG8YWudAo=" {
+	if hash != "mIfuGTgHS91DJtAE6KMVl2kcluxCzc9n3f6fi0YgWs8=" {
 		t.Errorf("unexpected hash: %s", hash)
 	}
 }
@@ -309,3 +309,32 @@ func TestConfig_initKOInvalidDebugPattern(t *testing.T) {
 
 	invalidPattern = dp
 }
+
+func TestConfig_initKOValidSetinvalidPattern(t *testing.T) {
+	dp := invalidPattern
+
+	invalidPattern = `^[^/]|/__(debug|echo|health)(/.*)?$`
+	subject := ServiceConfig{
+		Version: ConfigVersion,
+		Host:    []string{"http://127.0.0.1:8080"},
+		Endpoints: []*EndpointConfig{
+			{
+				Endpoint: "/*",
+				Method:   "GET",
+				Backend: []*Backend{
+					{
+						URLPattern: "/",
+						Host:       []string{"https://api.github.com"},
+						AllowList:  []string{"authorizations_url", "code_search_url"},
+					},
+				},
+			},
+		},
+	}
+
+	if err := subject.Init(); err != nil {
+		t.Error(err)
+	}
+
+	invalidPattern = dp
+}
diff --git a/config/parser.go b/config/parser.go
@@ -146,6 +146,7 @@ type parseableServiceConfig struct {
 	WriteTimeout          string                     `json:"write_timeout"`
 	IdleTimeout           string                     `json:"idle_timeout"`
 	ReadHeaderTimeout     string                     `json:"read_header_timeout"`
+	MaxHeaderBytes        int                        `json:"max_header_bytes"`
 	DisableKeepAlives     bool                       `json:"disable_keep_alives"`
 	DisableCompression    bool                       `json:"disable_compression"`
 	DisableStrictREST     bool                       `json:"disable_rest"`
@@ -164,6 +165,7 @@ type parseableServiceConfig struct {
 	TLS                   *parseableTLS              `json:"tls,omitempty"`
 	ClientTLS             *parseableClientTLS        `json:"client_tls,omitempty"`
 	UseH2C                bool                       `json:"use_h2c,omitempty"`
+	DNSCacheTTL           string                     `json:"dns_cache_ttl"`
 }
 
 func (p *parseableServiceConfig) normalize() ServiceConfig {
@@ -181,6 +183,7 @@ func (p *parseableServiceConfig) normalize() ServiceConfig {
 		WriteTimeout:          parseDuration(p.WriteTimeout),
 		IdleTimeout:           parseDuration(p.IdleTimeout),
 		ReadHeaderTimeout:     parseDuration(p.ReadHeaderTimeout),
+		MaxHeaderBytes:        p.MaxHeaderBytes,
 		DisableKeepAlives:     p.DisableKeepAlives,
 		DisableCompression:    p.DisableCompression,
 		DisableStrictREST:     p.DisableStrictREST,
@@ -195,6 +198,7 @@ func (p *parseableServiceConfig) normalize() ServiceConfig {
 		OutputEncoding:        p.OutputEncoding,
 		Plugin:                p.Plugin,
 		UseH2C:                p.UseH2C,
+		DNSCacheTTL:           parseDuration(p.DNSCacheTTL),
 	}
 	if p.TLS != nil {
 		cfg.TLS = &TLS{
@@ -210,6 +214,9 @@ func (p *parseableServiceConfig) normalize() ServiceConfig {
 			EnableMTLS:               p.TLS.EnableMTLS,
 			DisableSystemCaPool:      p.TLS.DisableSystemCaPool,
 		}
+		for _, k := range p.TLS.Keys {
+			cfg.TLS.Keys = append(cfg.TLS.Keys, TLSKeyPair(k))
+		}
 	}
 	if p.ClientTLS != nil {
 		cfg.ClientTLS = &ClientTLS{
@@ -242,18 +249,24 @@ func (p *parseableServiceConfig) normalize() ServiceConfig {
 	return cfg
 }
 
+type parseableTLSKeyPair struct {
+	PublicKey  string `json:"public_key"`
+	PrivateKey string `json:"private_key"`
+}
+
 type parseableTLS struct {
-	IsDisabled               bool     `json:"disabled"`
-	PublicKey                string   `json:"public_key"`
-	PrivateKey               string   `json:"private_key"`
-	CaCerts                  []string `json:"ca_certs"`
-	MinVersion               string   `json:"min_version"`
-	MaxVersion               string   `json:"max_version"`
-	CurvePreferences         []uint16 `json:"curve_preferences"`
-	PreferServerCipherSuites bool     `json:"prefer_server_cipher_suites"`
-	CipherSuites             []uint16 `json:"cipher_suites"`
-	EnableMTLS               bool     `json:"enable_mtls"`
-	DisableSystemCaPool      bool     `json:"disable_system_ca_pool"`
+	IsDisabled               bool                  `json:"disabled"`
+	PublicKey                string                `json:"public_key"`
+	PrivateKey               string                `json:"private_key"`
+	CaCerts                  []string              `json:"ca_certs"`
+	MinVersion               string                `json:"min_version"`
+	MaxVersion               string                `json:"max_version"`
+	CurvePreferences         []uint16              `json:"curve_preferences"`
+	PreferServerCipherSuites bool                  `json:"prefer_server_cipher_suites"`
+	CipherSuites             []uint16              `json:"cipher_suites"`
+	EnableMTLS               bool                  `json:"enable_mtls"`
+	DisableSystemCaPool      bool                  `json:"disable_system_ca_pool"`
+	Keys                     []parseableTLSKeyPair `json:"keys"`
 }
 
 type parseableClientTLS struct {

diff --git a/config/parser_test.go b/config/parser_test.go
@@ -15,6 +15,7 @@ func TestNewParser_ok(t *testing.T) {
     "port": 8080,
     "cache_ttl": "3600s",
     "timeout": "3s",
+    "max_header_bytes": 10000,
     "tls": {
 		"public_key":  "cert.pem",
 		"private_key": "key.pem"
@@ -108,6 +109,9 @@ func TestNewParser_ok(t *testing.T) {
 	if err != nil {
 		t.Error("Unexpected error. Got", err.Error())
 	}
+	if serviceConfig.MaxHeaderBytes != 10000 {
+		t.Errorf("unexpected max_header_bytes value. have %d, want 10000", serviceConfig.MaxHeaderBytes)
+	}
 	testExtraConfig(serviceConfig.ExtraConfig, t)
 
 	if endpoints := len(serviceConfig.Endpoints); endpoints != 3 {

diff --git a/core/version.go b/core/version.go
@@ -1,11 +1,14 @@
 // SPDX-License-Identifier: Apache-2.0
 
 /*
-   Package core contains some basic constants and variables
+Package core contains some basic constants and variables
 */
 package core
 
-import "fmt"
+import (
+	"fmt"
+	"runtime"
+)
 
 // KrakendHeaderName is the name of the custom KrakenD header
 const KrakendHeaderName = "X-KRAKEND"
@@ -14,7 +17,7 @@ const KrakendHeaderName = "X-KRAKEND"
 var KrakendVersion = "undefined"
 
 // GoVersion is the version of the go compiler used at build time
-var GoVersion = "undefined"
+var GoVersion = runtime.Version()
 
 // GlibcVersion is the version of the glibc used by CGO at build time
 var GlibcVersion = "undefined"

diff --git a/docs/BENCHMARKS.md b/docs/BENCHMARKS.md
@@ -100,7 +100,7 @@ Here you'll find some benchmarks of the different components of the Lura framewo
     BenchmarkEntityFormatter_grouping/with_20_elements-8            20000000           298 ns/op         384 B/op          3 allocs/op
     BenchmarkEntityFormatter_grouping/with_25_elements-8            20000000           298 ns/op         384 B/op          3 allocs/op
 
-## Repsonse property mapping
+## Response property mapping
 
     BenchmarkEntityFormatter_mapping/with_0_elements_with_0_extra_fields-8          100000000           61.1 ns/op        48 B/op          1 allocs/op
     BenchmarkEntityFormatter_mapping/with_1_elements_with_0_extra_fields-8          100000000           63.5 ns/op        48 B/op          1 allocs/op

diff --git a/go.mod b/go.mod
@@ -15,8 +15,9 @@ require (
 )
 
 require (
+	golang.org/x/net v0.23.0
 	golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
-	golang.org/x/text v0.13.0
+	golang.org/x/text v0.14.0
 )
 
 require (
@@ -36,9 +37,8 @@ require (
 	github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
 	github.com/ugorji/go/codec v1.2.11 // indirect
 	golang.org/x/arch v0.3.0 // indirect
-	golang.org/x/crypto v0.14.0 // indirect
-	golang.org/x/net v0.17.0 // indirect
-	golang.org/x/sys v0.13.0 // indirect
-	google.golang.org/protobuf v1.30.0 // indirect
+	golang.org/x/crypto v0.21.0 // indirect
+	golang.org/x/sys v0.18.0 // indirect
+	google.golang.org/protobuf v1.33.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 )