test: make the message version configurable in the subscriber testcase

This change makes the version of the input message configurable in the subscriber testcase. Until now it was always set to 0, which made testing a transformer for any version other than 0 impossible.

db: db.Client gets sqlx Named Functions

What's Changed

• Expose named functionality of sqlx on db client by @bt-justtrack in #1121

Full Changelog: v0.20.0...v0.20.1

db: adding a retry executor to the client

This release includes small clean ups which result in breaking changes and optional automatic retries on db connection issues:

db.Settings as pointer

Most of the settings structs are passed as pointers in gosoline. To have some more consistency, the db package is doing it the same now. If you're using the settings struct directly, this is a breaking change.

proper "provide" handling of db connections

This change is about increasing consistency, too. Usually gosoline uses the appctx to provide services. Until now the db package still had its own custom implementation. This PR gets in in line with the rest of the packages. As a result the ctx is needed now to get a db connection, client or db repo. This is a breaking change, too.

retries on db connection issues

The db.Client has an backoff executor included which can be enabled via config. The executor will retry the sql statement in case of the following errors:

// pkg/db/executor.go:21
executor := exec.NewBackoffExecutor(logger, res, &executorSettings, []exec.ErrorChecker{
	exec.CheckConnectionError,
	exec.CheckTimeoutError,
	CheckInvalidConnection,
	CheckBadConnection,
	CheckIoTimeout,
}, notifier...)

The executor and retry backoff settings can be configured like:

db:
  default:
    retry:
      enabled: true
      backoff:
        initial_interval: 10ms
        max_attempts: 0
        max_elapsed_time: 10s
        max_interval: 100ms 

db: small improvements

This release adds several small improvements to the db package:

Sqler and ExecMultiInTx

We added a convenient function to execute multiple sql statements in one transaction:

sqls := []db.Sqler{
	squirrel.Update("`schemas`").Set("is_default", false),
	squirrel.Insert("`schemas`").Options("IGNORE").Columns("name", "is_default").Values(name, true),
	squirrel.Update("`schemas`").Set("is_default", true).Where(squirrel.Eq{"name": name}),
}

if _, err = client.ExecMultiInTx(ctx, sqls...); err != nil {
	return fmt.Errorf("can not execute queries: %w", err)
}

MySQL logging

We've added the gosoline logger to the MySQL driver so that critical errors get not printed to os.Stderr but instead getting logged as an error.

Settings

We've added a couple of settings for the db connection. The following yaml shows how to set them. The example shows the default values:

db:
  default:
    charset: utf8mb4
    collation: utf8mb4_general_ci
    connection_max_idletime: 120s
    timeouts:
      readTimeout: 0
      writeTimeout: 0
      timeout: 0 # 0 means that the os default will get used

mdlsub: fix db output to replace existing item on create

This changes mdlsub.OutputDb to do an update instead of an insert in case an item with the provided primary key already exists in the db. Previously, this would fail with a duplicate key error.

cloud/aws: added servicediscovery client and ec2 metadata provider

The metadata provider service allows for easy access for things like the AWS instance availability zone or many other parameters.

conc: make task-runner opt-in

What's Changed

You have to opt-in to use the task runner. Additionally, the task runner is now tagged as a background module. This helps to orderly shutdown your applications, after all tasks have been processed.

Add scheduler and task runner module + bug fixes

This release adds a task runner and scheduler module and contains various bug fixes.

The task runner (enabled by default) allows you to submit new tasks to be run in a controlled environment. It should only be a method of last resort to run go routines while still (somehow) tracking their results. IF ANY OF THEM FAILS, THE WHOLE TASK RUNNER WILL BE SHUT DOWN.

The scheduler module (which currently uses the task runner in the background) allows you to batch requests for a specified time together (or until reaching a batch size) and execute them all at once.

Other changes and bug fixes:

• The profiling module can now be enabled via a config setting without changing the code
• A possible leak of go routines was fixed in the kernel (when you execute multiple kernels after each other)
• A race condition was fixed where the kernel would report a failed module as successful

v0.18.4

In this release i've added a CreateUriHandler func in the httpserver package to support Handlers which supply an input struct with the tag (uri:"xyz").

v0.18.3

Fixes the selection of default credentials when compiling with tag integration to use against localstack.