v0.24.0

Fixture loader refactoring

THIS RELEASE CONTAINS BREAKING CHANGES FOR USERS OF THE FIXTURES PACKAGE

Gosoline Fixture loader refactoring changes:

In this release I have refactored the fixture loading process to make it more flexible for custom implementations.

The FixtureBuilderFactory type was renamed to FixtureSetsFactory for clarity of what it is. Also its Signature changed, as can be seen below.

The WithFixtureBuilderFactory application option was renamed to WithFixtureSetFactory and gets a FixtureSetsFactory passed (func(ctx context.Context, config cfg.Config, logger log.Logger) ([]FixtureSet, error)).

The FixtureSet struct was renamed to simpleFixtureSet and should be used with its factory func NewSimpleFixtureSet.
FixtureSet is now an interface with only a Write(ctx context.Context) error func.
This makes the fixture loading more customizable, if you want to have your own implementation that is specific to your use-case (not covered by gosoline).

The FixtureLoader doesn't use a writer factory anymore and makes the user provide a writer (when using a simpleFixtureSet), also its interface doesn't get passed a []*fixtures.FixtureSet anymore but []fixtures.FixtureSet (slice of pointers to structs => slice of interfaces).

The FixtureBuilder interface was removed without replacement, as the FixtureSetsFactory is supposed to deliver the fixture sets directly.

The FixtureWriterFactory interface was removed without replacement, as the simpleFixtureSet contains a writer, when used. If a different FixtureSet implementation is used, it is supposed to take care of writing (or abstract it away).

The FixtureWriter now gets passed a []any as fixtures instead of a *FixtureSet, as the implementations always only accessed the fixtures key of the original FixtureSet anyways.

The SimpleFixtureBuilderFactory was removed in favor of the NewSimpleFixtureSet factory func (defaults to enabled=true and purge=false, otherwise specify any of fixtures.WithEnabled(false) or fixtures.WithPurge(true)).

The WithFixtureBuilderFactories test suite option was renamed to WithFixtureSetFactories and accepts any amount of FixtureSetsFactory args.

The logic for checking whether a fixture set is enabled, should purge or write has been moved to the SimpleFixtureSet. Custom implementations may want to adopt the changes accordingly to stay compatible with prior application behavior.

The NamedFixture struct is now mostly Generic compatible, making it more type-safe and nicer to deal with due to proper typing.

The Writer factory funcs were replaced and their signatures changed:

• BlobFixtureWriterFactory(settings *BlobFixturesSettings) FixtureWriterFactory => NewBlobFixtureWriter(ctx context.Context, config cfg.Config, logger log.Logger, settings *BlobFixturesSettings) (FixtureWriter, error)
• ConfigurableKvStoreFixtureWriterFactory[T any](name string) FixtureWriterFactory => NewConfigurableKvStoreFixtureWriter[T any](ctx context.Context, config cfg.Config, logger log.Logger, name string) (FixtureWriter, error)
• DynamoDbFixtureWriterFactory(settings *ddb.Settings, options ...DdbWriterOption) FixtureWriterFactory => NewDynamoDbFixtureWriter(ctx context.Context, config cfg.Config, logger log.Logger, settings *ddb.Settings, options ...DdbWriterOption) (FixtureWriter, error)
• DynamoDbKvStoreFixtureWriterFactory[T any](modelId *mdl.ModelId) FixtureWriterFactory => NewDynamoDbKvStoreFixtureWriter[T any](ctx context.Context, config cfg.Config, logger log.Logger, modelId *mdl.ModelId) (FixtureWriter, error)
• MysqlOrmFixtureWriterFactory(metadata *db_repo.Metadata) FixtureWriterFactory => NewMysqlOrmFixtureWriter(ctx context.Context, config cfg.Config, logger log.Logger, metadata *db_repo.Metadata) (FixtureWriter, error)
• MysqlPlainFixtureWriterFactory(metadata *MysqlPlainMetaData) FixtureWriterFactory => NewMysqlPlainFixtureWriter(ctx context.Context, config cfg.Config, logger log.Logger, metadata *MysqlPlainMetaData) (FixtureWriter, error)
• MysqlSqlxFixtureWriterFactory(metadata *MysqlSqlxMetaData) FixtureWriterFactory => NewMysqlSqlxFixtureWriter(ctx context.Context, config cfg.Config, logger log.Logger, metadata *MysqlSqlxMetaData) (FixtureWriter, error)
• RedisFixtureWriterFactory(name *string, operation *string) FixtureWriterFactory => NewRedisFixtureWriter(ctx context.Context, config cfg.Config, logger log.Logger, name string, operation string) (FixtureWriter, error)
• RedisKvStoreFixtureWriterFactory[T any](modelId *mdl.ModelId) FixtureWriterFactory => NewRedisKvStoreFixtureWriter[T any](ctx context.Context, config cfg.Config, logger log.Logger, modelId *mdl.ModelId) (FixtureWriter, error)

For each fixture kind there is now a factory func that allows you to easily create a fixture set that can be passed to fixtures.NewFixtureSetsFactory for usage with the fixture loader kernel middleware (fixtures.KernelMiddlewareLoader).

The MysqlPlainFixtureWriter was adjusted to be able to load larger fixture amounts for the same table/model.

Please check out the examples to see how to migrate from original to new usage.

Other notable changes:

• Fixtures are now loaded after the change history (if the app option is given), to ensure you can also load a predefined history.

• The mysql container for integration tests now specifies max_connection=1000 (original default was 151 as per mysql documentation). This should allow for bigger integration tests and parallel integration tests running more smoothly.

• db.SqlFmt func Signature changed

  • includes a formatValues []any parameter (to behave like fmt.* funcs) and a was renamed to args for better understanding of what the parameter is used for.
  • The ToSql receiver func first performs a fmt.Format on the passed format with the given formatValues (instead of a originally) and passes args back for later usage with e.g. sqlx for letting the db escape the values in a prepared statement to prevent potential sql injection

• There is now a NewAnonymousAuthenticator func in the httpserver/auth package to allow handlers to to database operations even when the user is not authenticated (yet)

• Redis clients are now being provided via the appctx.Provide func instead of via a sync.Mutex locking with local variable cache mechanism.

• Mack and Suzy were renamed to Bob and Alice as their parents did a mistake when filling out the birth papers.

v0.23.0

What's Changed

• currency: add health check to module;
• change history: drop columns;
• replace exp/slices with standard slices;

metric: added calculator and handlers for requests / messages per runner

This release introduces a more powerful version of the "messages per runner" metric from the stream package.
The MPR metric was produced by a kernel module and used to perform auto scaling for SQS consumers.
The whole functionality was implemented and limited to the stream package.\

The feature of doing metric calculations via a kernel module was moved to the metric/calculator package which implements a more generic approach to this. To achieve this, the calculator calls a set of provided handlers periodically. The handlers are returning the calculated metrics which should be published:

type Handler interface {
	GetMetrics(ctx context.Context) (metric.Data, error)
}

By default there are two registered handlers. The stream_messages_per_runner which is the replacement for the old MPR metric and the httpserver_requests_per_runner handler from the httpserver package which calculates the count of requests served by one instance of the server.

Once enabled via metric.calculator.enabled: true the default settings will look similar to:

metric:
    calculator:
        cloudwatch:
            client: "default"
        dynamodb:
            naming:
                pattern: '{env}-{modelId}'
        ecs:
            cluster: '{env}'
            service: '{app_group}-{app_name}'
        enabled: true
        handlers:
            httpserver_requests_per_runner:
                enabled: true
                max_increase_percent: 200
                max_increase_period: 5m0s
                period: 1m0s
                target_value: 100
            stream_messages_per_runner:
                enabled: true
                max_increase_percent: 200
                max_increase_period: 5m0s
                period: 1m0s
                target_value: 100
        leader_election: metric_calculator
        period: 1m0s

What's Changed

• metric: added calculator and handlers for requests / messages per runner by @j4k4 in #1142

Full Changelog: v0.21.6...v0.22.0

httpserver/auth: don't crash on invalid JWT tokens

If we received a JWT token signed with the correct key, but with the wrong payload, we would try to cast the email field of the token to a string. If the field doesn't exist (because the token has the wrong payload), we would then crash the auth flow, causing the request to fail (instead of giving another authentication provider the chance to understand this token). Now we detect this case and return an error, allowing another authentication provider to accept the token.

sql improvements

This release adds two improvements to the db and fixtures package.

DB connection parameters

This release adds the possibility to add custom system variables to the db connection.
In the case of a mysql connection, these parameters will translate into session variables.
The following configuration would result in executing SET foobar = 42; on every new connection.

db:
  default:
    driver: mysql
    parameters:
      foobar: 42

SQLX fixtures

We've added a fixture writer to make it easier to load "db structs" as fixtures.
You can find a full example under gosoline/examples/fixtures/sqlx.

type User struct {
	Id       int    `db:"id"`
	Name     string `db:"name"`
	IsActive bool   `db:"is_active"`
}

var fixtureSets = []*fixtures.FixtureSet{
	{
		Enabled: true,
		Purge:   true,
		Writer:  fixtures.MysqlSqlxFixtureWriterFactory(&fixtures.MysqlSqlxMetaData{TableName: "users"}),
		Fixtures: []interface{}{
			User{
				Id:       1,
				Name:     "Mack",
				IsActive: true,
			},
			User{
				Id:       2,
				Name:     "Suzy",
				IsActive: false,
			},
		},
	},
}

test: reset the log recorder after each test

This change resets the log recorder after each test, so the WithSharedEnvironment option can now be used together with the WithLogRecording option.

v0.21.3: Auth extension

What's Changed

• fix: config UnmarshalKey has issues with default values for slice typ… by @applike-ss in #1127
• fix: blob store requires region to be set when creating the bucket an… by @applike-ss in #1128
• allow restricting available auth options through config by @bt-justtrack in #1139
• httpserver/auth: Added method to JwtTokenHandler to sign claims directly to make it extensible by @ajscholl in #1140

Allow restricting auth options through config

We can now restrict in the config.dist.yml what authentication options are available. The same can be done with env vars, or we can layer env vars over the config.dist.yml to allow again all (eg disallow one method by default and for certain environments it could be turned on again).

In code we specify our auth handler like this:

auth.NewChainHandler(auth.OnlyConfiguredAuthenticators(config, "default", map[string]auth.Authenticator{
		auth.ByApiKey:    apiKeyAuthHandler,
		auth.ByGoogle:    googleAuthHandler,
		auth.ByBasicAuth: emailAuthHandler,
	}))

and restrict them to eg just Google and BasicAuth like this in the config.dist.yml:

httpserver:
  default:
    auth:
      allowedAuthenticators:
        - google
        - basicAuth

All APIKey auths then result in 401 Unauthorized.
This can then be extended to allowing different auth options on different environments through either different config.dist.yml files or env vars.

Full Changelog: v0.21.2...v0.21.3

db: Fix a arg mismatch between ...any and any on db.Client.NamedExec

Fix a arg mismatch between ...any and any on db.Client.NamedExec which caused panics.

db: Add a new JSON type

This new type wraps the underlying types stored in a jsonb column in databases, eliminating the need to have dedicated marshal / unmarshal boilerplate code on converting to / from the db model.

Example usage

type DBModel struct {
    SomeModel db.JSON[SomeModel, db.NonNullable]
    SomeNullableModel db.JSON[*SomeModel, db.Nullable]
}

type SomeModel struct {
    Foo string
    Bar string
}

func NewDBModel(someModel SomeModel) DBModel {
    return DBModel{
        SomeModel: db.NewJSON(someModel, db.NonNullableBehaviour),
        SomeNullableModel: db.NewJSON((*SomeModel)(nil), db.NullableBehaviour),
    }
}

func (dbModel DBModel) ToSomeModel() SomeModel {
    return dbModel.SomeModel.Get()
}

Improve cloud/aws credential handling and cloud/aws/kinesis behavior on empty streams

This release adds support for resuming consumption on Kinesis streams without any messages stored on them (meaning, we resume where we left off and don't consume the whole empty stream again upon restart) by remembering the last shard iterator as a fallback to the last sequence number.

Additionally, this release changes the way we load AWS default credentials slightly as you can now specify a role to be assumed for all services in cloud.aws.defaults.assume_role. During this, the GetCredentialsProvider method was changed to no longer require a config as the second parameter and UnmarshalDefaultCredentials was deleted (it now happens automatically inside UnmarshalClientSettings).