3.2.0

This version introduces performance improvements and Azure EventHubs' OAuth2 support.

Changelog

Bump transitive jnr-posix to 3.1.8 (#419)
Prevent Kafka sink from blocking (#418)
Allow setting size limit on line and header length (#417)
Add debug logging and timeout configurations (#417)
Add timeout for body parsing (#417)
Expand default time limit (#417)
Log cats-effect warning at debug level (#414)
collector-kafka: authenticate with Event Hubs using OAuth2 (#401)
Add snowman job for tag builds
Cross compile to scala 2.12

3.1.2

This version brings an improvement to handling cookie bounce Location header for relative URIs.

Changelog

Improve relative redirect in cookie bounce feature (#413)

3.1.1

This release addresses a behavioural change for handling case-sensitive cookie header.

Changelog

Upcase cookie header name (#412)

3.1.0

This release adds support for HTTP Strict Transport Security (HSTS).

The HSTS header ensures that websites or services are not accessed via HTTP when HTTPS (the more secure protocol) is available.

This is most relevant for URLs that users might type into their browser — if they start with http:// by mistake, there is a risk that a prepared attacker impersonates the site and steals user data. Because users don’t typically access Collector endpoints directly, there is no significant attack vector. However, it’s a good security practice to use HSTS, and it helps pass automated security checks on the infrastructure.

If your Collector is behind a reverse proxy like Nginx, then you would have already been able to configure the HSTS header there. But if you are using something like an AWS or GCP load balancer, it can be convenient to send this header from the app itself, to be passed through by the load balancer.

The configuration looks like this:

hsts {
  enable = true
  maxAge = 365 days
}

This setting is off by default. Otherwise it could be problematic for users not exposing HTTPS endpoints.

You can find the relevant configuration properties in the docs for Collector.

Changelog

• Add an option to send HSTS header (#409)

3.0.1

This release is a minor fix release affecting a situation where events would be tracked as failed due to particular tracking setup and unintended behavioral change.

3.0.0

This is the first release of a new major collector version series: 3.0.0. It features an updated HTTP stack, a new license, unified configuration and a much more robust test harness.

The main change compared to the previous version (2.10) is that we moved from Akka HTTP to http4s as our HTTP framework.

The version of Akka we were using no longer receives security patches; subsequent versions of Akka are only available under a commercial license from Lightbend. http4s has no such limitations, as it’s released under the Apache 2.0 license. Moreover, we have been using http4 in all other http-enabled services (e.g. Enrich). Having the same framework across our codebase helps us share code between applications and develop faster.

We’ve strived to maintain full compatibility between 2.x and 3.x versions of the Collector, but there are still a few adjustments you will need to make to your configuration. See the upgrade guide for more details.

Note that per our recent licensing announcement, Snowplow Collector 3.0.0 is released under the Snowplow Limited Use License Agreement.

Changelog

Add mandatory SLULA license acceptance flag (#405)
Remove unused warmup config section
Use shortname for collector name (#403)
Add statsd metrics reporting (#404)
Add support for Do Not Track cookie (#400)
Add crossdomain.xml support (#399)
Add http root response (#397)
Deploy 2.13 scala assets to GH on CI (#392)
Use correct sqs buffer queue name with Kinesis bad sink (#393)
Sbt project modernization (#361)
Update the Pubsub UserAgent format (#362)
Add separate good/bad sink configurations (#388)
Add Kafka sink healthcheck (#387)
Make maxConnections and idleTimeout configurable (#386)
Add support for handling /robots.txt (#385)
Set installation id (#384)
Set maxBytes in the NsqSink (#383)
Add http4s Kafka support (#382)
Add http4s NSQ support (#348)
Add telemetry support (#381)
Use Blaze as default http4s backend (#380)
Add http4s SQS sink (#378)
Add http4s Kinesis sink (#379)
Add iglu routes spec (#377)
Add http4s PubSub sink (#376)
Add http4s SSL support (#374)
Add http4s redirect support (#373)
Load config (#326)
Add http4s anonymous tracking (#372)
Add http4s CORS support (#371)
Add http4s pixel endpoint (#370)
Add http4s GET and HEAD endpoints (#369)
Configure set-cookie header (#368)
Add test for the stdout sink (#367)
Add http4s POST endpoint (#366)
Add http4s graceful shutdown (#365)
Add http4s module (#364)
Add Snowplow Limited Use License (#346)

2.10.0

This release bumps dependencies for potential security vulnerabilities. Also, it updates Pubsub user agent format.

Changelog

• Update the Pubsub UserAgent format (#362)
• Bump sbt-snowplow-release to 0.3.1 (#363)

Version 2.9.2

Patch release adding optional statsd metrics tags to match the settings available in enrich and loaders.

CHANGELOG

Add ability to set custom tagset for statsd metrics (#340)

Version 2.9.1

Patch release fixing a bug for Kinesis collector running with SQS buffer, where in case of Kinesis issue, events would get sent to SQS forever, never switching back to Kinesis once available again.

PubSub collector got less verbose in the INFO logs.

CHANGELOG

• Run background check for Kinesis if it is made unhealthy and SQS buffer is activated (#315)
• PubSub: use debug for logging the inserts (#321)

Version 2.9.0

The primary reason for this release is to update the logic for /sink-health endpoint. It also comes with other nice improvements.

What's new ?

Improved logic for /sink-health endpoint (#276)

Until this release, the collector was starting with /sink-health endpoint healthy, until proven unhealthy (which was a problem in case of Kinesis outage for instance). The new logic is to start unhealthy until proven healthy, either by successfully sending events to the queue or by successfully checking the stream with AWS/GCP API, whichever comes first.

More details can be found on the related issue.

Kinesis and SQS: MaxRetries configurable (#295)

MaxRetries is used in 2 scenarios:

• In case of SQS collector or Kinesis collector without SQS buffer, /sink-health becomes unhealthy after this number of failed retries (even though it is retried forever).
• In case of Kinesis collector with SQS buffer, this is the number of retries to Kinesis before switching to SQS.

It was hard coded to 3 and is now configurable (here for Kinesis and here for SQS).

Pub/Sub: Pub/Sub emulator support (#270)

It is now possible to use Pub/Sub emulator (e.g. for testing).

Using it is as simple as running the Docker image and setting PUBSUB_EMULATOR_HOST.

Pub/Sub: second layer of retry (#304)

There is now a second layer of retry for inserting records into Pub/Sub, in addition to the built-in retry mechanism of Pub/Sub publisher. When a record can't be inserted after the built-in retries, it's added to a buffer and every retryInterval, the collector retries to insert the failed inserts.

MaxBytes configurable (#272)

In case a collector payload is too big, a size violation bad row is emitted.

The maximum size for each type of queue was hard coded (1 MB for Kinesis, 192 KB for SQS, 10 MB for Pub/Sub 1 MB for Kafka).

It is now configurable (Kinesis, SQS, Pub/Sub, Kafka).

CHANGELOG

• Bump protobuf-java to 3.21.7 (#308)
• PubSub: add second layer of retry (#304)
• Replace set-output in Github Actions (#305)
• Make MaxRetries configurable (#295)
• Update /sink-health endpoint logic (#276)
• Integration tests should work with both regular and distroless project (#301)
• Scan Docker images with Snyk container monitor in deploy.yml (#296)
• Add integration tests for Set-Cookie (#287)
• Add integration test for doNotTrackCookie (#289)
• Add integration test for X-Forwarded-For (#288)
• Add integration test for custom paths (#286)
• Add integration test for /health endpoint (#285)
• Kinesis: add integration tests (#283)
• Validate cookie.fallbackDomain config option on startup (#278)
• PubSub: add integration tests (#274)
• PubSub: make it possible to use PubSub emulator (#270)
• Put MaxBytes in default application.conf instead of hard coding it (#272)