v0.11.15

This is a patch release for 0.11 adding opt-in logical type support for Parquet reads and writes.

🚀 Enhancements

• Add opt-in Logical type support to 0.11.x branch by @clairemcginty in #4792

Full Changelog: v0.11.14...v0.11.15

v0.12.8

🚀 Enhancements

• Parquet supports logical types on an opt-in basis (#4772, #4782) @clairemcginty
• Relax transform coder equality (#4780) @RustedBones
• Added more examples on Join types (#4769) @shnapz
• Add java sql Date and Time coder (#4781) @RustedBones
• Add deduplicate option to all AsyncLookupDoFn impl (#4775) @RustedBones

🐛 Bug Fixes

• Ensures guava callback to complete chained futures (#4778) @RustedBones
• Fix skewedJoin chill side with normal join instead of hashJoin (#4787) @RustedBones
• Set provided name on transforms with side outputs (#4779) @RustedBones

🌱 Dependency Updates

• Update mysql-connector-java to 8.0.33 (#4786) @scala-steward
• Update testcontainers-scala-elasticsearch, ... to 0.40.15 (#4785) @scala-steward
• Update mysql-socket-factory to 1.11.1 (#4776) @scala-steward
• Sync joda-time with beam (#4771) @RustedBones
• update elasticsearch7 to 7.17.9 (#4770) @RustedBones
• Update elasticsearch-java to 8.7.0 (#4760) @scala-steward
• Update scalafmt-core to 3.7.3 (#4758) @scala-steward
• Update testcontainers-scala-elasticsearch, ... to 0.40.14 (#4754) @scala-steward

Contributors to this release

@RustedBones, @clairemcginty, @scala-steward and @shnapz

v0.12.7

🚀 Enhancements

• Add support for DML statements to MockBigQuery (#4759) @fallonchen
• Allow F type param in tuple coders (#4767) @RustedBones

🐛 Bug Fixes

• Generate proper getCoderArguments on tuple coders (#4766) @RustedBones
• Fix avro coder regression on custom SpecificCoder (#4764) @RustedBones

🏗️ Build Improvements

• Remove add-to-project GHA due to SSO token issue (#4765) @RustedBones

🌱 Dependency Updates

• Update sbt-mima-plugin to 1.1.2 (#4761) @scala-steward

Contributors to this release

@RustedBones, @fallonchen and @scala-steward

v0.12.6

This release includes an upgrade to Beam 2.46.

🚀 Enhancements

• Add fanout for CMS hot key computation in skewed joins (#4752) @RustedBones
• Delay deprecation of old skewed API (#4755) @RustedBones
• Expose cache in scio-grpc (#4734) @RustedBones
• Add magnolify imports to docs for methods using Funnel (#4739) @kellen
• Distribute CMS hot key set as side input (#4724) @RustedBones
• Support Parquet in BQ ExtractOps (#4728) @clairemcginty
• Neo4j parametrized from existing SCollection (#4719) @sumitsu
• Make SeqExampleSCollectionOps API consistent (#4731) @clairemcginty
• Add skewed job methods with topN and top percent (#4718) @RustedBones
• Modernize scalafix rules (#4687) @RustedBones

🐛 Bug Fixes

• Don't forget to fan out (#4740) @kellen
• Don't re-use Configuration objects in Parquet SMB (#4727) @clairemcginty
• apply extendedErrorInfo input parameter (#4723) @farzad-sedghi

🏗️ Build Improvements

• Fork all tests. Default to java 11 (#4748) @RustedBones
• Factorize java options and propagate GCP settings (#4746) @RustedBones
• Add java 17 configuration for integration tests (#4745) @RustedBones
• Remove test matrix for integration test (#4737) @RustedBones
• Setup MiMa filter for planed breaking changes (#4733) @RustedBones

🌱 Dependency Updates

• Update beam to 2.46.0 (#4744) @RustedBones
• Update mysql-socket-factory to 1.11.0 (#4730) @scala-steward
• Update circe-core, circe-generic, ... to 0.14.5 (#4735) @scala-steward
• Bump actions/add-to-project from 0.4.0 to 0.4.1 (#4729) @dependabot
• Update scalafmt-core to 3.7.2 (#4720) @scala-steward
• Update elasticsearch-java to 8.6.2 (#4715) @scala-steward

Contributors to this release

@RustedBones, @clairemcginty, @dependabot, @dependabot[bot], @farzad-sedghi, @kellen, @scala-steward and @sumitsu

v0.12.5

This release adds support for Java 17 and contains coder bug fixes. It also includes Beam 2.45.0 dependency update.

🚀 Enhancements

• Unique tuple tags by @farzad-sedghi in #4711
• Set default JdkAddOpenModules for Java 17 by @clairemcginty in #4710

🐛 Bug Fixes

• Avoid using outputfield cache by @shnapz in #4701
• Force scio coders on all input SCollection by @RustedBones in #4713
• replace assert with com.google.common.base.Verify by @farzad-sedghi in #4716

📗 Documentation

• Explain Hadoop configuration in Parquet.md by @clairemcginty in #4703
• Update site releases by @RustedBones in #4717

🏗️ Build Improvements

• Add Java 17 to GHA setup by @clairemcginty in #4659
• update smb multi join code gen to the latest by @farzad-sedghi in #4702
• Update sbt-scoverage to 2.0.7 by @scala-steward in #4706
• Update sbt-assembly to 2.1.1 by @scala-steward in #4708

🌱 Dependency Updates

• Update beam to version 2.45.0 by @RustedBones in #4712
• Update circe-core, circe-generic, ... to 0.14.4 by @scala-steward in #4705
• Update mysql-socket-factory to 1.10.0 by @scala-steward in #4704

Contributors to this release

@RustedBones, @clairemcginty, @farzad-sedghi, @scala-steward and @shnapz

Full Changelog: v0.12.4...v0.12.5

v0.12.4

🚀 Enhancements

• Set BQ staging max time travel to minimum 2 days by @shnapz in #4697

🐛 Bug Fixes

• Fix SingletonCoder serialization by @RustedBones in #4691
• Unwrap internal SCollection coders to be reused in transforms by @RustedBones in #4690
• Make Parquet-SplittableDoFn opt-in, not opt-out by @clairemcginty in #4700

📗 Documentation

• Update 0.12 migration with lookup do fn by @RustedBones in #4695
• Document ScioIO methods, fixes #3937 by @kellen in #4689
• Update SBF docs, fixes #4528 by @kellen in #4693

🌱 Dependency Updates

• Update scalafmt-core to 3.7.1 by @scala-steward in #4682
• Update sbt-jmh to 0.4.4 by @scala-steward in #4688
• Use slf4j version from beam by @RustedBones in #4696

Full Changelog: v0.12.3...v0.12.4

v0.12.3

🐛 Bug Fixes

• ParquetBucketMetadata should not write null secondary keys by @clairemcginty in #4684

🌱 Dependency Updates

• Update mysql-socket-factory to 1.9.0 by @scala-steward in #4681
• Update elasticsearch-java to 8.6.1 by @scala-steward in #4680
• Update sbt-mdoc to 2.3.7 by @scala-steward in #4685

Full Changelog: v0.12.2...v0.12.3

v0.12.2

This release contains a bugfix and a bump to Beam 2.44.0.

🚀 Enhancements

• scalafix for the rest of the BQ API changes by @farzad-sedghi in #4625
• Improve style of REPL test by @shnapz in #4629

🐛 Bug Fixes

• Fix SMB producer compatibility issues by @clairemcginty in #4667
• Respecting required field of inner case classes in Coder macro by @shnapz in #4645
• fix bq lib import for scalafix by @farzad-sedghi in #4666

📗 Documentation

• Update Parquet docs to add example of case-class predicates by @clairemcginty in #4658

🌱 Dependency Updates

• Update beam to 2.44 by @RustedBones in #4668
• Update featran-core, featran-scio, ... to 0.8.0 by @scala-steward in #4661
• Update elasticsearch-java to 8.6.0 by @scala-steward in #4651
• Update elasticsearch-java to v7.17.8 by @RustedBones in #4649
• Update sbt-bloop to 1.5.6 by @scala-steward in #4631
• Update sbt-assembly to 2.1.0 by @scala-steward in #4632
• Update mysql-socket-factory to 1.8.3 by @scala-steward in #4660
• Update mysql-connector-java to 8.0.32 by @scala-steward in #4662
• Update sbt-tpolecat to 0.4.2 by @scala-steward in #4646
• Update sbt, sbt-dependency-tree to 1.8.2 by @scala-steward in #4647
• Update scalactic to 3.2.15 by @scala-steward in #4648
• Update neo4j-java-driver to 4.4.11 by @scala-steward in #4637
• Update scalafmt-core to 3.7.0 by @scala-steward in #4663
• Update sparkey to 3.2.5 by @scala-steward in #4654
• Update magnolia to 1.1.3 by @scala-steward in #4656
• Update circe-core, circe-generic, ... to 0.14.3 by @scala-steward in #4524
• Update tensorflow to 0.4.2 by @RustedBones in #4669
• Update breeze to 2.1.0 by @RustedBones in #4670
• Update cats-core to 2.9.0 by @RustedBones in #4671
• Update scalatest to 3.2.15 by @RustedBones in #4672
• Update algebra to 2.9.0 by @RustedBones in #4673

Full Changelog: v0.12.1...v0.12.2

v0.11.14

🚀 Enhancements

• Added SA impersonation for BigQuery client (#4616)
• Exposed SMB ExternalSorter tempLocation as System Property (#4618)

🐛 Bug Fixes

• Fix SMB metadata compatibility issues (#4617)

🌱 Dependency Updates

• Update pprint to 0.8.1 (#4613)
• Update sbt-ci-release to 1.5.11 (#4559)
• Update elasticsearch-java to 8.5.2 (#4610)

Full Changelog: v0.11.13...v0.11.14

v0.12.1

This release contains mostly enhancements, dependency upgrades and bug fixes. It includes Beam 2.43.0.

🚀 Enhancements

• PubsubIO: expose more params & fix passing of write params (#4615) @Illedran
• Exposed ExternalSorter tempLocation as System Property (#4618) @shnapz
• Added SA impersonation for BigQuery client (#4616) @shnapz

🐛 Bug Fixes

• Fix consistent casing for jlist (#4611) @RustedBones
• Fix SMB metadata compatibility issues (#4617) @clairemcginty
• Scalafix 0.12.0 bug fixes (#4581) @farzad-sedghi
• Do not wrap already nullable coders (#4603) @RustedBones

📗 Documentation

• Detailed release notes for 0.12.0 release (#4599) @clairemcginty
• update Beam version matrix (#4579) @clairemcginty
• update release manual for scalafix rules (#4582) @farzad-sedghi

🏗️ Build Improvements

• Bump google-github-actions/auth from 0 to 1 (#4594) @dependabot
• Bump google-github-actions/setup-gcloud from 0 to 1 (#4593) @dependabot
• Add manually-triggerable GHA workflow for publishing GH site (#4580) @clairemcginty
• set project ID env for publish-gh-site GHA workflow (#4583) @clairemcginty

🌱 Dependency Updates

• Update beam to v2.43.0 (#4607) @RustedBones
• Update scalafmt-core to 3.6.1 (#4589) @scala-steward
• Update scala-collection-compat to 2.9.0 (#4614) @scala-steward
• Update neo4j-java-driver to 4.4.10 (#4624) @scala-steward
• Update sbt-bloop to 1.5.5 (#4621) @scala-steward
• Update sbt-ci-release to 1.5.11 (#4559) @scala-steward
• Update joda-time to 2.12.2 (#4623) @scala-steward
• Update testcontainers-scala-elasticsearch, ... to 0.40.12 (#4622) @scala-steward
• Update pprint to 0.8.1 (#4613) @scala-steward
• Update elasticsearch-java to 8.5.2 (#4610) @scala-steward
• Update sbt-scalafmt to 2.5.0 (#4601) @scala-steward
• Update sbt-header to 5.9.0 (#4606) @scala-steward
• Update sbt-scoverage to 2.0.6 (#4602) @RustedBones
• Update sbt, sbt-dependency-tree to 1.8.0 (#4600) @scala-steward
• Update sbt-mdoc to 2.3.6 (#4574) @scala-steward
• Update sbt-java-formatter to 0.8.0 (#4562) @scala-steward
• Update scalactic to 3.2.14 (#4573) @scala-steward
• Update mysql-socket-factory to 1.7.2 (#4586) @scala-steward
• Update commons-text to 1.10.0 (#4569) @scala-steward
• Update mysql-connector-java to 8.0.31 (#4566) @scala-steward
• Update sbt-paradox to 0.10.3 (#4561) @scala-steward
• Update sbt-assembly to 2.0.0 (#4558) @scala-steward
• Update cassandra-driver-core to 3.11.3 (#4556) @scala-steward
• Remove scala-xml override (#4612) @RustedBones

Contributors to this release

@Illedran, @RustedBones, @clairemcginty, @dependabot, @dependabot[bot], @farzad-sedghi, @scala-steward and @shnapz