Releases: GoogleCloudPlatform/spanner-migration-tool
Releases · GoogleCloudPlatform/spanner-migration-tool
v3.4.0
VERSION(3.4.0)
New Features/Improvements
- Support for FLOAT32 type in SMT.
Support for migration of spanner supported Foreign key actions from source database(MySQL and PostgreSQL) to spanner. - Support for custom transformation in forward migration and reverse replication pipelines(documentation).
Bug Fixes
- Fix for making GenerateIdSuffix thread safe
- Updated the Makefile to include
npm install - Updates for translating numeric primary keys to strings in the case of the PostgreSQL dialect in Spanner, as it doesn’t support numeric primary keys.
Documentation Updates
- Documentation for custom transformation.
Dependency Updates
- Upgraded pg_query to v5
General Changes
Please refer to the Spanner migration tool README for further details.
v3.3.0
VERSION(3.3.0)
New Features/Improvements
- Support for automatic migration of Sequences and UUID (documentation) in SMT has been added
- Support to configure metadata gcs bucket manually has been added
Bug Fixes
- Integration test fix for issue with loading sqlplus
- For orchestration and schema parallel tasks, queued tasks are now always completed even if one of the parallel task fails
- Fix for when changing order of index columns on Spanner side, it also changes the order of columns on the MySQL side
- Added validators for Index and Database names in UI
- Fix for error during column name modification
Documentation Updates
- Documentation for Autogen Keys
- Documentation for GCS Bucket fine tuning
Dependency Updates
No dependency updates
General Changes
Please refer to the Spanner migration tool README for further details.
v3.2.5
VERSION(3.2.5)
New Features/Improvements
- JSON based resource generation. Datastream connection profiles can now be created in batch by specifying the JSON configuration via both the CLI and the UI.
- Introduced dataflow-template flag. Gives the ability to override the version of the DatastreamToSpanner Dataflow template run by SMT for the migration.
- Change endpoint of the DatastreamToSpanner Dataflow template to point to the latest released tag.
- Updated version of the writer job for reverse replication in the reverse replication launcher script.
Bug Fixes
- Fixed ID to name mapping for CSV table migration.
Documentation Updates
- Added documentation for –dataflow-template flag added for schema, data and web commands. - TBD
- Add documentation on Dataflow firewall configuration in reverse replication user guide.
Dependency Updates
No dependency updates
General Changes
Please refer to the Spanner migration tool README for further details.
v3.2.4
VERSION(3.2.4)
New Features/Improvements
- Introduced migration project id. Gives the ability to have spanner in a separate project than Dataflow. For both Live migration (documentation link) and reverse replication (documentation link)
Bug Fixes
- No major bug fixes
Documentation Updates
- Added documentation for –project flag added for schema, data and schema-and-data commands. - Link
Dependency Updates
- No dependency updates
Please refer to the Spanner migration tool Documentation for further details.
v3.2.3
VERSION(3.2.3)
New Features/Improvements
- Create source and target profiles based on json configuration passed in the command line.
- The code was refactored, and accessors were added to enhance the overall code quality.
- Improved the unit test coverage.
- Update DatastreamToSpanner template to 2024-03-22-02_RC00 version.
Documentation Updates
- Added documentation for migration_shard_id - Link
- Updated documentation for Reverse Replication - Link
Dependency Updates
- Angular update to v17.2.0
Please refer to the Spanner migration tool Documentation for further details.
v3.2.2
VERSION(3.2.2)
New Features/Improvements
- Added basic rate limiting for DataStream and DataFlow calls to avoid quota issues during large sharded migrations
- Added idempotent retries for data stream creation calls
Documentation Updates
- Added documentation for Reverse Replication - Link
Please refer to the Spanner migration tool Documentation for further details.
v3.2.1
VERSION(3.2.1)
New Features/Improvements
- Support cleanup command via the CLI to delete generated resources, this command is currently not available on gCloud.
- Add IAM based authentication for CloudSQL MySQL and PostgreSQL databases
- UI improvements to display shard level generated resources during minimal downtime migrations in schema-and-data, data modes
- MySQL BIT(1) data type now maps to BOOLEAN instead of BYTES
- Enhance error reporting for data-only migration
Bug Fixes
- Fixed issues while validating existing spanner database schema in data-only mode for PostgreSQL dialect of Cloud Spanner
- Validate only tables, columns and primary key for data-only migration
Documentation Updates
- Added documentation for Monitoring Dashboards - Link
- Added documentation for running SMT on Cloud shell - Link
Please refer to the Spanner migration tool Documentation for further details.
v3.2.0
VERSION(3.2.0)
New Features/Improvements
- Add monitoring dashboard support for sharded and non-sharded migration
- Added heartbeat to the UI to check backend service health
- Added support for Datastream and GCS Tuning
- Angular Version updated to v16
- Added support for Postgresql to include objects config for datastream job
Please refer to the updated documentation for more details.
Bug Fixes
- During sharded migration configuration, setting shard source details after dataflow config overrides the set dataflow config
- Fix for using correct ProjectId when shared VPC is used
Documentation Updates
- Documentation updates on how to rerun for regular flow and for reprocessing DLQ directory flow
- Permissions Documentation for using Monitoring Dashboards
- Documentation for datastream and gcs tuning
Dependency Updates
cloud.google.com/go/monitoring added v1.16.1
Please refer to the Spanner migration tool Documentation for further details.
v3.1.3
VERSION(3.1.3)
New Features/Improvements
Spanner Migration Tool will switch to using pub/sub notifications from GCS buckets (to discover new objects) from the earlier poll mode for enhanced scalability.
Creation of pub/sub topics will be orchestrated by Spanner migration tool Additional permissions required for this have been documented
Please refer to the updated documentation for more details.
Bug Fixes
N/A
Documentation Updates
Documentation updates regarding the new pub/sub based dataflow jobs.
Dependency Updates
TiDB (v6.5.5)
General Changes
Please refer to the Spanner migration tool Documentation for further details.
v3.1.2
VERSION(3.1.2)
New Features/Improvements
Added Support for:
- DataFlow Configuration for Workers and Service Accounts
- Downloading json after migration starts for sharded migrations
- Displaying DataFlow gcloud cli equivalent for forward migrations
- Redesign of Assessment report.
- GCS Bucket created by SMT will be created in the same region as Spanner instance for improved data residency.
Bug Fixes
- Generated columns cannot be used for sharding id (#651)
- Process SQLServer CDC tables and UI bug fixes (#640)
- Dump file migration issue for PostgreSQL dialect (#645)
- Spanner instance length (#624)
Documentation Updates
- Revamp of SMT documentation
- Additions to troubleshooting section, reverse replication documentation.
Dependency Updates
- Postcss (8.4.31)
- Angular (16.2.5)
- Golang(v0.17.0)
- Spanner Emulator (v1.5.11)
General Changes
Please refer to the Spanner migration tool Documentation for further details.