diff --git a/docs/source/configuration.rst b/docs/source/configuration.rst index 70da683..fb8227c 100644 --- a/docs/source/configuration.rst +++ b/docs/source/configuration.rst @@ -331,15 +331,15 @@ The optional ``renames`` property lists the item columns to rename along the mig Savepoints ---------- -When migrating data from Apache Cassandra or DynamoDB, the migrator is able to resume an interrupted migration. To achieve this, it stores so-called “savepoints” along the process to remember which data items have already been migrated and should be skipped when the migration is restarted. +When migrating data from Apache Cassandra or DynamoDB, the migrator is able to :doc:`resume an interrupted migration `. To achieve this, it stores so-called “savepoints” along the process to remember which data items have already been migrated and should be skipped when the migration is restarted. .. code-block:: yaml savepoints: - # Whe should savepoint configurations be stored? This is a path on the host running + # Where should savepoint configurations be stored? This is a path on the host running # the Spark driver - usually the Spark master. path: /app/savepoints - # Interval in which savepoints will be created + # Interval at which savepoints will be created intervalSeconds: 300 ---------- diff --git a/docs/source/getting-started/index.rst b/docs/source/getting-started/index.rst index de1e2d7..1def847 100644 --- a/docs/source/getting-started/index.rst +++ b/docs/source/getting-started/index.rst @@ -46,6 +46,7 @@ You might also be interested in the following extra features: * :doc:`rename columns along the migration `, * :doc:`replicate changes applied to the source table after the initial snapshot transfer has completed `, +* :doc:`resume an interrupted migration where it left off `, * :doc:`validate that the migration was complete and correct `. .. toctree:: diff --git a/docs/source/index.rst b/docs/source/index.rst index bfea795..3679c11 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -33,6 +33,7 @@ Migrator Spark Scala run-the-migration stream-changes rename-columns + resume-interrupted-migration validate configuration tutorials/index diff --git a/docs/source/resume-interrupted-migration.rst b/docs/source/resume-interrupted-migration.rst new file mode 100644 index 0000000..60de10b --- /dev/null +++ b/docs/source/resume-interrupted-migration.rst @@ -0,0 +1,13 @@ +================================================= +Resume an Interrupted Migration Where it Left Off +================================================= + +.. note:: This feature is currently supported only when migrating from Apache Cassandra or DynamoDB. + +If, for some reason, the migration is interrupted (e.g., because of a networking issue, or if you need to manually stop it for some reason), the migrator is able to resume it from a “savepoints”. + +Savepoints are configuration files that contain information about the already migrated items, which can be skipped when the migration is resumed. The savepoint files are automatically generated during the migration. To use a savepoint, start a migration using it as configuration file. + +You can control the savepoints location and the interval at which they are generated in the configuration file under the top-level property ``savepoints``. See `the corresponding section of the configuration reference `_. + +During the migration, the savepoints are generated with file names like ``savepoint_xxx.yaml``, where ``xxx`` is a timestamp looking like ``1234567890``. To resume a migration, start a new migration with the latest savepoint as configuration file.