Live migration

Live migration is a new migration strategy to move a large amount of data (100 GB-10 TB+) with low downtime (on the order of few minutes). It is significantly more complex to execute than a migration with downtime using pg_dump/restore, but supports more use-cases and has fewer requirements than the dual-write and backfill method.

Important

Be aware that the live migration tooling is currently experimental. Concretely, this means that you may run into the following shortcomings:

• You may experience failure to migrate your database due to incompatibilities between the source and target (e.g. tables with generated columns cannot be replicated).
• Live migration does not yet support mutable compression (INSERT, UPDATE, DELETE on compressed data).
• By default, numeric fields containing NaN/+Inf/-Inf values are not correctly replicated, and will be converted to NULL. A workaround is available, but is not enabled by default.

Should you run into any problems, please open a support request before losing any time debugging issues.

You can open a support request directly from the Timescale console, or by email to [email protected].

Live migration leverages Postgres' built-in replication functionality to provide a seamless migration with very little application downtime.

Note

In the context of migrations, your existing production database is referred to as the "source" database, while the new Timescale database that you intend to migrate your data to is referred to as the "target" database.

Roughly, it consists of four steps:

1. Prepare and create replication slot in source database.
2. Copy schema from source to target, optionally enabling hypertables.
3. Copy data from source to target while capturing changes.
4. Apply captured changes from source to target.

Live migration works well when:

• Large, busy tables have primary keys, or don't have many UPDATE or DELETE statements.
• The insert workload does not exceed 20,000 rows per second, and inserts are batched. If your application exceeds this, you should use the dual-write and backfill migration method.

Live migration has the following shortcomings:

• It does not support schema changes (DDL) during the migration process

For more information, refer to the step-by-step migration guide:

• Live migration from PostgreSQL
• Live migration from TimescaleDB

If you are migrating from AWS RDS to Timescale, you can refer to this playbook for a detailed migration guide.

If you encounter any issues or have questions about live migration, refer to the FAQ section.