Live migration is a new migration strategy to move a large amount of data (100 GB-10 TB+) with low downtime (on the order of few minutes). It is significantly more complex to execute than a migration with downtime using pg_dump/restore, but supports more use-cases and has fewer requirements than the dual-write and backfill method.
Important
Be aware that the live migration tooling is currently experimental. Concretely, this means that you may run into the following shortcomings:
- You may experience failure to migrate your database due to incompatibilities between the source and target (e.g. tables with generated columns cannot be replicated).
- Live migration does not yet support mutable compression (
INSERT
,UPDATE
,DELETE
on compressed data). - By default, numeric fields containing
NaN
/+Inf
/-Inf
values are not correctly replicated, and will be converted toNULL
. A workaround is available, but is not enabled by default.
Should you run into any problems, please open a support request before losing any time debugging issues.
You can open a support request directly from the Timescale console, or by email to [email protected].
Live migration leverages Postgres' built-in replication functionality to provide a seamless migration with very little application downtime.
Note
In the context of migrations, your existing production database is referred to as the "source" database, while the new Timescale database that you intend to migrate your data to is referred to as the "target" database.
Roughly, it consists of four steps:
- Prepare and create replication slot in source database.
- Copy schema from source to target, optionally enabling hypertables.
- Copy data from source to target while capturing changes.
- Apply captured changes from source to target.
Live migration works well when:
- Large, busy tables have primary keys, or don't have many
UPDATE
orDELETE
statements. - The insert workload does not exceed 20,000 rows per second, and inserts are batched. If your application exceeds this, you should use the dual-write and backfill migration method.
Live migration has the following shortcomings:
- It does not support schema changes (DDL) during the migration process
For more information, refer to the step-by-step migration guide:
If you are migrating from AWS RDS to Timescale, you can refer to this playbook for a detailed migration guide.
If you encounter any issues or have questions about live migration, refer to the FAQ section.
Keywords
Found an issue on this page?
Report an issue!