Timescale Cloud scales PostgreSQL to ingest and query vast amounts of live data. Timescale Cloud provides a range of features and optimizations that supercharge your queries while keeping the costs down. For example:

  • The Hypercore row-columnar engine makes queries up to 350x faster, ingests 44% faster, and reduces storage by 90%.
  • Tiered storage seamlessly moves your data from high performance storage for frequently accessed data to low cost bottomless storage for rarely accessed data.

The following figure shows how Timescale Cloud optimizes your data for superfast real-time analytics and reduced costs:

Main features and tiered data

This page shows you how to rapidly implement the features in Timescale Cloud that enable you to ingest and query data faster while keeping the costs low.

To follow the steps on this page:

Time-series data represents how a system, process, or behavior changes over time. Hypertables are PostgreSQL tables that help you improve insert and query performance by automatically partitioning your data by time. Each hypertable is made up of child tables called chunks. Each chunk is assigned a range of time, and only contains data from that range. You can also tune hypertables to increase performance even more.

Main features and tiered data

Hypertables exist alongside regular PostgreSQL tables. You use regular PostgreSQL tables for relational data, and interact with hypertables and regular PostgreSQL tables in the same way.

This section shows you how to create regular tables and hypertables, and import relational and time-series data from external files.

  1. Import some time-series data into your hypertable

    1. Unzip

      to a <local folder>.

      This test dataset contains second-by-second trade data for the most-traded crypto-assets and a regular table of asset symbols and company names.

      To import up to 100GB of data directly from your current PostgreSQL-based database, migrate with downtime using native PostgreSQL tooling. To seamlessly import 100GB-10TB+ of data, use the live migration tooling supplied by Timescale. To add data from non-PostgreSQL data sources, see Import and ingest data.

    2. Upload data from the CSVs to your service:

    To more fully understand how hypertables work, and how to optimize them for performance by tuning chunk intervals and enabling chunk skipping, see the hypertables documentation.

  2. Have a quick look at your data

    You query hypertables in exactly the same way as you would a relational PostgreSQL table. Use one of the following SQL editors to run a query and see the data you uploaded:

    • Data mode: write queries, visualize data, and share your results in Timescale Console for all your Timescale Cloud services.
    • SQL editor: write, fix, and organize SQL faster and more accurately in Timescale Console for a Timescale Cloud service.
    • psql: easily run queries on your Timescale Cloud services or self-hosted TimescaleDB deployment from Terminal.
    SELECT * FROM crypto_ticks srt
    WHERE symbol='ETH/USD'
    ORDER BY time DESC
    LIMIT 10;
    -- Output

Aggregation is a way of combing data to get insights from it. Average, sum, and count are all examples of simple aggregates. However, with large amounts of data, aggregation slows things down, quickly. Continuous aggregates are a kind of hypertable that is refreshed automatically in the background as new data is added, or old data is modified. Changes to your dataset are tracked, and the hypertable behind the continuous aggregate is automatically updated in the background.

Reduced data calls with Continuous Aggregates

You create continuous aggregates on uncompressed data in high-performance storage. They continue to work on data in the columnstore and rarely accessed data in tiered storage. You can even create continuous aggregates on top of your continuous aggregates.

You use time buckets to create a continuous aggregate. Time buckets aggregate data in hypertables by time interval. For example, a 5-minute, 1-hour, or 3-day bucket. The data grouped in a time bucket uses a single timestamp. Continuous aggregates minimize the number of records that you need to look up to perform your query.

This section shows you how to run fast analytical queries using time buckets and continuous aggregates in Timescale Console. You can also do this using psql.

To see the change in terms of query time and data returned between a regular query and a continuous aggregate, run the query part of the continuous aggregate ( SELECT ...GROUP BY day, symbol; ) and compare the results.

Hypercore is the TimescaleDB hybrid row-columnar storage engine, designed specifically for real-time analytics and powered by time-series data. The advantage of Hypercore is its ability to seamlessly switch between row-oriented and column-oriented storage. This flexibility enables Timescale Cloud to deliver the best of both worlds, solving the key challenges in real-time analytics.

Move from rowstore and columstore in hypercore

When you convert chunks from the rowstore to the columnstore, multiple records are grouped into a single row. The columns of this row hold an array-like structure that stores all the data. Because a single row takes up less disk space, you can reduce your chunk size by more than 90%, and can also speed up your queries. This saves on storage costs, and keeps your queries operating at lightning speed.

Best practice is to compress data that is no longer needed for highest performance queries, but is still accessed regularly. For example, last week's market data.

  1. Enable Hypercore on a hypertable

    Create a job that automatically moves chunks in a hypertable to the columnstore at a specific time interval.

    ALTER TABLE crypto_ticks SET (
    timescaledb.enable_columnstore = true,
    timescaledb.segmentby = 'symbol');

    You segmentby to speed up queries.

  1. Add a policy to convert chunks to the columnstore at a specific time interval

    For example, yesterday's data:

    CALL add_columnstore_policy('crypto_ticks', after => INTERVAL '1d');

    See add_columnstore_policy.

  2. View your data space saving

    When you convert data to the columnstore, as well as being optimized for analytics, it is compressed by more than 90%. This saves on storage costs and keeps your queries operating at lightning speed. To see the amount of space saved:

    SELECT
    pg_size_pretty(before_compression_total_bytes) as before,
    pg_size_pretty(after_compression_total_bytes) as after
    FROM hypertable_compression_stats('crypto_ticks');

    You see something like:

    BeforeAfter
    32 MB3808 KB

In the previous sections, you used continuous aggregates to make fast analytical queries, and compression to reduce storage costs on frequently accessed data. To reduce storage costs even more, you create tiering policies to move rarely accessed data to the object store. The object store is low-cost bottomless data storage built on Amazon S3. However, no matter the tier, you can query your data when you need. Timescale Cloud seamlessly accesses the correct storage tier and generates the response.

Tiered storage

Data tiering is available in the scale and enterprise pricing plans for Timescale Cloud.

To set up data tiering:

  1. Enable data tiering

    1. In Timescale Console, select the service to modify.

      You see the Overview section.

    2. Scroll down, then click Enable tiered storage.

      Enable tiered storage

      When tiered storage is enabled, you see the amount of data in the tiered object storage.

  2. Set the time interval when data is tiered

    In Timescale Console, click SQL Editor, then enable data tiering on a hypertable with the following query:

    SELECT add_tiering_policy('assets_candlestick_daily', INTERVAL '3 weeks');
  3. Query tiered data

    You enable reads from tiered data for each query, for a session or for all future sessions. To run a single query on tiered data:

    1. Enable reads on tiered data:
      set timescaledb.enable_tiered_reads = true
    2. Query the data:
      SELECT * FROM crypto_ticks srt LIMIT 10
    3. Disable reads on tiered data:
      set timescaledb.enable_tiered_reads = false;
      For more information, see Querying tiered data.

By default, all Timescale Cloud services have rapid recovery enabled. However, if your app has very low tolerance for downtime, Timescale Cloud offers High Availability (HA) replicas. HA replicas are exact, up-to-date copies of your database hosted in multiple AWS availability zones (AZ) within the same region as your primary node. HA replicas automatically take over operations if the original primary data node becomes unavailable. The primary node streams its write-ahead log (WAL) to the replicas to minimize the chances of data loss during failover.

Move from rowstore and columstore in hypercore

High availability is available in the scale and enterprise pricing plans for Timescale Cloud.

  1. In Timescale Console, select the service to enable replication for.

  2. Click Operations, then select High availability.

  3. Choose your replication strategy, then click Change configuration.

    Creating a service replica in Timescale Cloud

  4. In Change high availability configuration, click Change config.

For more information, see High availability.

What next? See the use case tutorials, interact with the data in your Timescale Cloud service using your favorite programming language, integrate your Timescale Cloud service with a range of third-party tools, plain old Use Timescale, or dive into the API.

Found an issue on this page?Report an issue or Edit this page in GitHub.