Hypercore

Hypercore is the TimescaleDB hybrid row-columnar storage engine, designed specifically for real-time analytics and powered by time-series data. The advantage of hypercore is its ability to seamlessly switch between row-oriented and column-oriented storage. This flexibility enables Timescale Cloud to deliver the best of both worlds, solving the key challenges in real-time analytics:

High ingest throughput
Low-latency ingestion
Fast query performance
Efficient handling of data updates and late-arriving data
Streamlined data management

Hypercore’s hybrid approach combines the benefits of row-oriented and column-oriented formats in each Timescale Cloud service:

Fast ingest with rowstore: new data is initially written to the rowstore, which is optimized for high-speed inserts and updates. This process ensures that real-time applications easily handle rapid streams of incoming data. Mutability—upserts, updates, and deletes happen seamlessly.
Efficient analytics with columnstore: as the data cools and becomes more suited for analytics, it is automatically converted to the columnstore. This columnar format enables fast scanning and aggregation, optimizing performance for analytical workloads while also saving significant storage space.
Faster queries on compressed data in columnstore: in the columnstore conversion, hypertable chunks are compressed by more than 90%, and organized for efficient, large-scale queries. Combined with chunk skipping, this helps you save on storage costs and keeps your queries operating at lightning speed.
Full mutability with transactional semantics: regardless of where data is stored, hypercore provides full ACID support. Like in a vanilla PostgreSQL database, inserts and updates to the rowstore and columnstore are always consistent, and available to queries as soon as they are completed.

Since TimescaleDB v2.18.0

Hypercore workflow

Best practice for using hypercore is to:

Enable columnstore

Use ALTER TABLE for a hypertable


ALTER TABLE crypto_ticks SET (
   timescaledb.enable_columnstore = true, 
   timescaledb.segmentby = 'symbol');

Use ALTER MATERIALIZED VIEW for a continuous aggregate


ALTER MATERIALIZED VIEW assets_candlestick_daily set (
   timescaledb.enable_columnstore = true, 
   timescaledb.segmentby = 'symbol' );

Add a policy to move chunks to the columnstore at a specific time interval
For example, 7 days after the data was added to the table:
```
CALL add_columnstore_policy('crypto_ticks', after => INTERVAL '7d');
```
See add_columnstore_policy.
View the policies that you set or the policies that already exist
```
SELECT * FROM timescaledb_information.jobs
WHERE proc_name='policy_compression';
```
See timescaledb_information.jobs.

You can also convert_to_columnstore and convert_to_rowstore manually for more fine-grained control over your data.

Limitations

Chunks in the columnstore have the following limitations:

ROW LEVEL SECURITY is not supported on chunks in the columnstore.
To add unique constraints on chunks in the columnstore convert_the chunk to rowstore, add the constraints to your data, then convert the chunk back to the columnstore.
SkipScan does not currently work on chunks in the columnstore.

Hypercore

Hypercore workflow

Limitations

Related Content