InfluxDB hybrid hot cold storage enables you to keep recent, high‑frequency time‑series data in fast local storage while moving older blocks to Amazon S3 for economical long‑term retention. By leveraging tiered compaction, you can automate the promotion and demotion of data based on age, query patterns, and cost targets. This approach reduces storage expenses without sacrificing the sub‑second response times required for real‑time monitoring dashboards.
InfluxDB Hybrid Hot‑Cold Storage Fundamentals
Time‑series workloads exhibit a clear access pattern: recent points are queried frequently for alerting and dashboards, whereas older data is mainly used for trend analysis or compliance. Storing every point on high‑performance disks quickly becomes cost‑prohibitive. A hot‑cold split separates the working set (hot) from the archive (cold), allowing you to tune each tier for its specific latency and durability requirements.
Why InfluxDB 2.x Needs Tiered Storage
InfluxDB 2.x stores data in a structured merge tree (TSM) that is optimized for write‑heavy ingest but still retains all partitions on local disk. As retention periods grow, disk usage can outpace budget limits, especially when using SSDs for performance. Offloading aged partitions to object storage like Amazon S3 frees expensive local space while preserving queryability through the engine’s remote‑read capabilities.
Core Components of the Hybrid Architecture
The hybrid design consists of three layers: the InfluxDB instance handling hot data, an Amazon S3 bucket configured with lifecycle rules for cold data, and a tiered compaction process that moves data between layers. The hot layer uses provisioned SSDs or memory‑optimized instances to sustain high ingest rates. The cold layer relies on S3 Standard‑IA or Glacier Deep Archive for low‑cost storage. Tiered compaction rewrites older TSM files into a format suitable for S3 and updates the manifest so queries can seamlessly span both tiers.
Hot Storage: InfluxDB Engine and Memory‑Optimized Settings
For the hot tier, enable the memory‑limit setting in the InfluxDB configuration to reserve a portion of RAM for the in‑memory index, reducing disk seeks. Set max‑series‑per‑database to a value that matches your cardinality expectations, and tune wal‑fsync‑delay to balance durability with throughput. Use provisioned IOPS SSD (gp3) with at least 3 000 IOPS to handle burst writes during traffic spikes.
Cold Storage: Amazon S3 Lifecycle and Tiered Compaction
Create an S3 bucket with a lifecycle rule that transitions objects to S3 Standard‑IA after 30 days and to Glacier Deep Archive after 365 days. In InfluxDB, enable the remote‑store endpoint pointing to this bucket and configure the compaction‑tier parameter to “cold”. The tiered compaction job reads local TSM files older than the hot threshold, rewrites them as S3‑optimized objects, and updates the shard metadata so future queries fetch the needed blocks from S3.
Designing Tiered Compaction Policies
Effective policies balance three variables: age threshold, query frequency, and cost per GB. Start by measuring the average age of data that satisfies 90 % of your read‑throughput (e.g., the last 7 days). Set the hot‑to‑cold transition at that age plus a safety margin (e.g., 10 days). Adjust the compaction schedule to run during low‑ingest windows, such as 02:00–04:00 UTC, to avoid resource contention.
- Define the hot‑data retention period (e.g., 7 days) in
retention‑policy. - Set the
tiered‑compaction‑ageto the hot period plus margin (e.g., 17 days). - Configure the S3 bucket lifecycle: Standard‑IA after 30 days, Glacier Deep Archive after 365 days.
- Enable remote store in InfluxDB:
remote‑store.enabled = trueandremote‑store.endpoint = 'https://s3.amazonaws.com/influxdb-cold'. - Start the compaction service and monitor logs for
tiered‑compactionevents.
Tradeoffs and Performance Considerations
Moving data to S3 introduces latency for cold‑tier queries, typically ranging from 100 ms to 500 ms depending on object size and network path. However, the majority of analytical queries can tolerate this delay because they scan large time ranges. Write performance remains unaffected as new data lands exclusively in the hot tier.
Cost vs Latency
Storing one month of high‑resolution metrics on SSD can cost ≈ $0.12 per GB‑month, whereas the same data in S3 Standard‑IA is ≈ $0.0125 per GB‑month—a 90 % reduction. The added read latency is often offset by caching frequent query results in Redis or using InfluxDB’s built‑in query cache.
Impact on Query Performance
Queries that filter to the hot tier retain sub‑second response times. Cross‑tier queries (e.g., comparing last month vs. same month last year) experience a modest increase in latency proportional to the amount of data fetched from S3. Proper shard key design (e.g., by time) ensures that the engine can prune cold shards quickly, minimizing unnecessary S3 reads.
Implementation Example: Configuration Snippet
Below is a minimal config.yaml excerpt that ties the pieces together. Adjust values to match your environment.
[http]
bind-address = ':8086'
[data]
dir = '/var/lib/influxdb/data'
index-version = 'inmem'
memory-limit = '1GB'
[retention]
enabled = true
check-interval = '30m'
[remote-store]
enabled = true
endpoint = 'https://s3.amazonaws.com/influxdb-cold'
access-key = '${AWS_ACCESS_KEY_ID}'
secret-key = '${AWS_SECRET_ACCESS_KEY}'
bucket = 'influxdb-cold'
[tiered-compaction]
enabled = true
age-threshold = '17d'
schedule = '0 2 * * *'
Conclusion
InfluxDB hybrid hot cold storage is the foundation of this approach. By defining a clear hot‑to‑cold threshold, leveraging Amazon S3 lifecycle rules, and automating tiered compaction, you achieve a self‑healing archive that scales with your data volume. Start with a seven‑day hot window, monitor query latency, and adjust the age threshold until your cost‑performance target is met.