其他分享
首页 > 其他分享> > citus 以及timescaledb对于时许数据存储的处理

citus 以及timescaledb对于时许数据存储的处理

作者:互联网

从专业程度来说timescaledb 处理时许数据更加方便强大,citus 可以基于pg的分区可以很好的支持时许数据处理

citus 操作流程

 
CREATE TABLE github_events (
  event_id bigint,
  event_type text,
  event_public boolean,
  repo_id bigint,
  payload jsonb,
  repo jsonb,
  actor jsonb,
  org jsonb,
  created_at timestamp
) PARTITION BY RANGE (created_at);
 
SELECT create_time_partitions(
  table_name         := 'github_events',
  partition_interval := '1 month',
  end_at             := now() + '12 months'
);
 
SELECT create_distributed_table('github_events', 'repo_id');
 
SELECT partition
  FROM time_partitions
 WHERE parent_table = 'github_events'::regclass;
 
SELECT cron.schedule('create-partitions', '0 0 1 * *', $$
  SELECT create_time_partitions(
      table_name         := 'github_events',
      partition_interval := '1 month',
      end_at             := now() + '12 months'
  )
$$);
 
-- 2. (optional) ensure we never have more than one year of data
 
SELECT cron.schedule('drop-partitions', '0 0 1 * *', $$
  CALL drop_old_time_partitions(
      'github_events',
      now() - interval '12 months' /* older_than */
  );
$$);
SELECT cron.schedule('compress-partitions', '0 0 1 * *', $$
  CALL alter_old_partitions_set_access_method(
    'github_columnar_events',
    now() - interval '6 months' /* older_than */,
    'columnar'
  );
$$);

timescaledb 的处理

timescaledb 处理此类的就比较专业了,因为就是为了支持时许处理的,套路上很多与citus 类似,timescaledb 自己开发了一个定时任务处理的

说明

citus 与timescaledb对于时许数据的处理即有相似的地方,也有差异的地方,使用上大家都比较类似,但是timescaledb更加擅长处理时许处理

参考资料

https://docs.citusdata.com/en/stable/use_cases/timeseries.html#timeseries-data
https://docs.timescale.com/
https://www.citusdata.com/blog/2021/09/17/citus-10-2-extension-to-postgres-whats-new/#partition-mgmt
https://docs.timescale.com/timescaledb/latest/how-to-guides/data-retention/
https://docs.timescale.com/timescaledb/latest/getting-started/data-retention/#learn-more-about-data-retention

标签:citus,timescaledb,时许,events,SELECT,partitions
来源: https://www.cnblogs.com/rongfengliang/p/16254859.html