incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vicent Llongo <villo...@gmail.com>
Subject Re: Timeseries with TTL
Date Sun, 06 Apr 2014 14:25:47 GMT
Hi,

Most of the queries to that table are just getting a range of values for a
metric:
SELECT val FROM metrics_5min WHERE uid = ? AND metric = ? AND ts >= ? AND
ts <= ?

I'm not sure from the logs what kind of compactions they are. This is what
I see in system.log (grepping for that specific table):

...
INFO [CompactionExecutor:742] 2014-04-06 13:30:11,223 CompactionTask.java
(line 105) Compacting
[SSTableReader(path='/mnt/disk1/cassandra/data/keyspace/metrics_5min/keyspace-metrics_5min-ic-14991-Data.db'),
SSTableReader(path='/mnt/disk1/cassandra/data/keyspace/metrics_5min/keyspace-metrics_5min-ic-14990-Data.db')]
INFO [CompactionExecutor:753] 2014-04-06 13:35:22,495 CompactionTask.java
(line 105) Compacting
[SSTableReader(path='/mnt/disk1/cassandra/data/keyspace/metrics_5min/keyspace-metrics_5min-ic-14992-Data.db'),
SSTableReader(path='/mnt/disk1/cassandra/data/keyspace/metrics_5min/keyspace-metrics_5min-ic-14993-Data.db')]
INFO [CompactionExecutor:770] 2014-04-06 13:41:09,146 CompactionTask.java
(line 105) Compacting
[SSTableReader(path='/mnt/disk1/cassandra/data/keyspace/metrics_5min/keyspace-metrics_5min-ic-14995-Data.db'),
SSTableReader(path='/mnt/disk1/cassandra/data/keyspace/metrics_5min/keyspace-metrics_5min-ic-14994-Data.db')]
INFO [CompactionExecutor:783] 2014-04-06 13:46:21,250 CompactionTask.java
(line 105) Compacting
[SSTableReader(path='/mnt/disk1/cassandra/data/keyspace/metrics_5min/keyspace-metrics_5min-ic-14996-Data.db'),
SSTableReader(path='/mnt/disk1/cassandra/data/keyspace/metrics_5min/keyspace-metrics_5min-ic-14997-Data.db')]
INFO [CompactionExecutor:798] 2014-04-06 13:51:28,369 CompactionTask.java
(line 105) Compacting
[SSTableReader(path='/mnt/disk1/cassandra/data/keyspace/metrics_5min/keyspace-metrics_5min-ic-14998-Data.db'),
SSTableReader(path='/mnt/disk1/cassandra/data/keyspace/metrics_5min/keyspace-metrics_5min-ic-14999-Data.db')]
INFO [CompactionExecutor:816] 2014-04-06 13:57:17,585 CompactionTask.java
(line 105) Compacting
[SSTableReader(path='/mnt/disk1/cassandra/data/keyspace/metrics_5min/keyspace-metrics_5min-ic-15000-Data.db'),
SSTableReader(path='/mnt/disk1/cassandra/data/keyspace/metrics_5min/keyspace-metrics_5min-ic-15001-Data.db')]
...

As you can see every ~5 minutes there's a compaction going on.




On Sun, Apr 6, 2014 at 4:33 PM, Sergey Murylev <sergeymurylev@gmail.com>wrote:

>  Hi Vincent,
>
>
> Is that a good pattern for Cassandra? Is there some compaction tunings I
> should take into account?
>
> Actually it depends on how you use Cassandra :). If you use it as
> key-value storage TTL works fine. But if you would use rather complex CQL
> queries to this table I not sure that it would be good.
>
>
> With this structure is obvious that after one week inserting data, from
> that moment there's gonna be new expired columns every 5 minutes in that
> table. Because of that I've noticed that this table is being compacted
> every 5 minutes.
>
> Compaction doesn't triggered when some column expired. It triggered on
> gc_grace_seconds timeout and according compaction strategy. You can see
> more detailed description of LeveledCompactionStrategy in following
> article: Leveled compaction in Cassandra<http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra>.
>
>
> There are 2 types of compaction: minor and major, which kind of compaction
> do you see and how come to conclusion that compaction triggered every 5
> minutes? If you see major compaction that situation is very bad otherwise
> it is normal case.
>
> --
> Thanks,
> Sergey
>
>
>
> On 06/04/14 15:48, Vicent Llongo wrote:
>
>  Hi there,
>
>  I have this table where I'm inserting timeseries values with a TTL of
> 86400*7 (1week):
>
> CREATE TABLE metrics_5min (
>   object_id varchar,
>   metric varchar,
>   ts timestamp,
>   val double,
>   PRIMARY KEY ((object_id, metric), ts)
> )
> WITH gc_grace_seconds = 86400
> AND compaction = {'class': 'LeveledCompactionStrategy',
> 'sstable_size_in_mb' : 100};
>
>
>  With this structure is obvious that after one week inserting data, from
> that moment there's gonna be new expired columns every 5 minutes in that
> table. Because of that I've noticed that this table is being compacted
> every 5 minutes.
>
> Is that a good pattern for Cassandra? Is there some compaction tunings I
> should take into account?
>
>  Thanks!
>
>
>
>

Mime
View raw message