cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Shook (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-8371) DateTieredCompactionStrategy is always compacting
Date Mon, 01 Dec 2014 19:29:14 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-8371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14230287#comment-14230287
] 

Jonathan Shook edited comment on CASSANDRA-8371 at 12/1/14 7:28 PM:
--------------------------------------------------------------------

If we are to hold to the idea of minimally useful granularity, then we haven't reached that
point with DTCS yet. The only backwards-compatible change that provides something workable
is in-fact inconsistent with the parameter conventions. That's the rub. I am advocating that
we fix it in some useful way while it is still "early", in order to avoid having to deal with
"0.00069" and the like from now on. My suggestion for suffixes was just based on my own wishes
for something that was consistent, obvious, and easy to use.

My assertion is essentially that sub 1 day max times will be more common than initially assumed.

Regarding repairs, I do not feel comfortable with the notion that the value has to always
be high enough for repair to see it before the max age. There is no guarantee of that. If
there is repair involvement, then it should be handled as orthogonally as possible. What bad
things are we concerned about happening if repair has to supplement one of these files after
the max age?



was (Author: jshook):
If we are to hold to the idea of minimally useful granularity, then we haven't reached that
point with DTCS yet. The only backwards-compatible change that provides something workable
is in-fact inconsistent with the parameter conventions. That's the rub. I am advocating that
we fix it in some useful way while it is still "early", in order to avoid having to deal with
"0.00069" and the like from now on. My suggestion for suffixes was just base on my own wishes
for something that was consistent, obvious, and easy to use.

My assertion is essentially that sub 1 day max times will be more common than initially assumed.

Regarding repairs, I do not feel comfortable with the notion that the value has to always
be high enough for repair to see it before the max age. There is no guarantee of that. If
there is repair involvement, then it should be handled as orthogonally as possible. What bad
things are we concerned about happening if repair has to supplement one of these files after
the max age?


> DateTieredCompactionStrategy is always compacting 
> --------------------------------------------------
>
>                 Key: CASSANDRA-8371
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8371
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: mck
>            Assignee: Björn Hegerfors
>              Labels: compaction, performance
>         Attachments: java_gc_counts_rate-month.png, read-latency-recommenders-adview.png,
read-latency.png, sstables-recommenders-adviews.png, sstables.png, vg2_iad-month.png
>
>
> Running 2.0.11 and having switched a table to [DTCS|https://issues.apache.org/jira/browse/CASSANDRA-6602]
we've seen that disk IO and gc count increase, along with the number of reads happening in
the "compaction" hump of cfhistograms.
> Data, and generally performance, looks good, but compactions are always happening, and
pending compactions are building up.
> The schema for this is 
> {code}CREATE TABLE search (
>   loginid text,
>   searchid timeuuid,
>   description text,
>   searchkey text,
>   searchurl text,
>   PRIMARY KEY ((loginid), searchid)
> );{code}
> We're sitting on about 82G (per replica) across 6 nodes in 4 DCs.
> CQL executed against this keyspace, and traffic patterns, can be seen in slides 7+8 of
https://prezi.com/b9-aj6p2esft/
> Attached are sstables-per-read and read-latency graphs from cfhistograms, and screenshots
of our munin graphs as we have gone from STCS, to LCS (week ~44), to DTCS (week ~46).
> These screenshots are also found in the prezi on slides 9-11.
> [~pmcfadin], [~Bj0rn], 
> Can this be a consequence of occasional deleted rows, as is described under (3) in the
description of CASSANDRA-6602 ?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message