cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anshul Rathore <anshulrathor...@gmail.com>
Subject Re: Auto Compactions not running on Cassandra 3.10
Date Thu, 02 Aug 2018 06:19:17 GMT
Hi

It would be great if anyone can point us in right direction.

On Mon, Jul 30, 2018 at 12:07 PM Anshul Rathore <anshulrathore88@gmail.com>
wrote:

> Thanks Jeff for your response ,and apologies for such a delayed response ,
> had some personal emergency.
>
> So Following the config which we are using for that table
>  PRIMARY KEY ((customer_app_prefix, customer_session_id),
> beacon_client_type, sim_created_at)
> ) WITH CLUSTERING ORDER BY (beacon_client_type ASC, sim_created_at DESC)
>     AND bloom_filter_fp_chance = 0.01
>     AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
>     AND comment = ''
>     AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
>     AND compression = {'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
>     AND crc_check_chance = 1.0
>     AND dclocal_read_repair_chance = 0.1
>     AND default_time_to_live = 0
>     AND gc_grace_seconds = 864000
>     AND max_index_interval = 2048
>     AND memtable_flush_period_in_ms = 0
>     AND min_index_interval = 128
>     AND read_repair_chance = 0.0
>     AND speculative_retry = '99PERCENTILE';
>
>
> I didn't get the part *"are you using milliseconds or microseconds in
> your writes?" *,
> did you mean for the timestamp column in Primary Key , for that we using
> milliseconds .
>
> Thanks,
> Anshul
>
>
> On Wed, Jul 4, 2018 at 11:55 PM Jeff Jirsa <jjirsa@gmail.com> wrote:
>
>> The DTCS windowing algorithm is timestamp sensitive - are you using
>> milliseconds or microseconds in your writes? What’s your exact DTCS config
>> for that table?
>>
>> 3.10 has TWCS, too, which may be considerably easier to use. It also uses
>> timestamp units in its config, so you’ll need to know which resolution
>> you’re using on writes
>>
>>
>> --
>> Jeff Jirsa
>>
>>
>> On Jul 4, 2018, at 11:08 AM, Anshul Rathore <anshulrathore88@gmail.com>
>> wrote:
>>
>>
>> Hi
>>
>> We are using 4 node cassandra 3.10 cluster.
>> For some reason autocompaction is not running on one of the table which
>> uses DTCS. TTL on this table is 3 months. Table has high write load , and
>> medium read load.
>>
>> We have 4 disks per node , each disk growed to around 5k-6k sstables
>> going back to around 6-7 months.
>> I tried running major compaction which takes 1-2 days on each node, which
>> results in 20-30 sstables on all disks.
>> But also after running major compaction , we now see sstables have
>> started growing again. So minor/auto compaction is not running on this
>> table.
>>
>> *Few Observations*
>> *      - *There are 2-3 partition keys i see while doing  major
>> compaction throwing large partition warning(100mb threshold) , their sizes
>> around 100-200MB
>>       - All other tables use STCS , their auto compaction  is working
>> fine.
>>
>> Any pointers would be helpful
>>
>> Thanks,
>> Anshul
>>
>>
>
> --
> *Regards,*
> *Anshul Rathore*
>


-- 
*Regards,*
*Anshul Rathore*

Mime
View raw message