cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Schubert Zhang (JIRA)" <>
Subject [jira] Commented: (CASSANDRA-1041) Skip large size (Configurable) SSTable in minor or/and major compaction
Date Tue, 04 May 2010 02:50:57 GMT


Schubert Zhang commented on CASSANDRA-1041:

I know and  understand all of what you pointed out all above. 
Could you please also give ous more advices for my "reasons" in my application. 
(1) Want to avoid cost of disk IO and CPU and memory usage when minor compaction.
(2) Do not want too big SSTable files, such as a file sized 1TB,  but several 100GB files
are ok.
(3) Temporary work-round of the "whole row in memory" limitation. (I know the final fix of
this limitation may be not ready immediately.) But it is really important.

> Skip large size (Configurable) SSTable in minor or/and major compaction
> -----------------------------------------------------------------------
>                 Key: CASSANDRA-1041
>                 URL:
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Core
>            Reporter: Schubert Zhang
>            Priority: Minor
>         Attachments: CASSANDRA-1041-0.6.1.patch, CASSANDRA-1041-0.6.patch
> When the SSTable files are large enough, such as 100GB, the compaction (include minor
and major) cost is big (disk IO, CPU, memory), etc.
> In some applications, we accept not compcating all SSTables to the final very large ones.

> This feature provide two optional configurable attributes MinorCompactSkipInGB and MajorCompactSkipInGB
for each ColumnFamily. 
> The optional MinorCompactSkipInGB attribute specifies the maximum size of SSTables which
will be compcated in minor-compaction. The SSTables larger than MinorCompactSkipInGB will
be skipped. The optional MajorCompactSkipInGB attribute is same for major-compaction.
> The default of these attributes are 0, means do not skip, just as current 0.6.1.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message