cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Burton <bur...@spinn3r.com>
Subject Re: 2x disk space required for full compaction? Don't vnodes help this problem?
Date Thu, 24 Jul 2014 22:27:14 GMT
This was after a bootstrap… so I triggered a major compaction.  Should I
just turn on leveled compaction and then never do a major compaction?


On Thu, Jul 24, 2014 at 3:09 PM, DuyHai Doan <doanduyhai@gmail.com> wrote:

> If you're using SizeTieredCompactionStrategy the disk space may double
> temporarily during compaction. That's one of the big drawback of
> SizedTiered. Since you're on SSD, why not test switching  to
> LeveledCompaction ? Put a node on write survey mode to see if this change
> has any impact on your I/O, CPU and node stability.
>
>
> On Thu, Jul 24, 2014 at 11:56 PM, Kevin Burton <burton@spinn3r.com> wrote:
>
>> I just bootstrapped a new node.
>>
>> The box had about 220GB of data on it on a 400GB SSD drive.
>>
>> I triggered a full compaction after it bootstrapped, and it ran out of
>> disk space about 15 minutes later.  so now that node is dead :-(
>>
>> I would have assumed that vnodes meant that I could keep my drive near
>> 100% full…
>>
>> so during a major compaction it would just compact the first vnode, then
>> move on to the second.
>>
>> this would be analogous to bigtable / hbase regions.
>>
>> but … that doesn't seem to be the case.  (so bad assumption on my part)
>> Both in terms of me actually seeing the disk fill up, and also the case of
>> my disk not having separate SSTables for each vnode.
>>
>> So now I have these SSDs that I have to keep at > 50% capacity at all
>> times.
>>
>> I can see why on HDDs having too many files would be an issue.
>>
>> But on SSDs this is less of a problem.
>>
>> Perhaps some hybrid where vnodes are chunked together in one contiguous
>> region?
>>
>> Is there a way to fix this problem? I would like to get more usage out of
>> my SSDs...
>>
>> --
>>
>> Founder/CEO Spinn3r.com
>> Location: *San Francisco, CA*
>> blog: http://burtonator.wordpress.com
>> … or check out my Google+ profile
>> <https://plus.google.com/102718274791889610666/posts>
>> <http://spinn3r.com>
>>
>>
>


-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>

Mime
View raw message