cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From DuyHai Doan <doanduy...@gmail.com>
Subject Re: 2x disk space required for full compaction? Don't vnodes help this problem?
Date Thu, 24 Jul 2014 22:09:56 GMT
If you're using SizeTieredCompactionStrategy the disk space may double
temporarily during compaction. That's one of the big drawback of
SizedTiered. Since you're on SSD, why not test switching  to
LeveledCompaction ? Put a node on write survey mode to see if this change
has any impact on your I/O, CPU and node stability.


On Thu, Jul 24, 2014 at 11:56 PM, Kevin Burton <burton@spinn3r.com> wrote:

> I just bootstrapped a new node.
>
> The box had about 220GB of data on it on a 400GB SSD drive.
>
> I triggered a full compaction after it bootstrapped, and it ran out of
> disk space about 15 minutes later.  so now that node is dead :-(
>
> I would have assumed that vnodes meant that I could keep my drive near
> 100% full…
>
> so during a major compaction it would just compact the first vnode, then
> move on to the second.
>
> this would be analogous to bigtable / hbase regions.
>
> but … that doesn't seem to be the case.  (so bad assumption on my part)
> Both in terms of me actually seeing the disk fill up, and also the case of
> my disk not having separate SSTables for each vnode.
>
> So now I have these SSDs that I have to keep at > 50% capacity at all
> times.
>
> I can see why on HDDs having too many files would be an issue.
>
> But on SSDs this is less of a problem.
>
> Perhaps some hybrid where vnodes are chunked together in one contiguous
> region?
>
> Is there a way to fix this problem? I would like to get more usage out of
> my SSDs...
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
>

Mime
View raw message