incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ивaн Cобoлeв <sobol...@gmail.com>
Subject Re: Estimating write throughput with LeveledCompactionStrategy
Date Tue, 12 Feb 2013 06:43:59 GMT
Yup, we set it to 100M. Currently we have around 1Tb of data per
node(getting to level 5 now) + data pieces are rather large(small
tables would flush more often).

Yes, you're right, it's slower thus building mental models is more
time effective than experimenting :)

Ivan

2013/2/6 Wei Zhu <wz1975@yahoo.com>:
> I have been struggling with the LCS myself. I observed that for the higher
> level compaction,(from level 4 to 5) it involves much more SSTables than
> compacting from lower level. One compaction could take an hour or more. By
> the way, you set the your SSTable size to be 100M?
>
> Thanks.
> -Wei
>
> ________________________________
> From: Ивaн Cобoлeв <soboleiv@gmail.com>
> To: user@cassandra.apache.org
> Sent: Wednesday, February 6, 2013 2:42 AM
> Subject: Estimating write throughput with LeveledCompactionStrategy
>
> Dear Community,
>
> Could anyone please give me a hand with understanding what am I
> missing while trying to model how LeveledCompactionStrategy works:
> https://docs.google.com/spreadsheet/ccc?key=0AvNacZ0w52BydDQ3N2ZPSks2OHR1dlFmMVV4d1E2eEE#gid=0
>
> Logs mostly contain something like this:
> INFO [CompactionExecutor:2235] 2013-02-06 02:32:29,758
> CompactionTask.java (line 221) Compacted to
> [chunks-hf-285962-Data.db,chunks-hf-285963-Data.db,chunks-hf-285964-Data.db,chunks-hf-285965-Data.db,chunks-hf-285966-Data.db,chunks-hf-285967-Data.db,chunks-hf-285968-Data.db,chunks-hf-285969-Data.db,chunks-hf-285970-Data.db,chunks-hf-285971-Data.db,chunks-hf-285972-Data.db,chunks-hf-285973-Data.db,chunks-hf-285974-Data.db,chunks-hf-285975-Data.db,chunks-hf-285976-Data.db,chunks-hf-285977-Data.db,chunks-hf-285978-Data.db,chunks-hf-285979-Data.db,chunks-hf-285980-Data.db,].
> 2,255,863,073 to 1,908,460,931 (~84% of original) bytes for 36,868
> keys at 14.965795MB/s.  Time: 121,614ms.
>
> Thus spreadsheet is parameterized with throughput being 15Mb and
> survivor ratio of 0.9.
>
> 1) Projected result actually differs from what I observe - what am I
> missing?
> 2) Are there any metrics on write throughput with LCS per node anyone
> could possibly share?
>
> Thank you very much in advance,
> Ivan
>
>

Mime
View raw message