Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: local policy)
From: "Hiller, Dean" <Dean.Hiller@nrel.gov>
To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Date: Wed, 18 Sep 2013 13:15:55 -0600
Subject: Re: What is the ideal value for sstable_size_in_mb when using
 LeveledCompactionStrategy ?
Thread-Topic: What is the ideal value for sstable_size_in_mb when using
 LeveledCompactionStrategy ?
Thread-Index: Ac60o3/PhN7L8AlWS/yk61CSbR1DPw==
Message-ID: <CE5F57B8.32545%Dean.Hiller@nrel.gov>
In-Reply-To: 
 <CAH9Xgonha28ONvhEUw3-6u1d=tMmgQtki9cpomOoHOjrtHNXKw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
user-agent: Microsoft-MacOutlook/14.3.7.130812
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

 1.  Always in cassandra up your file descriptor limits on linux and even i=
n 0.7 that was the recommendation so cassandra could open tons of files
 2.  We use 50M for our LCS with no performance issues.  We had it 10M on o=
ur previous with no issues but a huge amount of files of course with our 30=
0T per node.

Dean

From: Jayadev Jayaraman <jdisalive@gmail.com<mailto:jdisalive@gmail.com>>
Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <us=
er@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Date: Wednesday, September 18, 2013 1:02 PM
To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cas=
sandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: What is the ideal value for sstable_size_in_mb when using LeveledC=
ompactionStrategy ?

We have set up a 24 node (m1.xlarge nodes, 1.7 TB per node) cassandra clust=
er on Amazon EC2 :

version=3D1.2.9
replication factor =3D 2
snitch=3DEC2Snitch
placement_strategy=3DNetworkTopologyStrategy (with 12 nodes each in 2 avail=
ability zones)

Background on our use-case :

We plan on using hadoop with sstableloader to load 10GB+ of analytics data =
per day ( 100 million+ row keys, 5 or so columns per day on average.) . We =
have chosen LeveledCompactionStrategy in the hope that it constrains the nu=
mber of SSTables that are read in order to retrieve a sliced-predicate for =
a row. We don't want too many file-sockets ( > 1000) open to SSTables by th=
e Cassandra JVM as this has caused us network / unreachability issues befor=
e. We faced this when we were on cassandra 0.8.9 and we were using SizeTier=
edCompactionStrategy and in order to mitigate this, we ran minor compaction=
 daily and major compaction semi-regularly to ensure as few SSTable files a=
s possible on disk.


If we use LeveledCompactionStrategy with a small value for sstable_size_in_=
mb ( default =3D 5 MB ) , wouldn't that result in a very large number of SS=
Table files on disk ? How does that affect the number of file-sockets open =
(reading the docs, I get the impression that the number of SSTable seeks pe=
r query is reduced by a large margin) ? But if we use a larger value for ss=
table_size_in_mb, say around 200 MB, there will be 800 MB of small uncompac=
ted SSTables on disk per column-family to which there will inevitably be fi=
le-sockets open.

All in all, can someone help us figure out what we should set the value of =
sstable_size_in_mb to ? I figure it's not a very good idea to set it to a l=
arger value but I don't know how things perform if we set it to a small val=
ue. Do we have to run major compaction regularly in this case too ?

Thanks
Jayadev