incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sankalp kohli <kohlisank...@gmail.com>
Subject Re: Cassandra as storage for cache data
Date Tue, 25 Jun 2013 20:52:19 GMT
Apart from what Jeremy said, you can try these
1) Use replication = 1. It is cache data and you dont need persistence.
2) Try playing with memtable size.
3) Use netflix client library as it will reduce one hop. It will chose the
node with data as the co ordinator.
4) Work on your schema. You might want to have fewer columns in each row.
With fatter rows, bloom filter will give out more sstables which are
eligible.

-Sankalp


On Tue, Jun 25, 2013 at 9:04 AM, Jeremy Hanna <jeremy.hanna1234@gmail.com>wrote:

> If you have rapidly expiring data, then tombstones are probably filling
> your disk and your heap (depending on how you order the data on disk).  To
> check to see if your queries are affected by tombstones, you might try
> using the query tracing that's built-in to 1.2.
> See:
>
> http://www.datastax.com/dev/blog/cassandra-anti-patterns-queues-and-queue-like-datasets
-- has an example of tracing where you can see tombstones affecting the
> query
> http://www.datastax.com/dev/blog/tracing-in-cassandra-1-2
>
> You'll want to consider reducing the gc_grace period from the default of
> 10 days for those column families - with the understanding why gc_grace
> exists in the first place, see
> http://wiki.apache.org/cassandra/DistributedDeletes .  Then once the
> gc_grace period has passed, the tombstones will stay around until they are
> compacted away.  So there are two options currently to compact them away
> more quickly:
> 1) use leveled compaction - see
> http://www.datastax.com/dev/blog/when-to-use-leveled-compaction  Leveled
> compaction only requires 10% headroom (as opposed to 50% for size tiered
> compaction) for amount of disk that needs to be kept free.
> 2) if 1 doesn't work and you're still seeing performance degrading and the
> tombstones aren't getting cleared out fast enough, you might consider using
> size tiered compaction but performing regular major compactions to get rid
> of expired data.
>
> Keep in mind though that if you use gc_grace of 0 and do any kind of
> manual deletes outside of TTLs, you probably want to do the deletes at
> ConsistencyLevel.ALL or else if a node goes down, then comes back up,
> there's a chance that deleted data may be resurrected.  That only applies
> to non-ttl data where you manually delete it.  See the explanation of
> distributed deletes for more information.
>
>
> On 25 Jun 2013, at 13:31, Dmitry Olshansky <dmitry.olshansky@gridnine.com>
> wrote:
>
> > Hello,
> >
> > we are using Cassandra as a data storage for our caching system. Our
> application generates about 20 put and get requests per second. An average
> size of one cache item is about 500 Kb.
> >
> > Cache items are placed into one column family with TTL set to 20 - 60
> minutes. Keys and values are bytes (not utf8 strings). Compaction strategy
> is SizeTieredCompactionStrategy.
> >
> > We setup Cassandra 1.2.6 cluster of 4 nodes. Replication factor is 2.
> Each node has 10GB of RAM and enough space on HDD.
> >
> > Now when we're putting this cluster into the load it's quickly fills
> with our runtime data (about 5 GB on every node) and we start observing
> performance degradation with often timeouts on client side.
> >
> > We see that on each node compaction starts very frequently and lasts for
> several minutes to complete. It seems that each node usually busy with
> compaction process.
> >
> > Here the questions:
> >
> > What are the recommended setup configuration for our use case?
> >
> > Is it makes sense to somehow tell Cassandra to keep all data in memory
> (memtables) to eliminate flushing it to disk (sstables) thus decreasing
> number of compactions? How to achieve this behavior?
> >
> > Cassandra is starting with default shell script that gives the following
> command line:
> >
> > jsvc.exec -user cassandra -home
> /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/../ -pidfile
> /var/run/cassandra.pid -errfile &1 -outfile /var/log/cassandra/output.log
> -cp <CLASSPATH_SKIPPED> -Dlog4j.configuration=log4j-server.properties
> -Dlog4j.defaultInitOverride=true
> -XX:HeapDumpPath=/var/lib/cassandra/java_1371805844.hprof
> -XX:ErrorFile=/var/lib/cassandra/hs_err_1371805844.log -ea
> -javaagent:/usr/share/cassandra/lib/jamm-0.2.5.jar -XX:+UseThreadPriorities
> -XX:ThreadPriorityPolicy=42 -Xms2500M -Xmx2500M -Xmn400M
> -XX:+HeapDumpOnOutOfMemoryError -Xss180k -XX:+UseParNewGC
> -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8
> -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75
> -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseTLAB
> -Djava.net.preferIPv4Stack=true -Dcom.sun.management.jmxremote.port=7199
> -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.authenticate=false
> org.apache.cassandra.service.CassandraDaemon
> >
> > --
> > Best regards,
> > Dmitry Olshansky
> >
>
>

Mime
View raw message