incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Gallamore <mike.e.gallam...@googlemail.com>
Subject Re: writes to Cassandra failing occasionally
Date Fri, 09 Apr 2010 14:54:47 GMT
That makes sense. My data is coming in from the internet and is being
processed in chunks as it is using Active MQ with the stomp package. I'm
getting the log lines in 20-1000 line chunks (depending on the busyness of
customer sites) so there definitely is the potential for a lot of
parallelism. Some of my data will likely be in cache during write because of
the nature of the work. It's a reputation system so I first get a query from
the customer for the reputation and then afterwards within a minute or so
I'll get feedback back from them what the current events "score" was that
feedbacks into the system for updates of the value. Anyways lots of
parallelism opportunities.

2010/4/9 Ted Zlatanov <tzz@lifelogs.com>

> On Thu, 08 Apr 2010 11:50:38 -0700 Mike Gallamore <
> mike.e.gallamore@googlemail.com> wrote:
>
> MG> Yes I agree single threaded is probably not the best. I wonder how
> MG> much of a performance hit it is on a single CPU machine though? I
> MG> guess I still would be blocking on ram writes but isn't like there is
> MG> multiple CPUs I need to keep busy or anything.
>
> Cassandra may have to load data from disk for a particular query but
> another may already be in memory.  A third may cause a hit on another
> cluster node.  So if you issue queries serially you'll see performance
> drop off with the total number of queries because they are dependent on
> each other's performance, while the distribution of the performance of
> independent parallel queries will have skew and kurtosis much closer to
> a normal distribution.  In other words, your slowest (or unluckiest)
> queries are less damaging when you issue them in parallel.
>
> On the client side you still have slow serialization/deserialization and
> not much can be done about that.
>
> Ted
>
>

Mime
View raw message