cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tamar Fraenkel <ta...@tok-media.com>
Subject Re: READ messages dropped
Date Wed, 10 Oct 2012 20:10:50 GMT
Hi!
Thanks for the answer.
I don't see much change in the load this Cassandra cluster is under, so why
is the sudden surge of such messages?
What I did noticed while looking at the logs (which are also running
OpsCenter), is that there is some correlation between the dropped reads and
flushes of OpsCenter column families to disk and or compactions. What are
the rollups CFs? why is there so much traffic in them?
Thanks,
*Tamar Fraenkel *
Senior Software Engineer, TOK Media

[image: Inline image 1]

tamar@tok-media.com
Tel:   +972 2 6409736
Mob:  +972 54 8356490
Fax:   +972 2 5612956





On Wed, Oct 10, 2012 at 1:00 AM, aaron morton <aaron@thelastpickle.com>wrote:

> or how to solve it?
>
> Simple solution is move to m1.xlarge :)
>
> In the last 3 days I see many messages of "READ messages dropped in last
> 5000ms" on one of my 3 nodes cluster.
>
> The node is not able to keep up with the load.
>
> Possible causes include excessive GC, aggressive compaction, or simply too
> many requests.
>
> it also a good idea to take a look at iostat to see if the disk is keeping
> up.
>
> Hope that helps
>
>   -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 9/10/2012, at 9:08 AM, Tamar Fraenkel <tamar@tok-media.com> wrote:
>
> Hi!
> In the last 3 days I see many messages of "READ messages dropped in last
> 5000ms" on one of my 3 nodes cluster.
> I see no errors in the log.
> There are also messages of "Finished hinted handoff of 0 rows to endpoint"
> but I had those for a while now, so I don't know if they are related.
> I am running Cassandra 1.0.8 on a 3 node cluster on EC2 m1.large
> instances. Rep factor 3 (Quorum read and write)
>
> Does anyone have a clue what I should be looking for, or how to solve it?
> Thanks,
>
> *Tamar Fraenkel *
> Senior Software Engineer, TOK Media
>
> <tokLogo.png>
>
>
> tamar@tok-media.com
> Tel:   +972 2 6409736
> Mob:  +972 54 8356490
> Fax:   +972 2 5612956
>
>
>
>
>

Mime
View raw message