incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Burruss <bburr...@real.com>
Subject Re: read latency creaping up
Date Tue, 15 Dec 2009 02:03:49 GMT
I plan to investigate that next.  There isn't a straightforward "disconnect" that I could easily
see.

Ian Holsman <ian@holsman.net> wrote:


can you make it so that the client restarts the connection every 30m or so ?
It could be an issue in thrift or something with long-lived connections.

On Dec 15, 2009, at 10:16 AM, Brian Burruss wrote:

> i agree.  i don't know anything about thrift, and i don't know how it keeps connections
open or manages resources from a client or server perspective, but this situation suggests
that maybe killing the clients is forcing the server to free something.
>
> how's that sound :)
>
> ________________________________________
> From: Jonathan Ellis [jbellis@gmail.com]
> Sent: Monday, December 14, 2009 3:12 PM
> To: cassandra-user@incubator.apache.org
> Subject: Re: read latency creaping up
>
> hmm, me neither
>
> but, I can't think how restarting the client would, either :)
>
> On Mon, Dec 14, 2009 at 4:59 PM, Brian Burruss <bburruss@real.com> wrote:
>> Well not sure how that would affect he latency as reported by the Cassandra server
using nodeprobe cfstats
>>
>> Jonathan Ellis <jbellis@gmail.com> wrote:
>>
>>
>> possibly the clients are running into memory pressure?
>>
>> On Mon, Dec 14, 2009 at 4:27 PM, Brian Burruss <bburruss@real.com> wrote:
>>> thx, i'm actually the "B. Todd Burruss" in that thread ..  we changed our email
system and well now, i'm just Brian .. long story.
>>>
>>> anyway, in this case it isn't compaction pendings as i can kill the clients and
immediately restart and the latency is back to a reasonable number.  i'm still investigating.
>>>
>>> thx!
>>> ________________________________________
>>> From: Eric Evans [eevans@rackspace.com]
>>> Sent: Monday, December 14, 2009 8:23 AM
>>> To: cassandra-user@incubator.apache.org
>>> Subject: RE: read latency creaping up
>>>
>>> On Sun, 2009-12-13 at 13:18 -0800, Brian Burruss wrote:
>>>> if this isn't a known issue, lemme do some more investigating.  my
>>>> test client becomes "more random" with reads as time progresses, so
>>>> possibly this is what causes the latency issue.  however, all that
>>>> being said, the performance really becomes bad after a while.
>>>
>>> Have a look at the following thread:
>>>
>>> http://thread.gmane.org/gmane.comp.db.cassandra.user/1402
>>>
>>>
>>> --
>>> Eric Evans
>>> eevans@rackspace.com
>>>
>>>
>>

--
Ian Holsman
Ian@Holsman.net




Mime
View raw message