cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Burruss <>
Subject RE: MultiThread Client problem with thrift
Date Tue, 22 Dec 2009 19:28:28 GMT
i don't close the connection to a server unless i get exceptions.  and when i close the connection
i try a new server in the cluster just to keep the connections spread across the cluster.

should i be closing them?  if the connection is closed by client or server i'll just reconnect.

From: Ran Tavory []
Sent: Tuesday, December 22, 2009 9:50 AM
Subject: Re: MultiThread Client problem with thrift

I don't know how keeping the connections open affects at scale. I suppose if you have 10 to
1 ratio of cassandra clients to cassandra server (probably a typical ratio) then you may be
using too much server resources

On Tue, Dec 22, 2009 at 4:46 PM, matthew hawthorne <<>>
On Tue, Dec 22, 2009 at 9:10 AM, Ran Tavory <<>>
> Not at expert in this field, but I think what you want is use a connection
> pool and NOT close the connections - reuse them. Only idle connections are
> released after, say 1sec. Also, with a connection pool it's easy
> to throttle the application, you can tell the pool to block if all 50
> connections, or how many you define are allowed.

I did something very similar to this.  A difference in my approach is
that I did not release idle connections after a specific time period,
instead I performed a liveness check on each connection after
obtaining it from the pool, like this:

// get client connection from pool
Cassandra.Client client =

try {
} catch (TTransportException e) {
 // connection is invalid, obtain new connection

It seemed to work during my testing, not sure if the thrift specifics
are 100% correct (meaning I'm not sure if the catch block will work
for all situations involving stale or expired connections).


> On Tue, Dec 22, 2009 at 4:01 PM, Richard Grossman <<>>
> wrote:
>> So I can't use it.
>> But I've make my own connection pool. This are not fix nothing because the
>> problem is lower than even java. In fact the socket is closed and java
>> consider it as close but the system keep the Socket in the  state TIME_WAIT.
>> Then the port used is actually still in use.
>> So my question is that is there people that manage to open multiple
>> connection and ride off the TIME_WAIT. No matter in which language PHP or
>> Python etc...
>> Thanks
>> On Tue, Dec 22, 2009 at 2:55 PM, Ran Tavory <<>>
>>> I don't have a 0.5.0-beta2 version, no. It's not too difficult to add it,
>>> but I haven't done so myself, I'm using 0.4.2
>>> On Tue, Dec 22, 2009 at 2:42 PM, Richard Grossman <<>>
>>> wrote:
>>>> Yes of course but do you have updated to cassandra 0.5.0-beta2 ?
>>>> On Tue, Dec 22, 2009 at 2:30 PM, Ran Tavory <<>>
>>>>> Would connection pooling work for you?
>>>>> This Java client has
>>>>> connection pooling.
>>>>> I haven't put the client under stress yet so I can't testify, but this
>>>>> may be a good solution for you
>>>>> On Tue, Dec 22, 2009 at 2:22 PM, Richard Grossman <<>>
>>>>> wrote:
>>>>>> I agree it's solve my problem but can give a bigger one.
>>>>>> The problem is I can't succeed to prevent opening a lot of connection
>>>>>> On Tue, Dec 22, 2009 at 1:51 PM, Jaakko <<>>
>>>>>> wrote:
>>>>>>> Hi,
>>>>>>> I don't know the particulars of java implementation, but if it
>>>>>>> the same way as Unix native socket API, then I would not recommend
>>>>>>> setting linger to zero.
>>>>>>> SO_LINGER option with zero value will cause TCP connection to
>>>>>>> aborted immediately as soon as the socket is closed. That is,
>>>>>>> remaining data in the send buffer will be discarded, (2) no proper
>>>>>>> disconnect handshake and (3) receiving end will get TCP reset.
>>>>>>> Sure this will avoid TIME_WAIT state, but TIME_WAIT is our friend
>>>>>>> is there to avoid packets from old connection being delivered
to new
>>>>>>> incarnation of the connection. Instead of avoiding the state,
>>>>>>> application should be changed so that TIME_WAIT will not be a
>>>>>>> problem.
>>>>>>> How many open files you can see when the exception happens? Might
>>>>>>> that you're out of file descriptors.
>>>>>>> -Jaakko
>>>>>>> On Tue, Dec 22, 2009 at 8:17 PM, Richard Grossman
>>>>>>> <<>>
>>>>>>> > Hi
>>>>>>> > To all is interesting I've found a solution seems not recommended
>>>>>>> > but
>>>>>>> > working.
>>>>>>> > When opening a Socket set this:
>>>>>>> >    tSocket.getSocket().setReuseAddress(true);
>>>>>>> >    tSocket.getSocket().setSoLinger(true, 0);
>>>>>>> > it's prevent to have a lot of connection TIME_WAIT state
but not
>>>>>>> > recommended.
>>>>>>> >

View raw message