incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From matthew hawthorne <mhawtho...@gmail.com>
Subject Re: MultiThread Client problem with thrift
Date Tue, 22 Dec 2009 14:46:22 GMT
On Tue, Dec 22, 2009 at 9:10 AM, Ran Tavory <rantav@gmail.com> wrote:
> Not at expert in this field, but I think what you want is use a connection
> pool and NOT close the connections - reuse them. Only idle connections are
> released after, say 1sec. Also, with a connection pool it's easy
> to throttle the application, you can tell the pool to block if all 50
> connections, or how many you define are allowed.

I did something very similar to this.  A difference in my approach is
that I did not release idle connections after a specific time period,
instead I performed a liveness check on each connection after
obtaining it from the pool, like this:

// get client connection from pool
Cassandra.Client client =

try {
  client.getInputProtocol().getTransport().flush();
} catch (TTransportException e) {
  // connection is invalid, obtain new connection
}

It seemed to work during my testing, not sure if the thrift specifics
are 100% correct (meaning I'm not sure if the catch block will work
for all situations involving stale or expired connections).

-matt


> On Tue, Dec 22, 2009 at 4:01 PM, Richard Grossman <richiesgr@gmail.com>
> wrote:
>>
>> So I can't use it.
>> But I've make my own connection pool. This are not fix nothing because the
>> problem is lower than even java. In fact the socket is closed and java
>> consider it as close but the system keep the Socket in the  state TIME_WAIT.
>> Then the port used is actually still in use.
>> So my question is that is there people that manage to open multiple
>> connection and ride off the TIME_WAIT. No matter in which language PHP or
>> Python etc...
>> Thanks
>> On Tue, Dec 22, 2009 at 2:55 PM, Ran Tavory <rantav@gmail.com> wrote:
>>>
>>> I don't have a 0.5.0-beta2 version, no. It's not too difficult to add it,
>>> but I haven't done so myself, I'm using 0.4.2
>>>
>>> On Tue, Dec 22, 2009 at 2:42 PM, Richard Grossman <richiesgr@gmail.com>
>>> wrote:
>>>>
>>>> Yes of course but do you have updated to cassandra 0.5.0-beta2 ?
>>>>
>>>> On Tue, Dec 22, 2009 at 2:30 PM, Ran Tavory <rantav@gmail.com> wrote:
>>>>>
>>>>> Would connection pooling work for you?
>>>>> This Java client http://code.google.com/p/cassandra-java-client/ has
>>>>> connection pooling.
>>>>> I haven't put the client under stress yet so I can't testify, but this
>>>>> may be a good solution for you
>>>>>
>>>>> On Tue, Dec 22, 2009 at 2:22 PM, Richard Grossman <richiesgr@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> I agree it's solve my problem but can give a bigger one.
>>>>>> The problem is I can't succeed to prevent opening a lot of connection
>>>>>>
>>>>>> On Tue, Dec 22, 2009 at 1:51 PM, Jaakko <rosvopaallikko@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I don't know the particulars of java implementation, but if it
works
>>>>>>> the same way as Unix native socket API, then I would not recommend
>>>>>>> setting linger to zero.
>>>>>>>
>>>>>>> SO_LINGER option with zero value will cause TCP connection to
be
>>>>>>> aborted immediately as soon as the socket is closed. That is,
(1)
>>>>>>> remaining data in the send buffer will be discarded, (2) no proper
>>>>>>> disconnect handshake and (3) receiving end will get TCP reset.
>>>>>>>
>>>>>>> Sure this will avoid TIME_WAIT state, but TIME_WAIT is our friend
and
>>>>>>> is there to avoid packets from old connection being delivered
to new
>>>>>>> incarnation of the connection. Instead of avoiding the state,
the
>>>>>>> application should be changed so that TIME_WAIT will not be a
>>>>>>> problem.
>>>>>>> How many open files you can see when the exception happens? Might
be
>>>>>>> that you're out of file descriptors.
>>>>>>>
>>>>>>> -Jaakko
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Dec 22, 2009 at 8:17 PM, Richard Grossman
>>>>>>> <richiesgr@gmail.com> wrote:
>>>>>>> > Hi
>>>>>>> > To all is interesting I've found a solution seems not recommended
>>>>>>> > but
>>>>>>> > working.
>>>>>>> > When opening a Socket set this:
>>>>>>> >    tSocket.getSocket().setReuseAddress(true);
>>>>>>> >    tSocket.getSocket().setSoLinger(true, 0);
>>>>>>> > it's prevent to have a lot of connection TIME_WAIT state
but not
>>>>>>> > recommended.
>>>>>>> >
>>>>>>
>>>>>
>>>>
>>>
>>
>
>

Mime
View raw message