hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Nitschinger <mich...@nitschinger.at>
Subject Re: BasicNIOConnPool Connection Lease
Date Mon, 16 Dec 2013 14:06:20 GMT

On 16 Dec 2013, at 14:34, Oleg Kalnichevski <olegk@apache.org> wrote:

> On Mon, 2013-12-16 at 11:56 +0100, Michael Nitschinger wrote:
>> Hi folks (looks like my first post wasn’t acknowledged by the mailing list service
since it overlapped with subscribing),
>> I’ve posted an enhancement request for the docs shown here https://issues.apache.org/jira/browse/HTTPCORE-369
but I still have troubles making it work in general. 
>> So, basically I’m having a List<HttpHost> where I round-robin and send it
through the AsyncRequester. This all works great, but during runtime this list can change.
Adding is easy because it will pick it up on the fly. I’m more thinking about removing HttpHosts
from this list and the underlying BasicNIOConnPool management.
> Hi Michael
> HttpAsyncRequester provides several convenience methods that take full
> care of connection leasing and releasing automatically. In case you want
> to exert a greater control over the process of connection allocation /
> deallocation you might want to interact with the pool directly. You get
> more control at the price of greater complexity.

Hi Oleg

thanks for your initial response. I’ve been looking at the HttpAsyncRequester for some time
now but I think - as you said - it doesn’t help in my case? I need to release all connections,
wether available or leased at a specific point in time. 

>> I think I have a few questions:
>> 	- What is the preferred way to deal with connections that need to be removed from
the pool?
> I cannot think of any special requirements other than making sure to
> always return a leased connection back to pool regardless of the
> execution flow and operation result. If you want to have the connection
> removed from the pool, simply close it prior to returning it back.

That makes sense. I guess something isn’t clear in my understanding: does releasing mean
the underlying socket is also closed? Or it is just available to lease from somebode else.
Because I really need to close the underlying sockets (the use case is: a node has been removed
out of the cluster, but in theory it could still accept a tcp/http connection attempt).

What could also work is to just tell by configuration that a connection, if idle for N minutes,
just gets removed? is that possible? Because then I don’t need to take care of and close
idle sockets.. and if there is really so less traffic on the box then they maybe also don’t
care about that initial socket connect again.

>> 	- Is there something I need to be aware of?
> I cannot think of anything special. You just need to be mindful of
> having to deal with connection release in case of abnormal execution
> flows as well as normal. Whereas it is trivial to ensure resources
> deallocation with a try-finally when executing in a context of a thread,
> exception handling can get very tricky very fast with an event driven
> model.
>> I thought about iterating over the Entries, but there doesn’t seem to be a way
to do so. I’m using 4.3. If you are interested in the code, it’s here: https://gist.github.com/daschl/7985206
.. it will be open source anyway (this is part of a rework of the sdk couchbase driver) but
I haven’t pushed it out there yet.
> BasicNIOConnPool provides protected methods to enumerate leased and
> available connections. Those methods are not made public by default to
> not tempt people into messing with the internals of the connect pool
> without a good reason. So, you need to extend the default pool
> implementation. Generally I would always expect users to extend
> AbstractNIOConnPool instead of using ultra-basic BasicNIOConnPool 

I found the enumEntries method when I did override it, but I didn’t find how to get access
to the iterator over the entries? Also I didnt find a test case that uses it - do you have
at least a small code snippet on how to do it? Much appreciated :)

>> If you have any other remarks on how the code is used please shoot. Cheers,
> I would probably caution against the (ab)use of
> BasicAsyncResponseConsumer due to its buffering message content in
> memory. You might want to consider using a custom
> HttpAsyncResponseConsumer that can digest response content on-the-fly
> chunk by chunk. This would also enable you to consume content of
> Content-Length delimited and identity encoded messages directly from the
> underlying NIO channel without _any_ intermediate buffering.

Thanks for that info. Currently the code behind isn’t suited for chunking so aggregating
it is okay for now. We definitely want to improve that in the future and then I’ll implement
my own.
> Hope this helps
> Oleg    

> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org

To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org

View raw message