hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: Asnyc and NIO Components
Date Tue, 09 May 2006 14:13:18 GMT
On Tue, 2006-05-09 at 09:59 -0400, Sam Berlin wrote: 
> Is the HttpNIO project going to be for the select/callback based
> architecture (to reduce the number of overall threads) or for processing the
> streams/channels with NIO CharEncoder / CharDecoder?  As I understood it, it
> was for the latter, which wouldn't quite solve what Brian is looking for.

Hi Sam,

In fact it is the former. The purpose of HttpNIO (the way I see it) is
to provide a non-blocking version of core HTTP components based exactly
on the select/callback architecture.

> 
> I have long had the desire to create this select-based HttpClient project,
> and in my head have been calling HttpNB (nb for non-blocking).  Unless Oleg
> corrects me to tell me that HttpNIO is planning to do this, I had slotted
> myself to get to this in approximately two.  I have just finished converting
> LimeWire's downloading code to be non-blocking, and next is uploading code.
> Then I'll be separating our I/O package into a separate jar and attempt to
> tweak Roland's HttpAsync module for use with a non-blocking HttpClient that
> has 1 thread processing many clients simultaneously.

This is not to discourage from pursuing the HttpNB project
independently, I just want to let know you are very welcome to
contribute to HttpCore/HttpNIO as well, should you have any spare cycles
left.

Cheers,

Oleg

> 
> HttpAsync appears very well suited to the task, except I spotted a few
> things that might have been better using delegation as opposed to
> inheritence, and the need to separate out the creation & connection of a
> socket, as well as the need for a callback when a socket connects.
> 
> Thanks,
>  Sam
> 
> On 5/9/06, Roland Weber <http-async@dubioso.net> wrote:
> >
> > Hi Brian,
> >
> > > Is there a guesstimated timeline for any work to start on a select  and
> > > callback based http client component? Callback not in the sense  of the
> > > description of the Async implementation on the wiki (2 threads  per
> > > connection) but in N threads per M connections where M is much  bigger
> > > than N?
> >
> > I am less optimistic than Oleg, but that may be due to my
> > missing knowledge about NIO, in Java and in HttpComponents.
> > I also don't know yet what plans Oleg has for this component.
> >
> > The interface in HttpCore is synchronous. So even without any
> > background threads, you will need M application threads to
> > make use of M connections simultaneously. If that is what
> > you've asked for, Oleg's answer is appropriate.
> >
> > For HttpAsync, I will be happy if it is functionally complete
> > and unit tested with a plain IO implementation by next spring.
> > The implementation itself should not be a problem, although it
> > depends on connection management functionality not yet available.
> > But unit testing will be a tough nut to crack. Test calls to
> > some setters and getters may suffice to trick the statistics
> > (or not), but the real problem that needs to be tested is the
> > thread interaction and synchronization. Imagine a fake test
> > server that secretly synchronizes with the test case threads
> > to trigger specific situations in the dispatcher...
> > I do not intend to spend time on implementing a NIO based dispatcher
> > before the plain one is complete, at least with a non-pipelining
> > implementation (1 thread per connection). Somebody else might jump
> > ahead and work on that before the plain IO dispatcher is complete,
> > but even then I guess that there are dependencies on my progress.
> >
> > I am bound for another two months by lecturing. Then there are
> > almost three months in which I have to find a balance between
> > working on HttpComponents and enjoying the summer sun. The work
> > on HttpComponents will be distributed between HttpAsync and
> > some infrastructure stuff like build process or versioning,
> > which could affect HttpCore and therefore has a higher priority.
> > In October it's back to lecturing until the end of the year.
> >
> > I have posted a medium term plan in March:
> >
> > http://mail-archives.apache.org/mod_mbox/jakarta-httpclient-dev/200603.mbox/%3c44147B9F.4000606@dubioso.net%3e
> > Step 1 is complete, step 4 is likely to see some progress in the
> > following weeks. Step 2 will lay dormant for another 2 months.
> > That's where we stand right now.
> >
> > cheers,
> > Roland
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
> >
> >


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org


Mime
View raw message