hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Kalnichevski <ol...@apache.org>
Subject Re: current state of httpclient 4.0
Date Sun, 10 Jun 2007 13:14:34 GMT
On Sun, 2007-06-10 at 14:10 +0200, Daniel Mueller wrote:
> > >  And since my main interest lies in the nio part
> > > of the httpcore, what needs to be done to have this part working with
> > the
> > > httpclient (if there's anything which is not working, not sure about
> > that)?
> > > Some pointers would be great, so I can speed up my understanding of the
> > > code.
> > >
> >
> > This used to be and still is a major point of contention. I believe NIO
> > makes very little sense for 'pure' HTTP servers and clients. To date NIO
> > has always tended to be significantly slower than classic I/O as long as
> > the number of I/O worker threads was below several thousands. I still do
> > not see a valid use case where a pure HTTP client may need more than
> > just a handful of I/O threads (a few hundred at most). That basically
> > eliminates all advantages of NIO for HttpClient.
> 
> 
> 
> Having said all that, I am very, very, very open to helping people build
> > client side components based on HttpCore NIO to be used in some HTTP
> > proxy or gateway services.
> >
> 
> I have a quite limited knowledge of NIO and IO in general as a matter of
> fact. Not much beside mostly theoretical stuff and these things often don't
> apply one to one in the field. Having said that my use case is the
> following:
> I want to build an access layer to the amazon S3 service. The S3 service has
> a SOAP and a REST API and I'm currently opting for the REST variant. This
> access layer will be used as one of the back ends of a processing pipeline
> inside a server. The rest of the server is built in SEDA style with event
> driven/asynchronous interfaces between the stages. That's why I need my
> access layer to be event driven/asynchronous. The logical step was then to
> think about implementing the access layer also in a asynchronous manner
> which is where NIO came into view because it encourages that kind of design.
> While investigating S3, REST, NIO and asynchronous (http) clients I stumbled
> accross several interesting things and one of them is the restlet API[1]. It
> looks as if it would solve a good portion of my demands for an asynchronous
> handling of resources, but has no asynchronous client implementation as of
> today. The one they do have is built on httpclient 3.x. While following that
> trail I ended up with your still alpha version of httpcomponents and decided
> to have a look.
> If you have a look at my use case and then think a bit further along that
> line of thought you end up with something that almost behaves like a gateway
> but uses webservices (REST or SOAP) as the backend. WS* and REST is all the
> rage today. While it certainly does not solve all the advertised problems,
> it does solve a couple. And building your servers in a non-blocking fashion
> is also gaining momentum (I have no good pointers for this one, it's simply
> a gut feeling). And the last link in the chain is the call to backend
> systems of any sort (web services, databases, file system, whatever). So
> maybe from this point of view, NIO makes sense after all.
> 

I am getting old and grumpy ;-) So, let me turn this around. If the raw
throughput and simplicity are not overriding concerns for your
architecture, I does make sense to take a very close look at the event
driven, non-blocking transport layer based on NIO. In your particular
case NIO does seem like a better choice.   


> Currently most of these ideas are still being formulated and I'm simply
> looking for ways to achieve what I have in mind. I've spotted your view on
> NIO http clients already somewhere else in the list-archives and would
> certainly be very interested if you could elaborate a bit.

All I am saying one should base their decisions on the set of
requirements and goals they are striving to achieve, not hype or trends.
NIO solves some problems posed by classic I/O paradigm at the expense of
introducing others (complexity and tendency to be prone to out of memory
conditions). So, as the old saying goes: pick up the right tool for the
job.

I ran enough performance tests to be convinced NIO always tended to be
perform worse than classic I/O in terms of raw throughput. Every new SUN
JRE to date that offered better NIO performance also happened to be more
efficient at switching contexts, so all gains made by NIO were
immediately offset by a more efficient threading support.  

Bottom-line, NIO comes at a certain price, see for yourself if it is
worth paying given your particular problem domain.

>  Those plans I
> have aren't set in stone and I want a solution which solves my problem,
> nothing more.
> 
> But if the architecture makes any sense at all, I do have to have the
> components to achieve it. And I will help to get things rolling where I can
> in the httpcomponents project.
> 

Any contribution to the project would be hugely appreciated. I guess all
present project committers would love to see more people on board. 

Cheers

Oleg

> Thanks and cheers
> Daniel
> 
> [1] http://www.restlet.org/


---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org


Mime
View raw message