hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sam Berlin <sber...@gmail.com>
Subject Re: [HttpCommon] NIO is one huge disappointment
Date Wed, 17 Aug 2005 21:06:25 GMT
Hi Oleg,

While it may not make a difference, one important thing with using the
NIO package in non-blocking mode is to read as much as possible from
the channel during a single select.  That is, in the
NIOSelectReceiever.process method, the part that currently is:

            for (;;) {
                selector.select(SO_TIMEOUT);
                int l = channel.read(tmp);
                if (l == -1) {
                    break;
                }
                tmp.clear();
            }

should be:

            for (;;) {
                selector.select(SO_TIMEOUT);
                int l = channel.read(tmp);
                while( l > 0 ) {
                    // process data.
                    tmp.clear();
                    l = channel.read(tmp);
                }

                if (l == -1) {
                    break;
                }
                
                tmp.clear();
            }

This ensures that as much data as is on the Socket/Channel can be
processed in a single statement without incurring the wrath of
multiple selects.

Thanks,
 Sam


On 8/17/05, Oleg Kalnichevski <olegk@apache.org> wrote:
> Folks,
> I have spend past several miserable nights analyzing the performance of
> the new Coyote HTTP connector. I have discovered that HttpCommon code
> was horribly slow for larger request/response bodies, especially
> chunk-encoded, on my Linux box [1], whereas it seemed almost fine on a
> much slower WinXP laptop of my wife [2]. To cut a long and sad story
> short, after some investigations I found out that the culprit was NIO.
> The way I see it, NIO, as presently implemented in Sun's JREs for Linux,
> simply sucks. Actually blocking NIO appears more or less okay. The real
> problem is the NIO channel selector, which proves horribly expensive in
> terms of performance (we DO have to use a selector on the socket
> channel, because it is the only way (I know of) to implement socket
> timeout with NIO).
> 
> I have written a small test app to demonstrate the problem:
> http://svn.apache.org/repos/asf/jakarta/httpclient/trunk/http-common/src/test/tests/performance/NIOvsOldIO.java
> 
> This is what I get on my Linux box
> =========================================
> Old IO average time (ms): 1274
> Blocking NIO average time (ms): 1364
> NIO with Select average time (ms): 4981
> =========================================
> 
> Bottom line: NIO may still be a better model for some special cases such
> as instant messaging where one can have thousands of mostly idle
> connections with fairly small and infrequent data packets. At the same
> time, I have come to a conclusion that NIO makes no sense of what so
> ever for synchronous HTTP (servlets, for instance), where large
> request/response entities need to be consumed/produced using
> InputStream/OutputStream interfaces, data tends to come in steady
> streams of chunks, and connections are relatively short-lived.
> 
> I intent to remove all the NIO related class from HttpCommon and put
> them in the HttpAsynch module, where they may serve as a starting point
> for the asynchronous HTTP implementation. Please take a look at the test
> app and complain loudly if you think something is wrong. Otherwise I'll
> go ahead and get rid of NIO code in HttpCommon.
> 
> Oleg
> ===
> [1] Dell Dimension 8300, Pentium 4 3.00GHz, 512MB, Fedora Core 4,
> 2.6.11-1.1369_FC4smp
> [2] A pile of old trash running Windows XP Home SP2 (rather badly)
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
> 
>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org


Mime
View raw message