hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Boxer, Aaron" <Aaron.Bo...@uhn.ca>
Subject RE: ZeroCopyPut mystery
Date Fri, 17 Jan 2014 17:03:09 GMT

On Fri, 2014-01-17 at 16:28 +0000, Boxer, Aaron wrote:
>  I made a simple test: I wrote a few lines of Java using  File channel and transferTo(...)
method to read files from my DVD and write to the hard drive.  With this simple test,  the
transferTo(...) method gets  about 4 MB /S transfer rate to disk.
> 
> Then, I put some timing code into the LengthDelimitedEncoder.transfer(...) method, and
ran my  application, which copies files from DVD
> to a socket.  And the transferTo(...) from file to socket was getting about 0.7 MB /s
.   This is a 6 X degradation in performance.  
> And, if you recall, if the files are already in the OS file cache, I get about 100 MB/S,
so the socket is not slowing me down.
> 
> I have done a little more testing:  transferTo from DVD to local socket gets 3-4 MB/S
transfer rate.   
> I tried using a thread pool, and performance degrades (not surprisingly) as thread count
rises from 1.
> 
> Since DVD seeks are very slow, multiple threads trying to read at the same time will
kill performance.
> 
> But, I am not sure this is the issue here, because I have set
> 
>         connManager.setMaxTotal(1);
>         connManager.setDefaultMaxPerRoute(1);
> 		
> 
> So, there should only be one thread doing the transfer (is this true?)
> 

Actually HttpAsyncClient uses multiple i/o dispatch threads, one per CPU core. I suppose even
with a cap of 1 total connection multiple threads might end up trying to access the same physical
device.

You can force HttpAsyncClient to use only one i/o dispatch thread by using a custom IOReactorConfig


Thanks, Oleg.  Here is my reactor config:

   IOReactorConfig ioReactorConfig = IOReactorConfig.custom()
                .setIoThreadCount(1)
                .setConnectTimeout(30000)
                .setSoTimeout(30000)
                .setTcpNoDelay(true)
                .build();

And the problem is still there with a thread count of 1.




---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org


This e-mail may contain confidential and/or privileged information for the sole use of the
intended recipient. 
Any review or distribution by anyone other than the person for whom it was originally intended
is strictly prohibited. 
If you have received this e-mail in error, please contact the sender and delete all copies.

Opinions, conclusions or other information contained in this e-mail may not be that of the
organization.

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org

Mime
View raw message