ant-ivy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Buck, Robert" <rb...@verisign.com>
Subject RE: ivy:retrieve performance
Date Thu, 14 Jun 2007 14:08:31 GMT
Oh, to add one more comment. The performance should not be an issue with
NIO if the code follows a few commonly documented strategies.

What is an issue are the gaps in Sun's implementation of NIO. We, at
VeriSign, discovered a number of issues in NIO where Sun failed to
complete the NIO code base. There are calls, that when made to NFS for
instance, will literally cause an exception to be thrown indicating
"SuchAndSuch API is not presently supported on NFS" -- this is where the
real frustration with NIO is in my own mind. We have found some places
where we MAY have to literally write some code in C++, write a JNI
interface, and use our own IO logic instead of Sun's.

Summary: For the simple case NIO is probably fine. If you try to do
something slightly on the edge, you may run into all sorts of problems
with NIO.

-Bob

> -----Original Message-----
> From: Buck, Robert [mailto:rbuck@verisign.com] 
> Sent: Thursday, June 14, 2007 6:46 AM
> To: ivy-user@incubator.apache.org
> Subject: RE: ivy:retrieve performance
> 
> Folks,
> 
> We benchmarked a number of JDK IO API's for an internal 
> project. To neutralize any questions regarding NIO vs Old IO, 
> please take a look at the attached diagram. These rates will 
> be largely identical on both Linux and Windows. Blue line 
> NIO, red line old io.
> 
> If you do not get the attached JPEG file, let me know. I can 
> send it directly to you if you so request.
> 
> Cheers,
> 
> Bob 
> 
> > -----Original Message-----
> > From: Xavier Hanin [mailto:xavier.hanin@gmail.com]
> > Sent: Thursday, June 14, 2007 6:22 AM
> > To: ivy-user@incubator.apache.org
> > Subject: Re: ivy:retrieve performance
> > 
> > On 6/14/07, Gilles Scokart <gscokart@gmail.com> wrote:
> > >
> > > > 2. The buffer in FileUtils.java is too small. It's set at
> > 8192. It
> > > > seems to
> > > > > be much better for me to set it much larger. This is 
> due to the 
> > > > > fact
> > > > that
> > > > > it
> > > > > needs to read and write simultaneously. The bigger the
> > buffer is,
> > > > > the smaller number of time, HD header has to move. For
> > me, 65536
> > > > > seems to perform much better but I haven't tried 
> other numbers.
> > > >
> > > >
> > > > I'd like to get more feedback on this. One use case is
> > not the other.
> > > This
> > > > size has been borrowed from Ant copy mechanism. Maybe
> > what we could
> > > > do
> > > is
> > > > make this configurable, so that one could adapt to its
> > needs. Or try
> > > > to guess a good size depending on the size (when it's 
> possible to 
> > > > get an
> > > idea
> > > > of the size before copying).
> > > >
> > > > Xavier
> > > >
> > >
> > >
> > > Couldn't we use the nio for that?  (See
> > > http://www.javalobby.org/java/forums/t17036.html)
> > 
> > 
> > According to comments 10 and 11 NIO have bad performance for large 
> > files on linux, and input stream with byte buffer is pretty 
> close to 
> > NIO for small files (see comment 13 conclusion). So I'm not sure 
> > switching to NIO would indeed help a lot. According to the tests in 
> > the thread you pointed using a 64kB buffer seems to be a 
> good choice 
> > (which confirms  testn tests), at least for large files. 
> OTOH the last 
> > conclusion (comment 17) is different.
> > So I don't really know what to think about that. We should 
> make some 
> > tests on several platforms and jvms to draw conclusion 
> ourself, but it 
> > takes time.
> > 
> > 
> > Xavier
> > 
> > Gilles
> > >
> > >
> > >
> > >
> > 
> > 
> > --
> > Xavier Hanin - Independent Java Consultant Manage your 
> > dependencies with Ivy!
> > http://incubator.apache.org/ivy/
> > 
> 

Mime
View raw message