ant-ivy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Buck, Robert" <rb...@verisign.com>
Subject RE: ivy:retrieve performance
Date Thu, 14 Jun 2007 13:45:31 GMT
Folks,

We benchmarked a number of JDK IO API's for an internal project. To
neutralize any questions regarding NIO vs Old IO, please take a look at
the attached diagram. These rates will be largely identical on both
Linux and Windows. Blue line NIO, red line old io.

If you do not get the attached JPEG file, let me know. I can send it
directly to you if you so request.

Cheers,

Bob 

> -----Original Message-----
> From: Xavier Hanin [mailto:xavier.hanin@gmail.com] 
> Sent: Thursday, June 14, 2007 6:22 AM
> To: ivy-user@incubator.apache.org
> Subject: Re: ivy:retrieve performance
> 
> On 6/14/07, Gilles Scokart <gscokart@gmail.com> wrote:
> >
> > > 2. The buffer in FileUtils.java is too small. It's set at 
> 8192. It 
> > > seems to
> > > > be much better for me to set it much larger. This is due to the 
> > > > fact
> > > that
> > > > it
> > > > needs to read and write simultaneously. The bigger the 
> buffer is, 
> > > > the smaller number of time, HD header has to move. For 
> me, 65536 
> > > > seems to perform much better but I haven't tried other numbers.
> > >
> > >
> > > I'd like to get more feedback on this. One use case is 
> not the other.
> > This
> > > size has been borrowed from Ant copy mechanism. Maybe 
> what we could 
> > > do
> > is
> > > make this configurable, so that one could adapt to its 
> needs. Or try 
> > > to guess a good size depending on the size (when it's possible to 
> > > get an
> > idea
> > > of the size before copying).
> > >
> > > Xavier
> > >
> >
> >
> > Couldn't we use the nio for that?  (See
> > http://www.javalobby.org/java/forums/t17036.html)
> 
> 
> According to comments 10 and 11 NIO have bad performance for 
> large files on linux, and input stream with byte buffer is 
> pretty close to NIO for small files (see comment 13 
> conclusion). So I'm not sure switching to NIO would indeed 
> help a lot. According to the tests in the thread you pointed 
> using a 64kB buffer seems to be a good choice (which confirms 
>  testn tests), at least for large files. OTOH the last 
> conclusion (comment 17) is different.
> So I don't really know what to think about that. We should 
> make some tests on several platforms and jvms to draw 
> conclusion ourself, but it takes time.
> 
> 
> Xavier
> 
> Gilles
> >
> >
> >
> >
> 
> 
> --
> Xavier Hanin - Independent Java Consultant Manage your 
> dependencies with Ivy!
> http://incubator.apache.org/ivy/
> 

Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message