ant-ivy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From testn <te...@doramail.com>
Subject Re: ivy:retrieve performance
Date Thu, 14 Jun 2007 14:41:25 GMT

I use ivy 2.0 alpha 1. I will open an issue. BTW, 8192 is the magic number in
BufferedReader/BufferedWriter library. It came from the old day where you
don't have a lot of memory. I think it's now different. We have much more
memory available. We should kick it up a notch.



testn wrote:
> 
> Hi Hanin,
> 
> I'm pretty sure. I fired up filemon to check it. I use <ivy:retrieve
> sync="true" /> and it seems to copy all the file everytime. 
> 
> 10:11:55 AM	java.exe:6380	READ
> C:\workspace\repo\.ivy\cache\org.springframework\spring-core\jars\spring-core-2.0.3.jar
> SUCCESS	Offset: 163840 Length: 8192	
> 10:11:55 AM	java.exe:6380	WRITE
> C:\workspace\mystuff\lib\spring-core-2.0.3.jar	SUCCESS	Offset: 163840
> Length: 8192	
> 
> Does it have anything to do if I don't have md5 and sha1 files?
> 
> Thanks
> 
> 
> 
> Xavier Hanin wrote:
>> 
>> On 6/14/07, Gilles Scokart <gscokart@gmail.com> wrote:
>>>
>>> > 2. The buffer in FileUtils.java is too small. It's set at 8192. It
>>> seems
>>> > to
>>> > > be much better for me to set it much larger. This is due to the fact
>>> > that
>>> > > it
>>> > > needs to read and write simultaneously. The bigger the buffer is,
>>> the
>>> > > smaller number of time, HD header has to move. For me, 65536 seems
>>> to
>>> > > perform much better but I haven't tried other numbers.
>>> >
>>> >
>>> > I'd like to get more feedback on this. One use case is not the other.
>>> This
>>> > size has been borrowed from Ant copy mechanism. Maybe what we could do
>>> is
>>> > make this configurable, so that one could adapt to its needs. Or try
>>> to
>>> > guess a good size depending on the size (when it's possible to get an
>>> idea
>>> > of the size before copying).
>>> >
>>> > Xavier
>>> >
>>>
>>>
>>> Couldn't we use the nio for that?  (See
>>> http://www.javalobby.org/java/forums/t17036.html)
>> 
>> 
>> According to comments 10 and 11 NIO have bad performance for large files
>> on
>> linux, and input stream with byte buffer is pretty close to NIO for small
>> files (see comment 13 conclusion). So I'm not sure switching to NIO would
>> indeed help a lot. According to the tests in the thread you pointed using
>> a
>> 64kB buffer seems to be a good choice (which confirms  testn tests), at
>> least for large files. OTOH the last conclusion (comment 17) is
>> different.
>> So I don't really know what to think about that. We should make some
>> tests
>> on several platforms and jvms to draw conclusion ourself, but it takes
>> time.
>> 
>> 
>> Xavier
>> 
>> Gilles
>>>
>>>
>>>
>>>
>> 
>> 
>> -- 
>> Xavier Hanin - Independent Java Consultant
>> Manage your dependencies with Ivy!
>> http://incubator.apache.org/ivy/
>> 
>> 
> 
> 

-- 
View this message in context: http://www.nabble.com/ivy%3Aretrieve-performance-tf3907253.html#a11121501
Sent from the ivy-user mailing list archive at Nabble.com.


Mime
View raw message