Return-Path: <anonymous@nagoya.betaversion.org>
Mailing-List: contact dev-help@ant.apache.org; run by ezmlm
Delivered-To: mailing list dev@ant.apache.org
Received: (qmail 76985 invoked from network); 1 May 2003 14:46:37 -0000
Received: from exchange.sun.com (192.18.33.10)
  by daedalus.apache.org with SMTP; 1 May 2003 14:46:37 -0000
Received: (qmail 207 invoked by uid 50); 1 May 2003 14:48:38 -0000
Date: 1 May 2003 14:48:38 -0000
Message-ID: <20030501144838.206.qmail@nagoya.betaversion.org>
From: bugzilla@apache.org
To: dev@ant.apache.org
Cc: 
Subject: DO NOT REPLY [Bug 19516] New:  -
    uncompressed zipping takes too much memory
X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N

DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG 
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://nagoya.apache.org/bugzilla/show_bug.cgi?id=19516>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND 
INSERTED IN THE BUG DATABASE.

http://nagoya.apache.org/bugzilla/show_bug.cgi?id=19516

uncompressed zipping takes too much memory

           Summary: uncompressed zipping takes too much memory
           Product: Ant
           Version: 1.5.3
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: Enhancement
          Priority: Other
         Component: Core tasks
        AssignedTo: dev@ant.apache.org
        ReportedBy: glen_lewis@hotmail.com


I use the <ear> task with compress='false' to create a 33MB .ear file, 
including a 17MB .war file and some smaller .jars.

Zip.addResources() calls Zip.zipFile() here:
   zipFile(zf.getInputStream(ze), ... );
The zipFile() method in turn checks whether the InputStream param has 
markSupported: if so, it marks, reads, and resets.  Else, it buffers the entire 
InputStream into a ByteArrayOutputStream/byte[].  That byte[] (for my build 
script) is huge!

Solution: change the line of code above to an InputStream that supports marking.
    zipFile( new BufferedInputStream(zf.getInputStream(ze)), ... );
AND, probably, close that InputStream!

regards,
Glen Lewis

HPROF output:


SITES BEGIN (ordered by live bytes) Thu May 01 09:42:37 2003
          percent         live       alloc'ed  stack class
 rank   self  accum    bytes objs   bytes objs trace name
    1 58.28% 58.28% 33554448    1 110366624  250 13654 [B
    2 30.62% 88.89% 17629416    1 33432128   44 13656 [B
    3  4.10% 92.99%  2359296    2 2359296    2     0 [I
... remaining sites each consume <100kb of memory, not important


TRACE 13654:
	java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:104)
	org.apache.tools.ant.taskdefs.Zip.zipFile(Zip.java:609)
	org.apache.tools.ant.taskdefs.Jar.zipFile(Jar.java:238)
	org.apache.tools.ant.taskdefs.Zip.zipFile(Zip.java:651)
	org.apache.tools.ant.taskdefs.Jar.zipFile(Jar.java:220)
	org.apache.tools.ant.taskdefs.Ear.zipFile(Ear.java:134)
	org.apache.tools.ant.taskdefs.Zip.addFiles(Zip.java:394)
	org.apache.tools.ant.taskdefs.Zip.execute(Zip.java:271)
	org.apache.tools.ant.Task.perform(Task.java:217)
	org.apache.tools.ant.Target.execute(Target.java:184)
	org.apache.tools.ant.Target.performTasks(Target.java:202)
	org.apache.tools.ant.Project.executeTarget(Project.java:601)
	org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:266)
	org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:120)
	org.apache.tools.ant.Task.perform(Task.java:217)
	org.apache.tools.ant.Target.execute(Target.java:184)
	org.apache.tools.ant.Target.performTasks(Target.java:202)
	org.apache.tools.ant.Project.executeTarget(Project.java:601)
	org.apache.tools.ant.Project.executeTargets(Project.java:560)
	org.apache.tools.ant.Main.runBuild(Main.java:454)
	org.apache.tools.ant.Main.start(Main.java:153)
	org.apache.tools.ant.Main.main(Main.java:176)


TRACE 13656:
	java.io.ByteArrayOutputStream.toByteArray
(ByteArrayOutputStream.java:146)
	org.apache.tools.ant.taskdefs.Zip.zipFile(Zip.java:612)
	org.apache.tools.ant.taskdefs.Jar.zipFile(Jar.java:238)
	org.apache.tools.ant.taskdefs.Zip.zipFile(Zip.java:651)
	org.apache.tools.ant.taskdefs.Jar.zipFile(Jar.java:220)
	org.apache.tools.ant.taskdefs.Ear.zipFile(Ear.java:134)
	org.apache.tools.ant.taskdefs.Zip.addFiles(Zip.java:394)
	org.apache.tools.ant.taskdefs.Zip.execute(Zip.java:271)
	org.apache.tools.ant.Task.perform(Task.java:217)
	org.apache.tools.ant.Target.execute(Target.java:184)
	org.apache.tools.ant.Target.performTasks(Target.java:202)
	org.apache.tools.ant.Project.executeTarget(Project.java:601)
	org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:266)
	org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:120)
	org.apache.tools.ant.Task.perform(Task.java:217)
	org.apache.tools.ant.Target.execute(Target.java:184)
	org.apache.tools.ant.Target.performTasks(Target.java:202)
	org.apache.tools.ant.Project.executeTarget(Project.java:601)
	org.apache.tools.ant.Project.executeTargets(Project.java:560)
	org.apache.tools.ant.Main.runBuild(Main.java:454)
	org.apache.tools.ant.Main.start(Main.java:153)
	org.apache.tools.ant.Main.main(Main.java:176)