ant-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin Todorov (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (IVY-1197) OutOfMemoryError duriong ivy:publish
Date Mon, 25 Feb 2013 10:44:13 GMT

    [ https://issues.apache.org/jira/browse/IVY-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13585777#comment-13585777
] 

Martin Todorov edited comment on IVY-1197 at 2/25/13 10:44 AM:
---------------------------------------------------------------

This issue still exists in 2.3.0, despite the fact it is mentioned as fixed in the release
notes of 2.3.0.

It's ridiculous to not be able to deploy files which are larger than 64 MB. I have tried it
with the proposed workaround with httpclient by adding it to the CLASSPATH. Yes, it somewhat
works. It does upload the file. But in my case the file is over half a gig. Deploying that
with Ivy takes roughly around 2 hr 30 mins, whereas under Maven (with no extra memory settings)
this takes 5 mins 25 secs.

Instead of using the HTTPURLConnection, can't you use the httpclient library, or an alternative
implementation?

Even on a 1.6.0_31 64-bit SDK with -Xms1700m -Xmx2048m you can't deploy a file larger than
512 MB, before getting:
{code}
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOf(Arrays.java:2786)
        at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
        at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
        at org.apache.ivy.util.FileUtil.copy(FileUtil.java:183)
        at org.apache.ivy.util.FileUtil.copy(FileUtil.java:162)
        at org.apache.ivy.util.url.BasicURLHandler.upload(BasicURLHandler.java:256)
        at org.apache.ivy.util.url.URLHandlerDispatcher.upload(URLHandlerDispatcher.java:82)
        at org.apache.ivy.util.FileUtil.copy(FileUtil.java:150)
        at org.apache.ivy.plugins.repository.url.URLRepository.put(URLRepository.java:84)
        at org.apache.ivy.plugins.repository.AbstractRepository.put(AbstractRepository.java:130)
        at org.apache.ivy.plugins.resolver.RepositoryResolver.put(RepositoryResolver.java:234)
        at org.apache.ivy.plugins.resolver.RepositoryResolver.publish(RepositoryResolver.java:216)
        at org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:275)
        at org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:254)
        at org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:166)
        at org.apache.ivy.Ivy.publish(Ivy.java:615)
        at org.apache.ivy.Main.run(Main.java:304)
        at org.apache.ivy.Main.main(Main.java:179)
{code}

I am deploying a zip file to Nexus. I tried the same with Maven 3.0.4 and it took five and
a half minutes with no extra settings.

I even tried further increasing the memory to twice the file size. That did not work either.

This is a show-stopper for us to adopt Ivy.

In theory, if this bug is fixed, how soon do you reckon the next version of Ivy will be released?
                
      was (Author: carlspring):
    This issue still exists in 2.3.0, despite the fact it is mentioned as fixed in the release
notes of 2.3.0.

It's ridiculous to not be able to deploy files which are larger than 64 MB. I have tried it
with the proposed workaround with httpclient by adding it to the CLASSPATH.

Instead of using the HTTPURLConnection, can't you use the httpclient library, or an alternative
implementation?

Even on a 1.6.0_31 64-bit SDK with -Xms1700m -Xmx2048m you can't deploy a file larger than
512 MB, before getting:
{code}
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOf(Arrays.java:2786)
        at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
        at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
        at org.apache.ivy.util.FileUtil.copy(FileUtil.java:183)
        at org.apache.ivy.util.FileUtil.copy(FileUtil.java:162)
        at org.apache.ivy.util.url.BasicURLHandler.upload(BasicURLHandler.java:256)
        at org.apache.ivy.util.url.URLHandlerDispatcher.upload(URLHandlerDispatcher.java:82)
        at org.apache.ivy.util.FileUtil.copy(FileUtil.java:150)
        at org.apache.ivy.plugins.repository.url.URLRepository.put(URLRepository.java:84)
        at org.apache.ivy.plugins.repository.AbstractRepository.put(AbstractRepository.java:130)
        at org.apache.ivy.plugins.resolver.RepositoryResolver.put(RepositoryResolver.java:234)
        at org.apache.ivy.plugins.resolver.RepositoryResolver.publish(RepositoryResolver.java:216)
        at org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:275)
        at org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:254)
        at org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:166)
        at org.apache.ivy.Ivy.publish(Ivy.java:615)
        at org.apache.ivy.Main.run(Main.java:304)
        at org.apache.ivy.Main.main(Main.java:179)
{code}

I am deploying a zip file to Nexus. I tried the same with Maven 3.0.4 and it took five and
a half minutes with no extra settings.

I even tried further increasing the memory to twice the file size. That did not work either.

This is a show-stopper for us to adopt Ivy.

                  
> OutOfMemoryError duriong ivy:publish
> ------------------------------------
>
>                 Key: IVY-1197
>                 URL: https://issues.apache.org/jira/browse/IVY-1197
>             Project: Ivy
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 2.0
>            Reporter: Michael Rumpf
>         Attachments: ASF.LICENSE.NOT.GRANTED--clipboard.txt
>
>
> When publishing a large file, an OutOfMemoryError occurs.
> {code}
> [ivy:publish] 	published ppg to XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
> BUILD FAILED
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:152:
The following error occurred while executing this line:
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:277:
java.lang.OutOfMemoryError: Java heap space
> 	at java.util.Arrays.copyOf(Arrays.java:2786)
> 	at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
> 	at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
> 	at org.apache.ivy.util.FileUtil.copy(FileUtil.java:168)
> 	at org.apache.ivy.util.url.BasicURLHandler.upload(BasicURLHandler.java:200)
> 	at org.apache.ivy.util.url.URLHandlerDispatcher.upload(URLHandlerDispatcher.java:82)
> 	at org.apache.ivy.util.FileUtil.copy(FileUtil.java:140)
> 	at org.apache.ivy.plugins.repository.url.URLRepository.put(URLRepository.java:85)
> 	at org.apache.ivy.plugins.repository.AbstractRepository.put(AbstractRepository.java:130)
> 	at org.apache.ivy.plugins.resolver.RepositoryResolver.put(RepositoryResolver.java:219)
> 	at org.apache.ivy.plugins.resolver.RepositoryResolver.publish(RepositoryResolver.java:209)
> 	at org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:282)
> 	at org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:261)
> 	at org.apache.ivy.core.publish.PublishEngine.publish(PublishEngine.java:170)
> 	at org.apache.ivy.Ivy.publish(Ivy.java:600)
> 	at org.apache.ivy.ant.IvyPublish.doExecute(IvyPublish.java:299)
> 	at org.apache.ivy.ant.IvyTask.execute(IvyTask.java:277)
> 	at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
> 	at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> 	at org.apache.tools.ant.Task.perform(Task.java:348)
> 	at org.apache.tools.ant.Target.execute(Target.java:390)
> 	at org.apache.tools.ant.Target.performTasks(Target.java:411)
> 	at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
> 	at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
> 	at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
> 	at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
> 	at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
> 	at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> Total time: 14 minutes 24 seconds
> Finished: FAILURE
> {code}
> The size of the file that is being uploaded is: 687712714, so around 650-700MB.
> The publish task is part of a Hudson Ant build where the artefacts are published to an
Artifactory repository at the end.
> I have given the Job 1300MB for the max heap size.
> It seems as if the whole file is loaded into memory for the upload.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message