ant-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin Todorov (JIRA)" <>
Subject [jira] [Commented] (IVY-1197) OutOfMemoryError duriong ivy:publish
Date Sun, 03 Mar 2013 03:07:13 GMT


Martin Todorov commented on IVY-1197:


I did try it with the httpclient included in the latest Ivy. I added it to the classpath.
Yes, indeed -- the OOME is no longer there, but uploading the file takes a ridiculously long
time. I tried with a file of 1.7 GB by deploying and artifact to Nexus with (first) Ivy and
(then) Maven. Ivy took ~ 2 hr 30 mins, whereas under Maven 3.0.4 (without any extra configuration)
it took ~ 5 min 25 sec. This is a huge difference, even with the httpclient on the classpath.

I can't invest more time into analyzing this myself, but I can tell you what is most-likely
happening. Check these two links for further details:
- [ Maven 3.0.4's release notes]
- [ Guide to HTTP settings]

Check the second link in regards to the http.protocol.expect-continue=true part. This explains
what is happening. Apart from that, the PosterOutputStream is actually extending the ByteArrayOutputStream.
If you have a look at it, it clones the byte array. This leads to using twice as much memory,
as the file (at least).

I think this should be properly addressed and that it is quite easily reproducible.

> OutOfMemoryError duriong ivy:publish
> ------------------------------------
>                 Key: IVY-1197
>                 URL:
>             Project: Ivy
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 2.0
>            Reporter: Michael Rumpf
>         Attachments: ASF.LICENSE.NOT.GRANTED--clipboard.txt
> When publishing a large file, an OutOfMemoryError occurs.
> {code}
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:152:
The following error occurred while executing this line:
> /export/build/hudson/jobs/ppg-rcp/workspace/ppg-rcp/com.daimler.ppg.rcp.builder/build-wrapper.xml:277:
java.lang.OutOfMemoryError: Java heap space
> 	at java.util.Arrays.copyOf(
> 	at
> 	at
> 	at org.apache.ivy.util.FileUtil.copy(
> 	at org.apache.ivy.util.url.BasicURLHandler.upload(
> 	at org.apache.ivy.util.url.URLHandlerDispatcher.upload(
> 	at org.apache.ivy.util.FileUtil.copy(
> 	at org.apache.ivy.plugins.repository.url.URLRepository.put(
> 	at org.apache.ivy.plugins.repository.AbstractRepository.put(
> 	at org.apache.ivy.plugins.resolver.RepositoryResolver.put(
> 	at org.apache.ivy.plugins.resolver.RepositoryResolver.publish(
> 	at org.apache.ivy.core.publish.PublishEngine.publish(
> 	at org.apache.ivy.core.publish.PublishEngine.publish(
> 	at org.apache.ivy.core.publish.PublishEngine.publish(
> 	at org.apache.ivy.Ivy.publish(
> 	at org.apache.ivy.ant.IvyPublish.doExecute(
> 	at org.apache.ivy.ant.IvyTask.execute(
> 	at
> 	at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> 	at java.lang.reflect.Method.invoke(
> 	at
> 	at
> 	at
> 	at
> 	at
> 	at
> 	at
> 	at
> 	at
> 	at
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> Total time: 14 minutes 24 seconds
> Finished: FAILURE
> {code}
> The size of the file that is being uploaded is: 687712714, so around 650-700MB.
> The publish task is part of a Hudson Ant build where the artefacts are published to an
Artifactory repository at the end.
> I have given the Job 1300MB for the max heap size.
> It seems as if the whole file is loaded into memory for the upload.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message