ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Reilly" <peter.kitt.rei...@gmail.com>
Subject Re: PermGen OutOfMemoryError in custom Iterator task, and ForEach - Suggestions Please
Date Tue, 24 Oct 2006 08:58:07 GMT
On 10/23/06, Robert Rice <robert@windermere.com> wrote:
> I recently created a custom task that is an Iterator.  The goal of the task
> is to accept a large date range ( like 3 years ) and split it into smaller
> date ranges ( like 1 day at a time ).  I do a lot of work with date ranges.
> Examples are running queries or fetching files that expect a time range.
>
> At each iteration, the task sets a couple of properties that represent the
> limits of the smaller date range.  At this point, the task acts very much
> like <antcall>.  It requires one or more targets.  CallTaget tasks are
> dynamically constructed for each target for each iteration.  They are then
> executed.   The targets that are called are rather large themselves,
> ultimately including about 10 targets or dependencies.
>
> The short is that I am running out of memory.  About 70 iterations in (
> representing 70 days of work that has been accomplished ), I get an
> OutOfMemoryError.  It looks something like:
>
> http-data:
>     [timer] Timer started.  Tasks started.
> Exception in thread "Thread-72" java.lang.OutOfMemoryError: PermGen space
>     [timer] Timer interrupted. Tasks completed.
>
> And an example stacktrace:
> Exception in thread "Thread-71"
> /home/robert/dvl/java/bin/RetsDataCollector.xml:161: The following error
> occurred while executing this
> line:/home/robert/dvl/java/bin/RetsDataCollector.xml:127: Could not create
> type mget due to java.lang.OutOfMemoryError: PermGen space        at
> org.apache.tools.ant.ProjectHelper.addLocationToBuildException(
> ProjectHelper.java:539)
>         at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:384)
>         at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java
> :107)
>         at net.sf.antcontrib.logic.ForEach.execute(Unknown Source)
>         at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java
> :275)
>         at org.apache.tools.ant.Task.perform(Task.java:364)
>         at com.windermere.collector.task.Timer$TasksThread.run(Unknown
> Source)
> Caused by: /home/robert/dvl/java/bin/RetsDataCollector.xml:127: Could not
> create type mget due to java.lang.OutOfMemoryError: PermGen space        at
> org.apache.tools.ant.AntTypeDefinition.createAndSet(AntTypeDefinition.java
> :281)
>         at org.apache.tools.ant.AntTypeDefinition.icreate(
> AntTypeDefinition.java:196)
>         at org.apache.tools.ant.AntTypeDefinition.create(
> AntTypeDefinition.java:183)
>         at org.apache.tools.ant.ComponentHelper.createComponent(
> ComponentHelper.java:199)
>         at org.apache.tools.ant.ComponentHelper.createComponent(
> ComponentHelper.java:176)
>         at org.apache.tools.ant.UnknownElement.makeObject(
> UnknownElement.java:388)
>         at org.apache.tools.ant.UnknownElement.maybeConfigure(
> UnknownElement.java:158)
>         at org.apache.tools.ant.Task.perform(Task.java:363)
>         at org.apache.tools.ant.Target.execute(Target.java:341)
>         at org.apache.tools.ant.Target.performTasks(Target.java:369)
>         at org.apache.tools.ant.Project.executeSortedTargets(Project.java
> :1216)
>         at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(
> SingleCheckExecutor.java:37)        at
> org.apache.tools.ant.Project.executeTargets(Project.java:1068)
>         at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:382)
>         ... 5 more
> Caused by: java.lang.OutOfMemoryError: PermGen space
>
> It looks to me like each iteration adds to the memory requirement, until I
> run out of memory.  Ideally, at each iteration, the dynamically created
> <antcall> tasks would no longer be needed and would be garbage collected.
> Is it because these <antcall> tasks are ultimately owned by the containing
> iterator task?
>
> I'm looking for suggestions on how to get around this problem.  Can I model
> my custom task in a different way internally to avoid this problem?

The best way would be to not use antcall, but try to do every thing
in the same project.

Use <macrodef> instread of targets
Use antcontrib:for as the looping construct
Use antcontrib:var@unset=true to reuse properties

> As an aside, I thought I would see if the ant-contrib <foreach> task had the
> same end result.  I chose to do this since it is similar.  It instead loops
> across a list, then constructing <antcall> tasks within each loop.  It does
> indeed also run out of memory, at about the same point as my custom task.
>
> I can provide code snippets of the custom task if that will help.  I'm
> guessing others have come across this problem and may have a solution
> without digging into the code.

Please try to provide a complete working example using the antcontrib:foreach.
I am currently squashing memory leakage bugs and am interrested
in simple examples that get OOME.

Also try with a svn copy of ant, some memory leakages have
been fixed since ant1.7.0beta3.

Peter



>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org


Mime
View raw message