ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Rice" <rob...@windermere.com>
Subject PermGen OutOfMemoryError in custom Iterator task, and ForEach - Suggestions Please
Date Mon, 23 Oct 2006 18:57:48 GMT
I recently created a custom task that is an Iterator.  The goal of the task
is to accept a large date range ( like 3 years ) and split it into smaller
date ranges ( like 1 day at a time ).  I do a lot of work with date ranges.
Examples are running queries or fetching files that expect a time range.

At each iteration, the task sets a couple of properties that represent the
limits of the smaller date range.  At this point, the task acts very much
like <antcall>.  It requires one or more targets.  CallTaget tasks are
dynamically constructed for each target for each iteration.  They are then
executed.   The targets that are called are rather large themselves,
ultimately including about 10 targets or dependencies.

The short is that I am running out of memory.  About 70 iterations in (
representing 70 days of work that has been accomplished ), I get an
OutOfMemoryError.  It looks something like:

http-data:
    [timer] Timer started.  Tasks started.
Exception in thread "Thread-72" java.lang.OutOfMemoryError: PermGen space
    [timer] Timer interrupted. Tasks completed.

And an example stacktrace:
Exception in thread "Thread-71"
/home/robert/dvl/java/bin/RetsDataCollector.xml:161: The following error
occurred while executing this
line:/home/robert/dvl/java/bin/RetsDataCollector.xml:127: Could not create
type mget due to java.lang.OutOfMemoryError: PermGen space        at
org.apache.tools.ant.ProjectHelper.addLocationToBuildException(
ProjectHelper.java:539)
        at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:384)
        at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java
:107)
        at net.sf.antcontrib.logic.ForEach.execute(Unknown Source)
        at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java
:275)
        at org.apache.tools.ant.Task.perform(Task.java:364)
        at com.windermere.collector.task.Timer$TasksThread.run(Unknown
Source)
Caused by: /home/robert/dvl/java/bin/RetsDataCollector.xml:127: Could not
create type mget due to java.lang.OutOfMemoryError: PermGen space        at
org.apache.tools.ant.AntTypeDefinition.createAndSet(AntTypeDefinition.java
:281)
        at org.apache.tools.ant.AntTypeDefinition.icreate(
AntTypeDefinition.java:196)
        at org.apache.tools.ant.AntTypeDefinition.create(
AntTypeDefinition.java:183)
        at org.apache.tools.ant.ComponentHelper.createComponent(
ComponentHelper.java:199)
        at org.apache.tools.ant.ComponentHelper.createComponent(
ComponentHelper.java:176)
        at org.apache.tools.ant.UnknownElement.makeObject(
UnknownElement.java:388)
        at org.apache.tools.ant.UnknownElement.maybeConfigure(
UnknownElement.java:158)
        at org.apache.tools.ant.Task.perform(Task.java:363)
        at org.apache.tools.ant.Target.execute(Target.java:341)
        at org.apache.tools.ant.Target.performTasks(Target.java:369)
        at org.apache.tools.ant.Project.executeSortedTargets(Project.java
:1216)
        at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(
SingleCheckExecutor.java:37)        at
org.apache.tools.ant.Project.executeTargets(Project.java:1068)
        at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:382)
        ... 5 more
Caused by: java.lang.OutOfMemoryError: PermGen space

It looks to me like each iteration adds to the memory requirement, until I
run out of memory.  Ideally, at each iteration, the dynamically created
<antcall> tasks would no longer be needed and would be garbage collected.
Is it because these <antcall> tasks are ultimately owned by the containing
iterator task?

I'm looking for suggestions on how to get around this problem.  Can I model
my custom task in a different way internally to avoid this problem?

As an aside, I thought I would see if the ant-contrib <foreach> task had the
same end result.  I chose to do this since it is similar.  It instead loops
across a list, then constructing <antcall> tasks within each loop.  It does
indeed also run out of memory, at about the same point as my custom task.

I can provide code snippets of the custom task if that will help.  I'm
guessing others have come across this problem and may have a solution
without digging into the code.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message