ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Rice" <rap...@gmail.com>
Subject Re: PermGen OutOfMemoryError in custom Iterator task, and ForEach - Suggestions Please
Date Thu, 26 Oct 2006 17:05:52 GMT
Peter,

Your suggestions were quite helpful.  The fix for me was to utilize a
post-ant1.7.0beta3 svn copy of ant.  I tested my Iterator-based task, as
well as <foreach> using the most recent svn ant.  In both cases, my builds
ran without OOME.  So it would seem that the memory leak squashing is paying
off.  Hopefully these will make it into the proposed beta4.

I do have a bonus, follow-up question.  Some of your dialog in the previous
post leads me to believe I am behind in ant-contrib knowledge.  I am using
ant-contrib-0.3, as made available through :
http://ant-contrib.sourceforge.net/.  While I realize that this is a rather
old build, I have found nothing newer.  In particular, your reference to
"Use antcontrib:var@unset=true to reuse properties" leaves me a little
puzzled as I have not seen this construct or a reference to a <var> task.
Also, you had suggested to not use antcall.  The version of <foreach> that I
have uses antcall internally, which leaves me further confused.

Please point me in the correct direction, especially if there is
documentation available.

Thanks,
Robert Rice

On 10/24/06, Peter Reilly <peter.kitt.reilly@gmail.com> wrote:
>
> On 10/23/06, Robert Rice <robert@windermere.com> wrote:
> > I recently created a custom task that is an Iterator.  The goal of the
> task
> > is to accept a large date range ( like 3 years ) and split it into
> smaller
> > date ranges ( like 1 day at a time ).  I do a lot of work with date
> ranges.
> > Examples are running queries or fetching files that expect a time range.
> >
> > At each iteration, the task sets a couple of properties that represent
> the
> > limits of the smaller date range.  At this point, the task acts very
> much
> > like <antcall>.  It requires one or more targets.  CallTaget tasks are
> > dynamically constructed for each target for each iteration.  They are
> then
> > executed.   The targets that are called are rather large themselves,
> > ultimately including about 10 targets or dependencies.
> >
> > The short is that I am running out of memory.  About 70 iterations in (
> > representing 70 days of work that has been accomplished ), I get an
> > OutOfMemoryError.  It looks something like:
> >
> > http-data:
> >     [timer] Timer started.  Tasks started.
> > Exception in thread "Thread-72" java.lang.OutOfMemoryError: PermGen
> space
> >     [timer] Timer interrupted. Tasks completed.
> >
> > And an example stacktrace:
> > Exception in thread "Thread-71"
> > /home/robert/dvl/java/bin/RetsDataCollector.xml:161: The following error
> > occurred while executing this
> > line:/home/robert/dvl/java/bin/RetsDataCollector.xml:127: Could not
> create
> > type mget due to java.lang.OutOfMemoryError: PermGen space        at
> > org.apache.tools.ant.ProjectHelper.addLocationToBuildException(
> > ProjectHelper.java:539)
> >         at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:384)
> >         at org.apache.tools.ant.taskdefs.CallTarget.execute(
> CallTarget.java
> > :107)
> >         at net.sf.antcontrib.logic.ForEach.execute(Unknown Source)
> >         at org.apache.tools.ant.UnknownElement.execute(
> UnknownElement.java
> > :275)
> >         at org.apache.tools.ant.Task.perform(Task.java:364)
> >         at com.windermere.collector.task.Timer$TasksThread.run(Unknown
> > Source)
> > Caused by: /home/robert/dvl/java/bin/RetsDataCollector.xml:127: Could
> not
> > create type mget due to java.lang.OutOfMemoryError: PermGen
> space        at
> > org.apache.tools.ant.AntTypeDefinition.createAndSet(
> AntTypeDefinition.java
> > :281)
> >         at org.apache.tools.ant.AntTypeDefinition.icreate(
> > AntTypeDefinition.java:196)
> >         at org.apache.tools.ant.AntTypeDefinition.create(
> > AntTypeDefinition.java:183)
> >         at org.apache.tools.ant.ComponentHelper.createComponent(
> > ComponentHelper.java:199)
> >         at org.apache.tools.ant.ComponentHelper.createComponent(
> > ComponentHelper.java:176)
> >         at org.apache.tools.ant.UnknownElement.makeObject(
> > UnknownElement.java:388)
> >         at org.apache.tools.ant.UnknownElement.maybeConfigure(
> > UnknownElement.java:158)
> >         at org.apache.tools.ant.Task.perform(Task.java:363)
> >         at org.apache.tools.ant.Target.execute(Target.java:341)
> >         at org.apache.tools.ant.Target.performTasks(Target.java:369)
> >         at org.apache.tools.ant.Project.executeSortedTargets(
> Project.java
> > :1216)
> >         at
> org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(
> > SingleCheckExecutor.java:37)        at
> > org.apache.tools.ant.Project.executeTargets(Project.java:1068)
> >         at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:382)
> >         ... 5 more
> > Caused by: java.lang.OutOfMemoryError: PermGen space
> >
> > It looks to me like each iteration adds to the memory requirement, until
> I
> > run out of memory.  Ideally, at each iteration, the dynamically created
> > <antcall> tasks would no longer be needed and would be garbage
> collected.
> > Is it because these <antcall> tasks are ultimately owned by the
> containing
> > iterator task?
> >
> > I'm looking for suggestions on how to get around this problem.  Can I
> model
> > my custom task in a different way internally to avoid this problem?
>
> The best way would be to not use antcall, but try to do every thing
> in the same project.
>
> Use <macrodef> instread of targets
> Use antcontrib:for as the looping construct
> Use antcontrib:var@unset=true to reuse properties
>
> > As an aside, I thought I would see if the ant-contrib <foreach> task had
> the
> > same end result.  I chose to do this since it is similar.  It instead
> loops
> > across a list, then constructing <antcall> tasks within each loop.  It
> does
> > indeed also run out of memory, at about the same point as my custom
> task.
> >
> > I can provide code snippets of the custom task if that will help.  I'm
> > guessing others have come across this problem and may have a solution
> > without digging into the code.
>
> Please try to provide a complete working example using the
> antcontrib:foreach.
> I am currently squashing memory leakage bugs and am interrested
> in simple examples that get OOME.
>
> Also try with a svn copy of ant, some memory leakages have
> been fixed since ant1.7.0beta3.
>
> Peter
>
>
>
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> For additional commands, e-mail: dev-help@ant.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message