ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Reilly" <peter.kitt.rei...@gmail.com>
Subject Re: PermGen OutOfMemoryError in custom Iterator task, and ForEach - Suggestions Please
Date Fri, 27 Oct 2006 08:39:41 GMT
On 10/26/06, Robert Rice <rapido@gmail.com> wrote:
> Peter,
>
> Your suggestions were quite helpful.  The fix for me was to utilize a
> post-ant1.7.0beta3 svn copy of ant.  I tested my Iterator-based task, as
> well as <foreach> using the most recent svn ant.  In both cases, my builds
> ran without OOME.  So it would seem that the memory leak squashing is paying
> off.  Hopefully these will make it into the proposed beta4.
>
> I do have a bonus, follow-up question.  Some of your dialog in the previous
> post leads me to believe I am behind in ant-contrib knowledge.  I am using
> ant-contrib-0.3, as made available through :
> http://ant-contrib.sourceforge.net/.  While I realize that this is a rather
> old build, I have found nothing newer.  In particular, your reference to
> "Use antcontrib:var@unset=true to reuse properties" leaves me a little
> puzzled as I have not seen this construct or a reference to a <var> task.
> Also, you had suggested to not use antcall.  The version of <foreach> that I
> have uses antcall internally, which leaves me further confused.
There is a newer version of ant-contrib (version 1.0b2 from 2005.02.03)
on:
http://sourceforge.net/project/showfiles.php?group_id=36177
This is feached from:
http://ant-contrib.sourceforge.net/
using the downloads hyper-link.
The content on the page (http://ant-contrib.sourceforge.net/) needs updating
as it refers to ant-contrib0.3

There is a new task called <for> which uses ant1.6 macrodef internally,
the <foreach> task is still present - for use with ant1.5 and for bc purposes.

If you like, you can upload the svn version of ant-contrib, it includes
some fixes and some new http tasks.

Peter

>
> Please point me in the correct direction, especially if there is
> documentation available.
>
> Thanks,
> Robert Rice
>
> On 10/24/06, Peter Reilly <peter.kitt.reilly@gmail.com> wrote:
> >
> > On 10/23/06, Robert Rice <robert@windermere.com> wrote:
> > > I recently created a custom task that is an Iterator.  The goal of the
> > task
> > > is to accept a large date range ( like 3 years ) and split it into
> > smaller
> > > date ranges ( like 1 day at a time ).  I do a lot of work with date
> > ranges.
> > > Examples are running queries or fetching files that expect a time range.
> > >
> > > At each iteration, the task sets a couple of properties that represent
> > the
> > > limits of the smaller date range.  At this point, the task acts very
> > much
> > > like <antcall>.  It requires one or more targets.  CallTaget tasks are
> > > dynamically constructed for each target for each iteration.  They are
> > then
> > > executed.   The targets that are called are rather large themselves,
> > > ultimately including about 10 targets or dependencies.
> > >
> > > The short is that I am running out of memory.  About 70 iterations in (
> > > representing 70 days of work that has been accomplished ), I get an
> > > OutOfMemoryError.  It looks something like:
> > >
> > > http-data:
> > >     [timer] Timer started.  Tasks started.
> > > Exception in thread "Thread-72" java.lang.OutOfMemoryError: PermGen
> > space
> > >     [timer] Timer interrupted. Tasks completed.
> > >
> > > And an example stacktrace:
> > > Exception in thread "Thread-71"
> > > /home/robert/dvl/java/bin/RetsDataCollector.xml:161: The following error
> > > occurred while executing this
> > > line:/home/robert/dvl/java/bin/RetsDataCollector.xml:127: Could not
> > create
> > > type mget due to java.lang.OutOfMemoryError: PermGen space        at
> > > org.apache.tools.ant.ProjectHelper.addLocationToBuildException(
> > > ProjectHelper.java:539)
> > >         at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:384)
> > >         at org.apache.tools.ant.taskdefs.CallTarget.execute(
> > CallTarget.java
> > > :107)
> > >         at net.sf.antcontrib.logic.ForEach.execute(Unknown Source)
> > >         at org.apache.tools.ant.UnknownElement.execute(
> > UnknownElement.java
> > > :275)
> > >         at org.apache.tools.ant.Task.perform(Task.java:364)
> > >         at com.windermere.collector.task.Timer$TasksThread.run(Unknown
> > > Source)
> > > Caused by: /home/robert/dvl/java/bin/RetsDataCollector.xml:127: Could
> > not
> > > create type mget due to java.lang.OutOfMemoryError: PermGen
> > space        at
> > > org.apache.tools.ant.AntTypeDefinition.createAndSet(
> > AntTypeDefinition.java
> > > :281)
> > >         at org.apache.tools.ant.AntTypeDefinition.icreate(
> > > AntTypeDefinition.java:196)
> > >         at org.apache.tools.ant.AntTypeDefinition.create(
> > > AntTypeDefinition.java:183)
> > >         at org.apache.tools.ant.ComponentHelper.createComponent(
> > > ComponentHelper.java:199)
> > >         at org.apache.tools.ant.ComponentHelper.createComponent(
> > > ComponentHelper.java:176)
> > >         at org.apache.tools.ant.UnknownElement.makeObject(
> > > UnknownElement.java:388)
> > >         at org.apache.tools.ant.UnknownElement.maybeConfigure(
> > > UnknownElement.java:158)
> > >         at org.apache.tools.ant.Task.perform(Task.java:363)
> > >         at org.apache.tools.ant.Target.execute(Target.java:341)
> > >         at org.apache.tools.ant.Target.performTasks(Target.java:369)
> > >         at org.apache.tools.ant.Project.executeSortedTargets(
> > Project.java
> > > :1216)
> > >         at
> > org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(
> > > SingleCheckExecutor.java:37)        at
> > > org.apache.tools.ant.Project.executeTargets(Project.java:1068)
> > >         at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:382)
> > >         ... 5 more
> > > Caused by: java.lang.OutOfMemoryError: PermGen space
> > >
> > > It looks to me like each iteration adds to the memory requirement, until
> > I
> > > run out of memory.  Ideally, at each iteration, the dynamically created
> > > <antcall> tasks would no longer be needed and would be garbage
> > collected.
> > > Is it because these <antcall> tasks are ultimately owned by the
> > containing
> > > iterator task?
> > >
> > > I'm looking for suggestions on how to get around this problem.  Can I
> > model
> > > my custom task in a different way internally to avoid this problem?
> >
> > The best way would be to not use antcall, but try to do every thing
> > in the same project.
> >
> > Use <macrodef> instread of targets
> > Use antcontrib:for as the looping construct
> > Use antcontrib:var@unset=true to reuse properties
> >
> > > As an aside, I thought I would see if the ant-contrib <foreach> task
had
> > the
> > > same end result.  I chose to do this since it is similar.  It instead
> > loops
> > > across a list, then constructing <antcall> tasks within each loop.  It
> > does
> > > indeed also run out of memory, at about the same point as my custom
> > task.
> > >
> > > I can provide code snippets of the custom task if that will help.  I'm
> > > guessing others have come across this problem and may have a solution
> > > without digging into the code.
> >
> > Please try to provide a complete working example using the
> > antcontrib:foreach.
> > I am currently squashing memory leakage bugs and am interrested
> > in simple examples that get OOME.
> >
> > Also try with a svn copy of ant, some memory leakages have
> > been fixed since ant1.7.0beta3.
> >
> > Peter
> >
> >
> >
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> > For additional commands, e-mail: dev-help@ant.apache.org
> >
> >
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org


Mime
View raw message