ibatis-user-java mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Clinton Begin" <clinton.be...@gmail.com>
Subject Re: groupby x out of memory
Date Fri, 11 Apr 2008 17:42:10 GMT
I can't imagine why that would happen.  If this was a bug, I would think far
more people would be experiencing it.  I have apps that have run for months,
and in some cases years, without running out of memory.  GroupBy doesn't do
anything special really.  It just skips duplicate rows for a parent class.

Try iBATIS 2.3.1.  If there is a problem, it might have to do with
session/transaction management.  2.3.1 improves the handling of such.

Clinton

On Fri, Apr 11, 2008 at 10:10 AM, Harvey Kim <cashtag@fastmail.fm> wrote:

> If you use groupBy, you will eventually run out of memory.  Few months
> ago, I e-mailed a sample program which showed my application doing
> absolute nothing except issuing a ibatis query.  It was using
> queryWithRowHandler.  Literally, all it did was call
> "queryWithRowHandler" and my rowHanlder did absolutely nothing.
> Literally this:
>
> public class IbatisDownloadThread implements Runnable, Serializable,
> RowHandler
> {
>  private SqlMapClient sqlMapClient = null;
>
>  public void run()
>  {
>      try
>      {
>        sqlMapClient = BaseDAO.getSqlMapClient();
>        sqlMapClient.queryWithRowHandler("getIdList", queryDto, this);
>     }
>  }
>
>  public void handleRow()
>  {
>  }
> }
>
> The above code eventually gets "Out of Memory" error and the query is
> using a groupBy.  When I took out the "groupBy", it went MUCH MUCH MUCH
> longer.  But even that ran out of memory after few hours of running.
> However, once I converted everything to a JDBC loop, "Out of Memory"
> error went away.  When I examined the available memory at runtime, I
> could see the available memory decreasing with every call to
> "handleRow()".  With JDBC, I examined the heap size inside the loop and
> it held steady for hours and finished without a problem.
>
> The moral of the story - you could probably get away with ibatis if you
> take out the groupBy (that was my quick fix).  I took out the groupBy
> and simply issued another ibatis command to get the list inside
> "handleRow()".  That was my quick fix and it is probably sufficient for
> now.
>
> You can still run out of memory if you use "queryWithRowHandler" but
> highly unlikely unless you have an application like mine which could
> have a thread running for days.  But to be really safe, you have to go
> back to JDBC.  Since I was on a tight deadline, I didn't bother going
> through the ibatis code but I may do that one of these days because it
> is a wonderful tool.
>
>
> On Tue, 8 Apr 2008 12:27:29 -0700 (PDT), "Carlos de Luna Saenz"
> <cdelunasaenz@yahoo.com.mx> said:
> > It seems like there is not enough data in your mail...
> > what data base are you accessing? how are you dealing with it
> (DataSource
> > is JDBC or via the App Server) your class is keeping the data for any
> > reason (maybe in a session or application bean)? what's the App Server?
> > etc, etc... if i were you i should look all that kind of variables and
> > monitor with a profiler to see if there's somehint wrong with the query,
> > the server, the managed memory, or all of the above.
> > Greetings
> > Carlos de Luna
> >
> > ----- Mensaje original ----
> > De: Fernando Simonetti <fernando@cosipa.com.br>
> > Para: "user-java@ibatis.apache.org" <user-java@ibatis.apache.org>
> > Enviado: martes, 8 de abril, 2008 7:51:26
> > Asunto: groupby x out of memory
> >
> > Eventually my aplication server is going down by "Out of Memory".
> > Analizing
> > de heap dumping, it's point for the same class allocating 600 Mb.
> >
> > This event can occur 3 hours or 3 days after I restart my server. This
> > class
> > is used thounsands times in the period. Monitoring the memory of the
> > server,
> > I could see 300MB to 700MB of memory been allocated in  3 minutes. It
> > hapens
> > eventualy but not in a determined period.
> >
> > The unique diference for this class for others, is the groupby clause in
> > Ibatis. First I thouth it could be a cartesian product, but I ran the
> > query
> > without parameters and found nothing.
> >
> > I going to implement this class without groupby clause, but first I
> would
> > like to see if there are other options.
> >
> >
> > thanks,
> > Fernando
> >
> >
> >
> ____________________________________________________________________________________
> > ¡Capacidad ilimitada de almacenamiento en tu correo!
> > No te preocupes más por el espacio de tu cuenta con Correo Yahoo!:
> > http://correo.espanol.yahoo.com/
> >
>
> --
> http://www.fastmail.fm - Accessible with your email software
>                          or over the web
>
>

Mime
View raw message