ibatis-user-java mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harvey Kim" <cash...@fastmail.fm>
Subject Re: groupby x out of memory
Date Fri, 11 Apr 2008 16:10:33 GMT
If you use groupBy, you will eventually run out of memory.  Few months
ago, I e-mailed a sample program which showed my application doing
absolute nothing except issuing a ibatis query.  It was using
queryWithRowHandler.  Literally, all it did was call
"queryWithRowHandler" and my rowHanlder did absolutely nothing. 
Literally this:

public class IbatisDownloadThread implements Runnable, Serializable,
  private SqlMapClient sqlMapClient = null;
  public void run()
        sqlMapClient = BaseDAO.getSqlMapClient();
        sqlMapClient.queryWithRowHandler("getIdList", queryDto, this);  

  public void handleRow()

The above code eventually gets "Out of Memory" error and the query is
using a groupBy.  When I took out the "groupBy", it went MUCH MUCH MUCH
longer.  But even that ran out of memory after few hours of running. 
However, once I converted everything to a JDBC loop, "Out of Memory"
error went away.  When I examined the available memory at runtime, I
could see the available memory decreasing with every call to
"handleRow()".  With JDBC, I examined the heap size inside the loop and
it held steady for hours and finished without a problem.

The moral of the story - you could probably get away with ibatis if you
take out the groupBy (that was my quick fix).  I took out the groupBy
and simply issued another ibatis command to get the list inside
"handleRow()".  That was my quick fix and it is probably sufficient for

You can still run out of memory if you use "queryWithRowHandler" but
highly unlikely unless you have an application like mine which could
have a thread running for days.  But to be really safe, you have to go
back to JDBC.  Since I was on a tight deadline, I didn't bother going
through the ibatis code but I may do that one of these days because it
is a wonderful tool.

On Tue, 8 Apr 2008 12:27:29 -0700 (PDT), "Carlos de Luna Saenz"
<cdelunasaenz@yahoo.com.mx> said:
> It seems like there is not enough data in your mail...
> what data base are you accessing? how are you dealing with it (DataSource
> is JDBC or via the App Server) your class is keeping the data for any
> reason (maybe in a session or application bean)? what's the App Server?
> etc, etc... if i were you i should look all that kind of variables and
> monitor with a profiler to see if there's somehint wrong with the query,
> the server, the managed memory, or all of the above.
> Greetings
> Carlos de Luna
> ----- Mensaje original ----
> De: Fernando Simonetti <fernando@cosipa.com.br>
> Para: "user-java@ibatis.apache.org" <user-java@ibatis.apache.org>
> Enviado: martes, 8 de abril, 2008 7:51:26
> Asunto: groupby x out of memory
> Eventually my aplication server is going down by "Out of Memory".
> Analizing
> de heap dumping, it's point for the same class allocating 600 Mb.
> This event can occur 3 hours or 3 days after I restart my server. This
> class
> is used thounsands times in the period. Monitoring the memory of the
> server,
> I could see 300MB to 700MB of memory been allocated in  3 minutes. It
> hapens
> eventualy but not in a determined period.
> The unique diference for this class for others, is the groupby clause in
> Ibatis. First I thouth it could be a cartesian product, but I ran the
> query
> without parameters and found nothing.
> I going to implement this class without groupby clause, but first I would
> like to see if there are other options.
> thanks,
> Fernando
>       ____________________________________________________________________________________
> ¡Capacidad ilimitada de almacenamiento en tu correo!
> No te preocupes más por el espacio de tu cuenta con Correo Yahoo!:        
> http://correo.espanol.yahoo.com/

http://www.fastmail.fm - Accessible with your email software
                          or over the web

View raw message