ibatis-user-java mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clinton Begin <clinton.be...@gmail.com>
Subject Re: ResultHandler - OutOfMemory Exception
Date Tue, 15 Sep 2009 17:04:30 GMT
Ah, then yes, iBATIS 3 might be more aggressive about using the nested
objects cache...(even without join mapping), I can't remember if iBATIS 2
would behave the same way.  Might be worth a try... if it works in 2, then
I'll revisit the design to see if I can get 3 to only cache if it detects a
nested result map.
Clinton

On Tue, Sep 15, 2009 at 10:44 AM, K. Arnold <akarnold@comcast.net> wrote:

>
> I have not used iBATIS 2 to do a comparison.  It is very well that the
> assumption given to me about how RowHandler works is false.  I am in the
> process of trying to figure out a good limit size to pick utilizing an
> AS400
> jdbc driver working on a DB2 database.
>
> We have decided that utilizing the select statement with the limit is the
> better decision choice for us.  Now we need to figure out how to optimize
> it.
>
> Any suggestions would be appreciated :ninja:
>
> Just to make sure I understand your reply for any future work.  I don't
> believe I am using a join mapper.  I have a POJO that maps directly to a
> custom (yet extremely complicated) sql statement.  The POJO doesn't have
> any
> collections, and is basically stand alone.  It is my understanding from
> your
> code, that my type of POJO isn't going to utilize the join mapper method,
> since the next POJO will be unique.
>
> Clinton Begin wrote:
> >
> > The difference between RowHandler and ResultHandler is only in name.
>  They
> > did the same thing.
> > I'd be interested to know if iBATIS 2.x works for you in this regard. As
> > far
> > as I can recall, 2.x had the same design if you invoked the join
> mapper...
> >
> > See, as soon as you try to use a JOIN to load a complex object graph, we
> > need to cache the results so that iBATIS knows to append to the parent,
> > rather than instantiate a new object.
> >
> > Clinton
> >
> > On Tue, Sep 15, 2009 at 9:04 AM, K. Arnold <akarnold@comcast.net> wrote:
> >
> >>
> >> Thank you for the reply.  Is it possible to make the join mapping cache
> a
> >> configurable attribute on the mapping file?   This is my first time
> using
> >> Ibatis and I was confidently informed that we could walk row by row over
> >> a
> >> result set.  I was told to look up the RowHandler interface.  It is my
> >> understanding that in Ibatis 3 that the ResultHandler has replaced the
> >> RowHandler.  Was there a specific design decision made not to allow a
> row
> >> by
> >> row walk through?
> >>
> >> As a work around I was looking into the Plugin feature.  It looks like I
> >> can
> >> Intercept the resultSetsHandler method, and write my own code to process
> >> it.
> >> In theory I could utilize the DefaultResultSetHandler code and remove
> the
> >> cache feature in my implementation.  However I am not sure if I have
> >> access
> >> to the same constructor parameters at the point of the intercept.
> >>
> >> There is some concern around the performance of also introducing
> multiple
> >> call backs to the Database for each "set" of data to process.  I need to
> >> balance the design choice of writing my own resultSetsHandler with the
> >> number of sql calls being made to the database via an offset Select
> call.
> >>
> >> I appreciate your replies to this subject.
> >>
> >> Clinton Begin wrote:
> >> >
> >> > The nestedResultObjects is necessary for join mapping.  One way to
> deal
> >> > with
> >> > this though, is to use batches of reads as well as writes.  Use the
> >> > pagination facilities and possibly even the proprietary offset/limit
> >> > features of your database to grab subsets of the results.
> >> > Incidentally I'm rewriting the DefaultResultSetHandler to be easier to
> >> > understand.  But I don't see the need for that cache going away
> anytime
> >> > soon...
> >> >
> >> > Clinton
> >> >
> >> > On Mon, Sep 14, 2009 at 1:47 PM, K. Arnold <akarnold@comcast.net>
> >> wrote:
> >> >
> >> >>
> >> >> I am trying to iterate over a result set of 2million records, for a
> >> large
> >> >> bulk load and transformation into a new ODS.  It appears that I am
> >> >> getting
> >> >> an OutOfMemoryException because the DefaultResultSetHandler is
> caching
> >> >> the
> >> >> object in the nestedResultObjects property.  Is there some property
I
> >> >> should
> >> >> set or statement/ method call I should be using that will allow me
to
> >> >> process one line at a time and not have the nestedResultObjects store
> >> >> each
> >> >> object?
> >> >>
> >> >> My goal:
> >> >> * Grab a row
> >> >> * Send it to be processed
> >> >> * Once processed, move on to the next row.
> >> >> Note: Once a row is processed I no longer need a tie back to the
> >> object.
> >> >>
> >> >>
> >> >>
> >> >> I have included the custom ResultHandler, the unit test and the
> >> >> configuration file.  Please let me know if you need other
> information.
> >> >>
> >> >> package com.primetherapeutics.benplanmgr.entity.rxclaim;
> >> >>
> >> >> import org.apache.ibatis.executor.result.ResultContext;
> >> >> import org.apache.ibatis.executor.result.ResultHandler;
> >> >> import org.apache.log4j.Logger;
> >> >>
> >> >> /**
> >> >>  * @author kjarnold
> >> >>  *
> >> >>  */
> >> >> public class GroupEligibilityResultHandler implements ResultHandler
{
> >> >>        Logger logger =
> >> >> Logger.getLogger(GroupEligibilityResultHandler.class);
> >> >>
> >> >>        int count = 0;
> >> >>
> >> >>        public void handleResult(ResultContext context) {
> >> >>                if(context.getResultObject() != null) {
> >> >>                        count++;
> >> >>                        logger.debug(count);
> >> >>                }
> >> >>                //context.stop();
> >> >>        }
> >> >>
> >> >>        public int getCount() {
> >> >>                return count;
> >> >>        }
> >> >>
> >> >> }
> >> >>
> >> >>        @Test
> >> >>        public void getGroupElibibilitiesByResultHandler() {
> >> >>                Map<String, String> parameterMap = new HashMap<String,
> >> >> String>();
> >> >>                parameterMap.put("gelThruDate", "1090101");
> >> >>                parameterMap.put("addDate", "1090911");
> >> >>                parameterMap.put("chgDate", "1090911");
> >> >>                parameterMap.put("planDate", "1090101");
> >> >>                try {
> >> >>                        GroupEligibilityResultHandler handler = new
> >> >> GroupEligibilityResultHandler();
> >> >>
> >> >>
> >> >>
> >>
> session.select("com.primetherapeutics.benplanmgr.entity.rxclaim.data.GroupEligibilityMapper.getGroupEligibilities",
> >> >>                                        parameterMap, handler);
> >> >>                        logger.debug(handler.getCount());
> >> >>
> >> >>                } finally {
> >> >>                        session.close();
> >> >>                }
> >> >>
> >> >>        }
> >> >>
> >> >>
> >> >> Here are my mapping files:
> >> >>
> >> >> <?xml version="1.0" encoding="UTF-8" ?>
> >> >> <!DOCTYPE configuration PUBLIC "-//ibatis.apache.org//DTD Config
> >> 3.0//EN"
> >> >>        "http://ibatis.apache.org/dtd/ibatis-3-config.dtd">
> >> >> <configuration>
> >> >> <settings>
> >> >>        <setting name="multipleResultSetsEnabled" value="false"/>
> >> >>        <setting name="defaultExecutorType" value="BATCH"/>
> >> >> </settings>
> >> >>        <mappers>
> >> >>                <mapper
> >> >>
> >> >>
> >>
> resource="com/primetherapeutics/benplanmgr/entity/rxclaim/data/BenefitMaxSchedule.xml"/>
> >> >>                <mapper
> >> >>
> >> >>
> >>
> resource="com/primetherapeutics/benplanmgr/entity/rxclaim/data/GroupEligibility.xml"/>
> >> >>        </mappers>
> >> >> </configuration>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> View this message in context:
> >> >>
> >>
> http://www.nabble.com/ResultHandler---OutOfMemory-Exception-tp25442025p25442025.html
> >> >> Sent from the iBATIS - User - Java mailing list archive at
> Nabble.com.
> >> >>
> >> >>
> >> >> ---------------------------------------------------------------------
> >> >> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
> >> >> For additional commands, e-mail: user-java-help@ibatis.apache.org
> >> >>
> >> >>
> >> >
> >> >
> >>
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/ResultHandler---OutOfMemory-Exception-tp25442025p25455710.html
> >> Sent from the iBATIS - User - Java mailing list archive at Nabble.com.
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
> >> For additional commands, e-mail: user-java-help@ibatis.apache.org
> >>
> >>
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/ResultHandler---OutOfMemory-Exception-tp25442025p25457565.html
> Sent from the iBATIS - User - Java mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
> For additional commands, e-mail: user-java-help@ibatis.apache.org
>
>

Mime
View raw message