ibatis-user-java mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clinton Begin <clinton.be...@gmail.com>
Subject Re: ResultHandler - OutOfMemory Exception
Date Tue, 15 Sep 2009 22:34:39 GMT
Hmmm... I don't think so. I suppose it would make sense to add that !  :-)
Jira ticket!

On Tue, Sep 15, 2009 at 11:57 AM, K. Arnold <akarnold@comcast.net> wrote:

>
> Out of curiosity, is it possible to create my own DefaultSetResultHandler?
>  I
> wrote an plugin that would intercept the call, and I was going to walk
> through the result set myself.  Unfortunately I'm not sure how to get the
> parameters that are used in the constructor.  Looking at the code it seems
> to jump through some hoops passing the objects through.  Is there a context
> that contains the references to those that I could use?
>
>
>
> Clinton Begin wrote:
> >
> > Ah, then yes, iBATIS 3 might be more aggressive about using the nested
> > objects cache...(even without join mapping), I can't remember if iBATIS 2
> > would behave the same way.  Might be worth a try... if it works in 2,
> then
> > I'll revisit the design to see if I can get 3 to only cache if it detects
> > a
> > nested result map.
> > Clinton
> >
> > On Tue, Sep 15, 2009 at 10:44 AM, K. Arnold <akarnold@comcast.net>
> wrote:
> >
> >>
> >> I have not used iBATIS 2 to do a comparison.  It is very well that the
> >> assumption given to me about how RowHandler works is false.  I am in the
> >> process of trying to figure out a good limit size to pick utilizing an
> >> AS400
> >> jdbc driver working on a DB2 database.
> >>
> >> We have decided that utilizing the select statement with the limit is
> the
> >> better decision choice for us.  Now we need to figure out how to
> optimize
> >> it.
> >>
> >> Any suggestions would be appreciated :ninja:
> >>
> >> Just to make sure I understand your reply for any future work.  I don't
> >> believe I am using a join mapper.  I have a POJO that maps directly to a
> >> custom (yet extremely complicated) sql statement.  The POJO doesn't have
> >> any
> >> collections, and is basically stand alone.  It is my understanding from
> >> your
> >> code, that my type of POJO isn't going to utilize the join mapper
> method,
> >> since the next POJO will be unique.
> >>
> >> Clinton Begin wrote:
> >> >
> >> > The difference between RowHandler and ResultHandler is only in name.
> >>  They
> >> > did the same thing.
> >> > I'd be interested to know if iBATIS 2.x works for you in this regard.
> >> As
> >> > far
> >> > as I can recall, 2.x had the same design if you invoked the join
> >> mapper...
> >> >
> >> > See, as soon as you try to use a JOIN to load a complex object graph,
> >> we
> >> > need to cache the results so that iBATIS knows to append to the
> parent,
> >> > rather than instantiate a new object.
> >> >
> >> > Clinton
> >> >
> >> > On Tue, Sep 15, 2009 at 9:04 AM, K. Arnold <akarnold@comcast.net>
> >> wrote:
> >> >
> >> >>
> >> >> Thank you for the reply.  Is it possible to make the join mapping
> >> cache
> >> a
> >> >> configurable attribute on the mapping file?   This is my first time
> >> using
> >> >> Ibatis and I was confidently informed that we could walk row by row
> >> over
> >> >> a
> >> >> result set.  I was told to look up the RowHandler interface.  It is
> my
> >> >> understanding that in Ibatis 3 that the ResultHandler has replaced
> the
> >> >> RowHandler.  Was there a specific design decision made not to allow
a
> >> row
> >> >> by
> >> >> row walk through?
> >> >>
> >> >> As a work around I was looking into the Plugin feature.  It looks
> like
> >> I
> >> >> can
> >> >> Intercept the resultSetsHandler method, and write my own code to
> >> process
> >> >> it.
> >> >> In theory I could utilize the DefaultResultSetHandler code and remove
> >> the
> >> >> cache feature in my implementation.  However I am not sure if I have
> >> >> access
> >> >> to the same constructor parameters at the point of the intercept.
> >> >>
> >> >> There is some concern around the performance of also introducing
> >> multiple
> >> >> call backs to the Database for each "set" of data to process.  I need
> >> to
> >> >> balance the design choice of writing my own resultSetsHandler with
> the
> >> >> number of sql calls being made to the database via an offset Select
> >> call.
> >> >>
> >> >> I appreciate your replies to this subject.
> >> >>
> >> >> Clinton Begin wrote:
> >> >> >
> >> >> > The nestedResultObjects is necessary for join mapping.  One way
to
> >> deal
> >> >> > with
> >> >> > this though, is to use batches of reads as well as writes.  Use
the
> >> >> > pagination facilities and possibly even the proprietary
> offset/limit
> >> >> > features of your database to grab subsets of the results.
> >> >> > Incidentally I'm rewriting the DefaultResultSetHandler to be easier
> >> to
> >> >> > understand.  But I don't see the need for that cache going away
> >> anytime
> >> >> > soon...
> >> >> >
> >> >> > Clinton
> >> >> >
> >> >> > On Mon, Sep 14, 2009 at 1:47 PM, K. Arnold <akarnold@comcast.net>
> >> >> wrote:
> >> >> >
> >> >> >>
> >> >> >> I am trying to iterate over a result set of 2million records,
for
> a
> >> >> large
> >> >> >> bulk load and transformation into a new ODS.  It appears that
I am
> >> >> >> getting
> >> >> >> an OutOfMemoryException because the DefaultResultSetHandler
is
> >> caching
> >> >> >> the
> >> >> >> object in the nestedResultObjects property.  Is there some
> property
> >> I
> >> >> >> should
> >> >> >> set or statement/ method call I should be using that will
allow me
> >> to
> >> >> >> process one line at a time and not have the nestedResultObjects
> >> store
> >> >> >> each
> >> >> >> object?
> >> >> >>
> >> >> >> My goal:
> >> >> >> * Grab a row
> >> >> >> * Send it to be processed
> >> >> >> * Once processed, move on to the next row.
> >> >> >> Note: Once a row is processed I no longer need a tie back
to the
> >> >> object.
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> I have included the custom ResultHandler, the unit test and
the
> >> >> >> configuration file.  Please let me know if you need other
> >> information.
> >> >> >>
> >> >> >> package com.primetherapeutics.benplanmgr.entity.rxclaim;
> >> >> >>
> >> >> >> import org.apache.ibatis.executor.result.ResultContext;
> >> >> >> import org.apache.ibatis.executor.result.ResultHandler;
> >> >> >> import org.apache.log4j.Logger;
> >> >> >>
> >> >> >> /**
> >> >> >>  * @author kjarnold
> >> >> >>  *
> >> >> >>  */
> >> >> >> public class GroupEligibilityResultHandler implements
> ResultHandler
> >> {
> >> >> >>        Logger logger =
> >> >> >> Logger.getLogger(GroupEligibilityResultHandler.class);
> >> >> >>
> >> >> >>        int count = 0;
> >> >> >>
> >> >> >>        public void handleResult(ResultContext context) {
> >> >> >>                if(context.getResultObject() != null) {
> >> >> >>                        count++;
> >> >> >>                        logger.debug(count);
> >> >> >>                }
> >> >> >>                //context.stop();
> >> >> >>        }
> >> >> >>
> >> >> >>        public int getCount() {
> >> >> >>                return count;
> >> >> >>        }
> >> >> >>
> >> >> >> }
> >> >> >>
> >> >> >>        @Test
> >> >> >>        public void getGroupElibibilitiesByResultHandler()
{
> >> >> >>                Map<String, String> parameterMap = new
> >> HashMap<String,
> >> >> >> String>();
> >> >> >>                parameterMap.put("gelThruDate", "1090101");
> >> >> >>                parameterMap.put("addDate", "1090911");
> >> >> >>                parameterMap.put("chgDate", "1090911");
> >> >> >>                parameterMap.put("planDate", "1090101");
> >> >> >>                try {
> >> >> >>                        GroupEligibilityResultHandler handler
= new
> >> >> >> GroupEligibilityResultHandler();
> >> >> >>
> >> >> >>
> >> >> >>
> >> >>
> >>
> session.select("com.primetherapeutics.benplanmgr.entity.rxclaim.data.GroupEligibilityMapper.getGroupEligibilities",
> >> >> >>                                        parameterMap, handler);
> >> >> >>                        logger.debug(handler.getCount());
> >> >> >>
> >> >> >>                } finally {
> >> >> >>                        session.close();
> >> >> >>                }
> >> >> >>
> >> >> >>        }
> >> >> >>
> >> >> >>
> >> >> >> Here are my mapping files:
> >> >> >>
> >> >> >> <?xml version="1.0" encoding="UTF-8" ?>
> >> >> >> <!DOCTYPE configuration PUBLIC "-//ibatis.apache.org//DTD
Config
> >> >> 3.0//EN"
> >> >> >>        "http://ibatis.apache.org/dtd/ibatis-3-config.dtd">
> >> >> >> <configuration>
> >> >> >> <settings>
> >> >> >>        <setting name="multipleResultSetsEnabled" value="false"/>
> >> >> >>        <setting name="defaultExecutorType" value="BATCH"/>
> >> >> >> </settings>
> >> >> >>        <mappers>
> >> >> >>                <mapper
> >> >> >>
> >> >> >>
> >> >>
> >>
> resource="com/primetherapeutics/benplanmgr/entity/rxclaim/data/BenefitMaxSchedule.xml"/>
> >> >> >>                <mapper
> >> >> >>
> >> >> >>
> >> >>
> >>
> resource="com/primetherapeutics/benplanmgr/entity/rxclaim/data/GroupEligibility.xml"/>
> >> >> >>        </mappers>
> >> >> >> </configuration>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> View this message in context:
> >> >> >>
> >> >>
> >>
> http://www.nabble.com/ResultHandler---OutOfMemory-Exception-tp25442025p25442025.html
> >> >> >> Sent from the iBATIS - User - Java mailing list archive at
> >> Nabble.com.
> >> >> >>
> >> >> >>
> >> >> >>
> >> ---------------------------------------------------------------------
> >> >> >> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
> >> >> >> For additional commands, e-mail: user-java-help@ibatis.apache.org
> >> >> >>
> >> >> >>
> >> >> >
> >> >> >
> >> >>
> >> >> --
> >> >> View this message in context:
> >> >>
> >>
> http://www.nabble.com/ResultHandler---OutOfMemory-Exception-tp25442025p25455710.html
> >> >> Sent from the iBATIS - User - Java mailing list archive at
> Nabble.com.
> >> >>
> >> >>
> >> >> ---------------------------------------------------------------------
> >> >> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
> >> >> For additional commands, e-mail: user-java-help@ibatis.apache.org
> >> >>
> >> >>
> >> >
> >> >
> >>
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/ResultHandler---OutOfMemory-Exception-tp25442025p25457565.html
> >> Sent from the iBATIS - User - Java mailing list archive at Nabble.com.
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
> >> For additional commands, e-mail: user-java-help@ibatis.apache.org
> >>
> >>
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/ResultHandler---OutOfMemory-Exception-tp25442025p25458817.html
> Sent from the iBATIS - User - Java mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-java-unsubscribe@ibatis.apache.org
> For additional commands, e-mail: user-java-help@ibatis.apache.org
>
>

Mime
View raw message