Return-Path: Delivered-To: apmail-ibatis-user-java-archive@www.apache.org Received: (qmail 76966 invoked from network); 16 Mar 2008 21:07:12 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 16 Mar 2008 21:07:12 -0000 Received: (qmail 27600 invoked by uid 500); 16 Mar 2008 21:07:07 -0000 Delivered-To: apmail-ibatis-user-java-archive@ibatis.apache.org Received: (qmail 27587 invoked by uid 500); 16 Mar 2008 21:07:07 -0000 Mailing-List: contact user-java-help@ibatis.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user-java@ibatis.apache.org Delivered-To: mailing list user-java@ibatis.apache.org Received: (qmail 27576 invoked by uid 99); 16 Mar 2008 21:07:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 16 Mar 2008 14:07:07 -0700 X-ASF-Spam-Status: No, hits=2.0 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [68.142.207.198] (HELO web32405.mail.mud.yahoo.com) (68.142.207.198) by apache.org (qpsmtpd/0.29) with SMTP; Sun, 16 Mar 2008 21:06:15 +0000 Received: (qmail 45517 invoked by uid 60001); 16 Mar 2008 21:06:34 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Message-ID; b=sslj5f8uA5FLMwAoEfp8IlfcVDjvsQayKNGbOMbBvCyVXcQN/2TySgw1dtqBS8Z25ijsQwikDMoG47wCnFBLkIQIP8qCP4xtoTt4PIzSAt5qVDrsfvY1jtZ6qzdtBakU5acjZeak3ma+iss7Fppb2cQS7fV+fOcdeSvsU+q1j9Y=; X-YMail-OSG: 4LP4kvgVM1kEv11PRjl71Ix1o.JP.CCXg2SK2P_TSi7Y01QsaYsRPhmRRbPvixvmLTiT1nEENifdKxE_X0u9XNx0llgG1KhOB02ZJ1jNlUOoFpwm.ek- Received: from [85.57.72.125] by web32405.mail.mud.yahoo.com via HTTP; Sun, 16 Mar 2008 14:06:34 PDT X-Mailer: YahooMailRC/902.35 YahooMailWebService/0.7.162 Date: Sun, 16 Mar 2008 14:06:34 -0700 (PDT) From: nch Subject: Re: select * causing " OutOfMemoryError: Java heap space" To: user-java@ibatis.apache.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-2041359295-1205701594=:45308" Message-ID: <86291.45308.qm@web32405.mail.mud.yahoo.com> X-Virus-Checked: Checked by ClamAV on apache.org --0-2041359295-1205701594=:45308 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable =0AYou're probably right. I'm using the table called "text" which contains = a field of type "mediumblob".=0ACREATE TABLE /*$wgDBprefix*/text (=0A old_= id int unsigned NOT NULL auto_increment,=0A old_text mediumblob NOT NULL,= =0A old_flags tinyblob NOT NULL,=0A PRIMARY KEY old_id (old_id)=0A) /*$wg= DBTableOptions*/ MAX_ROWS=3D10000000 AVG_ROW_LENGTH=3D10240;=0AYou can find= it's definition here:=0Ahttp://svn.wikimedia.org/viewvc/mediawiki/trunk/ph= ase3/maintenance/tables.sql?view=3Dmarkup=0A=0AI'll try to modify that fiel= d, first, and see what happens.=0AI've already tried "select * from your_wi= kipedia_dump limit $start$, $size$" and it seems to work fine, but I see a = possible issue with this, you can not remove any of the records you've alre= ady read during the whole process. This might not be an option.=0A=0AThank = you.=0A=0A----- Original Message ----=0AFrom: Larry Meadors =0ATo: user-java@ibatis.apache.org=0ASent: Sunday, March 16, 2008= 4:19:43 PM=0ASubject: Re: select * causing " OutOfMemoryError: Java heap s= pace"=0A=0AYeah, I just noticed that - it looks like it's failing in the jd= bc=0Adriver when it tries to read a blob, maybe?=0A=0AI know that some vers= ions of the postgresql jdbc driver load the=0Aentire contents of a resultse= t into a massive byte[][] structure. I=0Awonder if that is what is happenin= g here? I think you can get the=0Asource for the MySQL driver and check tha= t if you want to.=0A=0AIf it is, you may need to structure your app to take= smaller bites of=0Athis elephant.=0A=0ATry "select * from your_wikipedia_d= ump limit $start$, $size$" instead=0A- that will limit the size of the resu= lts to $size$ rows, starting on=0Arow $start$ (the $start$ value is zero-ba= sed). If you experiment to=0Asee how big $size$ can get, you should be able= to get pretty decent=0Aperformance out of this.=0A=0APsuedo-code would be = like this...=0A=0Aint start =3D 0;=0Aint size =3D 1000; // see how big you = can make this to improve performance=0Awhile(still_more_data){=0A still_mo= re_data =3D index(start, size);=0A start +=3D size;=0A}=0A=0AYour index me= thod will return true if the query returned any data. If=0Athe query return= s no data, it'll return false.=0A=0AI can't imagine what Hibernate is doing= differently here to make this=0Awork if the JDBC driver is failing when ex= ecuting the query.=0A=0ALarry=0A=0A=0AOn Sun, Mar 16, 2008 at 4:12 AM, nch = wrote:=0A>=0A>=0A> Sure. Please, see attached.= =0A> I don't think the problem is in the RowHandler, though, because the=0A= > OutOfMemoryError occurs before invoking RowHandle#handleRow.=0A>=0A> Chee= rs=0A>=0A>=0A> ----- Original Message ----=0A> From: Larry Meadors =0A> To: user-java@ibatis.apache.org=0A>=0A> Sent: Sunday,= March 16, 2008 3:13:27 AM=0A> Subject: Re: select * causing " OutOfMemoryE= rror: Java heap space"=0A>=0A> Can you post the row handler you are using.= =0A>=0A> Larry=0A>=0A>=0A> On Sat, Mar 15, 2008 at 1:14 PM, nch wrote:=0A> >=0A> >=0A> > Hi, Nathan.=0A> > I did so, but I'= m still having the same issue. Perhaps I'm not using it=0A> > correctly?=0A= > > See my last post to this forum entry:=0A> >=0A> > http://forum.compass-= project.org/thread.jspa?threadID=3D215278=0A> >=0A> > See the stack trace:= =0A> >=0A> > MemoryError: Java heap space:=0A> > java.lang.OutOfMemoryError= : Java heap space=0A> > at com.mysql.jdbc.Buffer.getBytes(Buffer.jav= a:198)=0A> > at com.mysql.jdbc.Buffer.readLenByteArray(Buffer.java:3= 18)=0A> > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1366)=0A> >= at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2333)=0A> >= at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:435)=0A> > = at=0A> > com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java= :2040)=0A> > at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1= 443)=0A> > at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:177= 7)=0A> > at com.mysql.jdbc.Connection.execSQL(Connection.java:3249)= =0A> > at=0A> >=0A> com.mysql.jdbc.PreparedStatement.executeInternal= (PreparedStatement.java:1268)=0A> > at=0A> > com.mysql.jdbc.Prepared= Statement.execute(PreparedStatement.java:843)=0A> > at=0A> >=0A> org= .apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPrepared= Statement.java:169)=0A> > at=0A> >=0A> org.apache.commons.dbcp.Deleg= atingPreparedStatement.execute(DelegatingPreparedStatement.java:169)=0A> > = at=0A> >=0A> com.ibatis.sqlmap.engine.execution.SqlExecutor.executeQ= uery(SqlExecutor.java:186)=0A> > at=0A> >=0A> com.ibatis.sqlmap.engi= ne.mapping.statement.GeneralStatement.sqlExecuteQuery(GeneralStatement.java= :205)=0A> > at=0A> >=0A> com.ibatis.sqlmap.engine.mapping.statement.= GeneralStatement.executeQueryWithCallback(GeneralStatement.java:173)=0A> > = at=0A> >=0A> com.ibatis.sqlmap.engine.mapping.statement.GeneralState= ment.executeQueryWithRowHandler(GeneralStatement.java:133)=0A> > at= =0A> >=0A> com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryWithRo= wHandler(SqlMapExecutorDelegate.java:649)=0A> > at=0A> >=0A> com.iba= tis.sqlmap.engine.impl.SqlMapSessionImpl.queryWithRowHandler(SqlMapSessionI= mpl.java:156)=0A> > at=0A> >=0A> org.myorg.compass.SqlMapClientGpsDe= viceWithRowHandler.doIndex(SqlMapClientGpsDeviceWithRowHandler.java:33)=0A>= > at=0A> >=0A> org.compass.gps.device.AbstractGpsDevice$1.doInCompa= ssWithoutResult(AbstractGpsDevice.java:93)=0A> > at=0A> >=0A> org.co= mpass.core.CompassCallbackWithoutResult.doInCompass(CompassCallbackWithoutR= esult.java:29)=0A> > at=0A> > org.compass.core.CompassTemplate.execu= te(CompassTemplate.java:132)=0A> > at=0A> >=0A> org.compass.gps.impl= .SingleCompassGps.executeForIndex(SingleCompassGps.java:161)=0A> > a= t=0A> > org.compass.gps.device.AbstractGpsDevice.index(AbstractGpsDevice.ja= va:91)=0A> > at=0A> >=0A> org.compass.spring.device.SpringSyncTransa= ctionGpsDeviceWrapper$1.doInTransactionWithoutResult(SpringSyncTransactionG= psDeviceWrapper.java:98)=0A> > at=0A> >=0A> org.springframework.tran= saction.support.TransactionCallbackWithoutResult.doInTransaction(Transactio= nCallbackWithoutResult.java:33)=0A> > at=0A> >=0A> org.springframewo= rk.transaction.support.TransactionTemplate.execute(TransactionTemplate.java= :128)=0A> > at=0A> >=0A> org.compass.spring.device.SpringSyncTransac= tionGpsDeviceWrapper.index(SpringSyncTransactionGpsDeviceWrapper.java:96)= =0A> > at=0A> >=0A> org.compass.gps.impl.SingleCompassGps$1.buildInd= exIfNeeded(SingleCompassGps.java:133)=0A> > at=0A> >=0A> org.compass= .core.lucene.engine.manager.DefaultLuceneSearchEngineIndexManager$8.firstSt= ep(DefaultLuceneSearchEngineIndexManager.java:233)=0A> > at=0A> >=0A= > org.compass.core.lucene.engine.manager.DefaultLuceneSearchEngineIndexMana= ger.doOperate(DefaultLuceneSearchEngineIndexManager.java:182)=0A> >=0A> >= =0A> >=0A> >=0A> >=0A> > ----- Original Message ----=0A> > From: Nathan Mav= es =0A> > To: user-java@ibatis.apache.org=0A> >=0A>= > Sent: Saturday, March 15, 2008 5:43:04 PM=0A> > Subject: Re: select * ca= using " OutOfMemoryError: Java heap space"=0A> >=0A> > this is only my 2 c= ents but I would throw that class out. Write your own=0A> > implementation= which uses a row handler. This is the type of situation in=0A> > which a = row handler could really help. almost no memory would be used.=0A> >=0A> >= =0A> >=0A> > On Thu, Mar 13, 2008 at 4:57 PM, nch wrote:=0A> > >=0A> > > Yes, I agree that dividing the query into several = queries should do as=0A> > long as none of them uses more memory than avail= able. Gracias, Carlos.=0A> > >=0A> > > Nevertheless, I looked into=0A> > or= g.compass.gps.device.ibatis.SqlMapClientGpsDevice#doIndex and,=0A> > fundam= entally, what it does is a queryForPaginatedList and iterates=0A> through= =0A> > the resulting PaginatedList with nextPage()=0A> > >=0A> > > I guess = that the reason why pages are kept in memory after performing=0A> > paginat= edList.nextPage() is because a paginatedList.previousPage() can be=0A> > is= sued but, should't it free pages if heap space is running out?=0A> > >=0A> = > >=0A> > > Carlos de Luna Saenz wrote:=0A> > >= =0A> > > Since we have a similar trouble around here i must say that you ha= ve lot=0A> > of options when going to compass..=0A> > > The first one is to= index "object by object" by hand, the second one is=0A> to=0A> > use the G= ps wich will use the complete bunch of data and make subindexes=0A> > (maki= ng parts of your "main index" with diferent "maps" for each subindex=0A> in= =0A> > your gpsDriver.=0A> > > if your indexing is as strong as ours i shou= ld look forward for numer 1=0A> or=0A> > number 3 depending of your needs. = (More a Compass than iBatis issue)=0A> > > Greetings=0A> > > Carlos de Luna= =0A> > >=0A> > > ----- Mensaje original ----=0A> > > De: nch=0A> > >=0A> > = > Para: user-java@ibatis.apache.org; larry.meadors@gmail.com=0A> > > Enviad= o: jueves, 13 de marzo, 2008 9:37:04=0A> > > Asunto: Re: select * causing "= OutOfMemoryError: Java heap space"=0A> > >=0A> > >=0A> > > Sorry. Yes, I n= eed all the data at one time, because the Compass API=0A> seems=0A> > to be= meant in that way.=0A> > >=0A> > > Larry Meadors wrote: OK, so the answer = to the question is...[ yes, i=0A> need=0A> > all the data at=0A> > >=0A> > = >=0A> > >=0A> > > one time | no, i do not need all of the data at one time = ].=0A> > >=0A> > > Larry=0A> > >=0A> > >=0A> > > On Thu, Mar 13, 2008 at 9:= 04 AM, nch wrote:=0A> > > >=0A> > > > Hi!=0A> > > >=0A> > > > I'm doing tes= ts both under tomcat and jetty. I set 1024MB of heap=0A> space=0A> > in=0A>= > > > both cases. I am using a profiler to look into what's happening duri= ng=0A> > > > execution of the indexing process and I can tell that's the ex= act=0A> amount=0A> > it=0A> > > > displays for the VM.=0A> > > >=0A> > > > = As I describe in the Compass user forum (see link bellow), I'm=0A> indexing= =0A> > a=0A> > > > Wikipedia dump of about 650000-675000 records. That's so= rt of 1-1.5GB=0A> of=0A> > > > data.=0A> > > >=0A> > > > Hibernate can hand= le all that data, I guess it's using some sort of=0A> > > > pagination. iBa= tis seems to divide the select statement into several=0A> > select=0A> > > = > statements, but it also seems each of such statements surpasses the=0A> >= limit of=0A> > > > 1024MB when trying to index only 300000 of the database= records.=0A> > > >=0A> > > > Thank you=0A> > > >=0A> > > >=0A> > > >=0A> >= > > Chris Lamey wrote:=0A> > > > What is you JVM heap size set to and how = much data is returned by=0A> select=0A> > *=0A> > > > from table?=0A> > > >= =0A> > > > If you're trying to pull back 1G worth of data into a JVM with a= heap=0A> > size=0A> > > > set to 64M, you will hit the heap limit pretty q= uick.=0A> > > >=0A> > > >=0A> > > > -----Original Message-----=0A> > > > Fr= om: nch [mailto:underscore_dot@yahoo..com]=0A> > > > Sent: Thu 3/13/2008 8:= 03 AM=0A> > > > To: user-java@ibatis.apache.org=0A> > > > Subject: select *= causing " OutOfMemoryError: Java heap space"=0A> > > >=0A> > > >=0A> > > >= Hi, everybody.=0A> > > > I'm testing the Compass search engine and, in par= ticular, how to index=0A> a=0A> > big=0A> > > > set of documents from a tab= le in a MySQL database. In order to do this=0A> I=0A> > > > issue a "select= * from table_name" using iBatis, but this seems to be=0A> > > > causing th= e application to use all heap space available.=0A> > > > I added a new entr= y into Compass user forum with details:=0A> > > > http://forum.compass-proj= ect.org/thread.jspa?threadID=3D215278=0A> > > >=0A> > > > Can you figure ou= t why is this happening?=0A> > > >=0A> > > > Many thanks=0A> > > >=0A> > > = >=0A> > > > ---------------------------------=0A> > > > Be a better friend,= newshound, and know-it-all with Yahoo! Mobile. Try=0A> > it=0A> > > > now.= =0A> > > >=0A> > > >=0A> > > >=0A> > > >=0A> > > > ________________________= ________=0A> > > > Never miss a thing. Make Yahoo your homepage.=0A> > >=0A= > > >=0A> > > Never miss a thing. Make Yahoo your homepage.=0A> > >=0A> > >= =0A> > >=0A> > >=0A> > >=0A> > >=0A> > >=0A> >=0A> ________________________= ____________________________________________________________=0A> > > =A1Cap= acidad ilimitada de almacenamiento en tu correo!=0A> > > No te preocupes m= =E1s por el espacio de tu cuenta con Correo Yahoo!:=0A> > > http://correo.e= spanol.yahoo.com/=0A> > >=0A> > >=0A> > >=0A> > >=0A> > >=0A> > >=0A> > >= =0A> > > ________________________________=0A> > Never miss a thing. Make Ya= hoo your homepage.=0A> >=0A> >=0A> >=0A> >=0A> > _________________________= _______=0A> > Never miss a thing. Make Yahoo your homepage.=0A>=0A>=0A> __= ______________________________=0A>=0A> Be a better friend, newshound, and k= now-it-all with Yahoo! Mobile. Try it=0A> now.=0A=0A=0A=0A=0A=0A=0A __= ___________________________________________________________________________= _______=0ANever miss a thing. Make Yahoo your home page. =0Ahttp://www.yah= oo.com/r/hs --0-2041359295-1205701594=:45308 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable

You're probably right. I'm using the table call= ed "text" which contains a field of type "mediumblob".
CREATE TABLE=
 /*$wgDBprefix*/text (
  old_id int unsigned NOT NULL auto_increment,  old_text mediumblob NOT NULL,
  old_flags tinyblob NOT NULL,
  PRI=
MARY KEY old_id (old_id)
) /*$wgDBTableOptions*/ MAX_ROWS=3D10000000 AVG=
_ROW_LENGTH=3D10240;
=0AYou can find it's definition here:
http://svn.wikimedia.org/v= iewvc/mediawiki/trunk/phase3/maintenance/tables.sql?view=3Dmarkup

I'll try to modify that field, first, and see what happens.
I'v= e already tried "select =0A* =0Afrom =0Ayour_wikipedia_dump =0Alimit =0A$st= art$, =0A$size$" and it seems to work fine, but I see a possible issue with= this, you can not remove any of the records you've already read during the= whole process. This might not be an option.

Thank you.

----- Original Message ----
From: Larry Meadors <larry.meadors@gma= il.com>
To: user-java@ibatis.apache.org
Sent: Sunday, March 16, 20= 08 4:19:43 PM
Subject: Re: select * causing " OutOfMemoryError: Java hea= p space"

=0AYeah, =0AI =0Ajust =0Anoticed =0Athat =0A- =0Ait =0Alook= s =0Alike =0Ait's =0Afailing =0Ain =0Athe =0Ajdbc
driver =0Awhen =0Ait = =0Atries =0Ato =0Aread =0Aa =0Ablob, =0Amaybe?

I =0Aknow =0Athat =0A= some =0Aversions =0Aof =0Athe =0Apostgresql =0Ajdbc =0Adriver =0Aload =0Ath= e
entire =0Acontents =0Aof =0Aa =0Aresultset =0Ainto =0Aa =0Amassive =0A= byte[][] =0Astructure. =0AI
wonder =0Aif =0Athat =0Ais =0Awhat =0Ais =0A= happening =0Ahere? =0AI =0Athink =0Ayou =0Acan =0Aget =0Athe
source =0Af= or =0Athe =0AMySQL =0Adriver =0Aand =0Acheck =0Athat =0Aif =0Ayou =0Awant = =0Ato.

If =0Ait =0Ais, =0Ayou =0Amay =0Aneed =0Ato =0Astructure =0Ay= our =0Aapp =0Ato =0Atake =0Asmaller =0Abites =0Aof
this =0Aelephant.
=
Try =0A"select =0A* =0Afrom =0Ayour_wikipedia_dump =0Alimit =0A$start$,= =0A$size$" =0Ainstead
- =0Athat =0Awill =0Alimit =0Athe =0Asize =0Aof = =0Athe =0Aresults =0Ato =0A$size$ =0Arows, =0Astarting =0Aon
row =0A$sta= rt$ =0A(the =0A$start$ =0Avalue =0Ais =0Azero-based). =0AIf =0Ayou =0Aexper= iment =0Ato
see =0Ahow =0Abig =0A$size$ =0Acan =0Aget, =0Ayou =0Ashould = =0Abe =0Aable =0Ato =0Aget =0Apretty =0Adecent
performance =0Aout =0Aof = =0Athis.

Psuedo-code =0Awould =0Abe =0Alike =0Athis...

int = =0Astart =0A=3D =0A0;
int =0Asize =0A=3D =0A1000; =0A// =0Asee =0Ahow = =0Abig =0Ayou =0Acan =0Amake =0Athis =0Ato =0Aimprove =0Aperformance
whi= le(still_more_data){
  =0Astill_more_data =0A=3D =0Aindex(start, = =0Asize);
  =0Astart =0A+=3D =0Asize;
}

Your =0Aindex =0A= method =0Awill =0Areturn =0Atrue =0Aif =0Athe =0Aquery =0Areturned =0Aany = =0Adata. =0AIf
the =0Aquery =0Areturns =0Ano =0Adata, =0Ait'll =0Areturn= =0Afalse.

I =0Acan't =0Aimagine =0Awhat =0AHibernate =0Ais =0Adoing= =0Adifferently =0Ahere =0Ato =0Amake =0Athis
work =0Aif =0Athe =0AJDBC = =0Adriver =0Ais =0Afailing =0Awhen =0Aexecuting =0Athe =0Aquery.

Lar= ry


On =0ASun, =0AMar =0A16, =0A2008 =0Aat =0A4:12 =0AAM, =0Anch = =0A<underscore_dot@yahoo.com> =0Awrote:
>
&= gt;
> =0ASure. =0APlease, =0Asee =0Aattached.
> =0AI =0Adon't = =0Athink =0Athe =0Aproblem =0Ais =0Ain =0Athe =0ARowHandler, =0Athough, =0A= because =0Athe
> =0AOutOfMemoryError =0Aoccurs =0Abefore =0Ainvoking = =0ARowHandle#handleRow.
>
> =0ACheers
>
>
> = =0A----- =0AOriginal =0AMessage =0A----
> =0AFrom: =0ALarry =0AMeador= s =0A<larry.meadors@gmail.com>
> =0ATo: =0Auser-java@ibatis.apache.org
>
> =0ASent: =0AS= unday, =0AMarch =0A16, =0A2008 =0A3:13:27 =0AAM
> =0ASubject: =0ARe: = =0Aselect =0A* =0Acausing =0A" =0AOutOfMemoryError: =0AJava =0Aheap =0Aspac= e"
>
>  =0ACan =0Ayou =0Apost =0Athe =0Arow =0Ahandler =0A= you =0Aare =0Ausing.
>
> =0ALarry
>
>
> =0AOn= =0ASat, =0AMar =0A15, =0A2008 =0Aat =0A1:14 =0APM, =0Anch =0A<underscore_dot@yahoo.com> =0Awrote:
> =0A>
> =0A&= gt;
> =0A> =0AHi, =0ANathan.
> =0A> =0AI =0Adid =0Aso, = =0Abut =0AI'm =0Astill =0Ahaving =0Athe =0Asame =0Aissue. =0APerhaps =0AI'm= =0Anot =0Ausing =0Ait
> =0A> =0Acorrectly?
> =0A> =0ASee= =0Amy =0Alast =0Apost =0Ato =0Athis =0Aforum =0Aentry:
> =0A>
= > =0A> =0Ahttp://forum.compass-project.org/thread.j= spa?threadID=3D215278
> =0A>
> =0A> =0ASee =0Athe =0A= stack =0Atrace:
> =0A>
> =0A> =0AMemoryError: =0AJava =0A= heap =0Aspace:
> =0A> =0Ajava.lang.OutOfMemoryError: =0AJava =0Ahe= ap =0Aspace
> =0A>  =0A  =0A  =0A  =0Aat =0Aco= m.mysql.jdbc.Buffer.getBytes(Buffer.java:198)
> =0A>  =0A&nbs= p; =0A  =0A  =0Aat =0Acom.mysql.jdbc.Buffer.readLenByteArray(Buff= er.java:318)
> =0A>  =0A  =0A  =0A  =0Aat =0Ac= om.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1366)
> =0A>  =0A&= nbsp; =0A  =0A  =0Aat =0Acom.mysql.jdbc.MysqlIO.readSingleRowSet(= MysqlIO.java:2333)
> =0A>  =0A  =0A  =0A  =0Aa= t =0Acom.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:435)
> =0A>&= nbsp; =0A  =0A  =0A  =0Aat
> =0A> =0Acom.mysql.jdbc= .MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:2040)
> =0A>&nbs= p; =0A  =0A  =0A  =0Aat =0Acom.mysql.jdbc.MysqlIO.readAllRes= ults(MysqlIO.java:1443)
> =0A>  =0A  =0A  =0A = =0Aat =0Acom.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1777)
> = =0A>  =0A  =0A  =0A  =0Aat =0Acom.mysql.jdbc.Connect= ion.execSQL(Connection.java:3249)
> =0A>  =0A  =0A = =0A  =0Aat
> =0A>
> =0Acom.mysql.jdbc.PreparedStatemen= t.executeInternal(PreparedStatement.java:1268)
> =0A>  =0A&nb= sp; =0A  =0A  =0Aat
> =0A> =0Acom.mysql.jdbc.PreparedSta= tement.execute(PreparedStatement.java:843)
> =0A>  =0A  = =0A  =0A  =0Aat
> =0A>
> =0Aorg.apache.commons.dbc= p.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:169)=
> =0A>  =0A  =0A  =0A  =0Aat
> =0A><= br>> =0Aorg.apache.commons.dbcp.DelegatingPreparedStatement.execute(Dele= gatingPreparedStatement.java:169)
> =0A>  =0A  =0A = =0A  =0Aat
> =0A>
> =0Acom.ibatis.sqlmap.engine.execut= ion.SqlExecutor.executeQuery(SqlExecutor.java:186)
> =0A>  = =0A  =0A  =0A  =0Aat
> =0A>
> =0Acom.ibatis.s= qlmap.engine.mapping.statement.GeneralStatement.sqlExecuteQuery(GeneralStat= ement.java:205)
> =0A>  =0A  =0A  =0A  =0Aat> =0A>
> =0Acom.ibatis.sqlmap.engine.mapping.statement.Genera= lStatement.executeQueryWithCallback(GeneralStatement.java:173)
> =0A&= gt;  =0A  =0A  =0A  =0Aat
> =0A>
> =0Aco= m.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.executeQueryWithR= owHandler(GeneralStatement.java:133)
> =0A>  =0A  =0A&nb= sp; =0A  =0Aat
> =0A>
> =0Acom.ibatis.sqlmap.engine.imp= l.SqlMapExecutorDelegate.queryWithRowHandler(SqlMapExecutorDelegate.java:64= 9)
> =0A>  =0A  =0A  =0A  =0Aat
> =0A>= ;
> =0Acom.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.queryWithRowHa= ndler(SqlMapSessionImpl.java:156)
> =0A>  =0A  =0A = =0A  =0Aat
> =0A>
> =0Aorg.myorg.compass.SqlMapClientG= psDeviceWithRowHandler.doIndex(SqlMapClientGpsDeviceWithRowHandler.java:33)=
> =0A>  =0A  =0A  =0A  =0Aat
> =0A><= br>> =0Aorg.compass.gps.device.AbstractGpsDevice$1.doInCompassWithoutRes= ult(AbstractGpsDevice.java:93)
> =0A>  =0A  =0A  = =0A  =0Aat
> =0A>
> =0Aorg.compass.core.CompassCallback= WithoutResult.doInCompass(CompassCallbackWithoutResult.java:29)
> =0A= >  =0A  =0A  =0A  =0Aat
> =0A> =0Aorg.compa= ss.core.CompassTemplate.execute(CompassTemplate.java:132)
> =0A>&n= bsp; =0A  =0A  =0A  =0Aat
> =0A>
> =0Aorg.com= pass.gps.impl.SingleCompassGps.executeForIndex(SingleCompassGps.java:161)> =0A>  =0A  =0A  =0A  =0Aat
> =0A> = =0Aorg.compass.gps.device.AbstractGpsDevice.index(AbstractGpsDevice.java:91= )
> =0A>  =0A  =0A  =0A  =0Aat
> =0A>=
> =0Aorg.compass.spring.device.SpringSyncTransactionGpsDeviceWrapper= $1.doInTransactionWithoutResult(SpringSyncTransactionGpsDeviceWrapper.java:= 98)
> =0A>  =0A  =0A  =0A  =0Aat
> =0A&g= t;
> =0Aorg.springframework.transaction.support.TransactionCallbackWi= thoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33)
&g= t; =0A>  =0A  =0A  =0A  =0Aat
> =0A>
>= ; =0Aorg.springframework.transaction.support.TransactionTemplate.execute(Tr= ansactionTemplate.java:128)
> =0A>  =0A  =0A  =0A&n= bsp; =0Aat
> =0A>
> =0Aorg.compass.spring.device.SpringSyncT= ransactionGpsDeviceWrapper.index(SpringSyncTransactionGpsDeviceWrapper.java= :96)
> =0A>  =0A  =0A  =0A  =0Aat
> =0A&= gt;
> =0Aorg.compass.gps.impl.SingleCompassGps$1.buildIndexIfNeeded(S= ingleCompassGps.java:133)
> =0A>  =0A  =0A  =0A&nbs= p; =0Aat
> =0A>
> =0Aorg.compass.core.lucene.engine.manager.= DefaultLuceneSearchEngineIndexManager$8.firstStep(DefaultLuceneSearchEngine= IndexManager.java:233)
> =0A>  =0A  =0A  =0A  = =0Aat
> =0A>
> =0Aorg.compass.core.lucene.engine.manager.Def= aultLuceneSearchEngineIndexManager.doOperate(DefaultLuceneSearchEngineIndex= Manager.java:182)
> =0A>
> =0A>
> =0A>
> = =0A>
> =0A>
> =0A> =0A----- =0AOriginal =0AMessage =0A= ----
> =0A> =0AFrom: =0ANathan =0AMaves =0A<nathan.mav= es@gmail.com>
> =0A> =0ATo: =0Auser-java= @ibatis.apache.org
> =0A>
> =0A> =0ASent: =0ASaturday= , =0AMarch =0A15, =0A2008 =0A5:43:04 =0APM
> =0A> =0ASubject: =0AR= e: =0Aselect =0A* =0Acausing =0A" =0AOutOfMemoryError: =0AJava =0Aheap =0As= pace"
> =0A>
> =0A>  =0Athis =0Ais =0Aonly =0Amy =0A= 2 =0Acents =0Abut =0AI =0Awould =0Athrow =0Athat =0Aclass =0Aout.  =0A= Write =0Ayour =0Aown
> =0A> =0Aimplementation =0Awhich =0Auses =0A= a =0Arow =0Ahandler.  =0AThis =0Ais =0Athe =0Atype =0Aof =0Asituation = =0Ain
> =0A> =0Awhich =0Aa =0Arow =0Ahandler =0Acould =0Areally = =0Ahelp.  =0Aalmost =0Ano =0Amemory =0Awould =0Abe =0Aused.
> = =0A>
> =0A>
> =0A>
> =0A> =0AOn =0AThu, =0AMa= r =0A13, =0A2008 =0Aat =0A4:57 =0APM, =0Anch =0A<underscore= _dot@yahoo.com> =0Awrote:
> =0A> =0A>
> =0A> = =0A> =0AYes, =0AI =0Aagree =0Athat =0Adividing =0Athe =0Aquery =0Ainto = =0Aseveral =0Aqueries =0Ashould =0Ado =0Aas
> =0A> =0Along =0Aas = =0Anone =0Aof =0Athem =0Auses =0Amore =0Amemory =0Athan =0Aavailable. =0AGr= acias, =0ACarlos.
> =0A> =0A>
> =0A> =0A> =0ANevert= heless, =0AI =0Alooked =0Ainto
> =0A> =0Aorg.compass.gps.device.ib= atis.SqlMapClientGpsDevice#doIndex =0Aand,
> =0A> =0Afundamentally= , =0Awhat =0Ait =0Adoes =0Ais =0Aa =0AqueryForPaginatedList =0Aand =0Aitera= tes
> =0Athrough
> =0A> =0Athe =0Aresulting =0APaginatedList= =0Awith =0AnextPage()
> =0A> =0A>
> =0A> =0A> =0AI= =0Aguess =0Athat =0Athe =0Areason =0Awhy =0Apages =0Aare =0Akept =0Ain =0A= memory =0Aafter =0Aperforming
> =0A> =0ApaginatedList.nextPage() = =0Ais =0Abecause =0Aa =0ApaginatedList.previousPage() =0Acan =0Abe
> = =0A> =0Aissued =0Abut, =0Ashould't =0Ait =0Afree =0Apages =0Aif =0Aheap = =0Aspace =0Ais =0Arunning =0Aout?
> =0A> =0A>
> =0A> = =0A>
> =0A> =0A> =0ACarlos =0Ade =0ALuna =0ASaenz =0A<cdelunasaenz@yahoo.com.mx> =0Awrote:
> =0A> =0A= >
> =0A> =0A> =0ASince =0Awe =0Ahave =0Aa =0Asimilar =0Atrou= ble =0Aaround =0Ahere =0Ai =0Amust =0Asay =0Athat =0Ayou =0Ahave =0Alot
= > =0A> =0Aof =0Aoptions =0Awhen =0Agoing =0Ato =0Acompass..
> = =0A> =0A> =0AThe =0Afirst =0Aone =0Ais =0Ato =0Aindex =0A"object =0Ab= y =0Aobject" =0Aby =0Ahand, =0Athe =0Asecond =0Aone =0Ais
> =0Ato
= > =0A> =0Ause =0Athe =0AGps =0Awich =0Awill =0Ause =0Athe =0Acomplete= =0Abunch =0Aof =0Adata =0Aand =0Amake =0Asubindexes
> =0A> =0A(ma= king =0Aparts =0Aof =0Ayour =0A"main =0Aindex" =0Awith =0Adiferent =0A"maps= " =0Afor =0Aeach =0Asubindex
> =0Ain
> =0A> =0Ayour =0AgpsDr= iver.
> =0A> =0A> =0Aif =0Ayour =0Aindexing =0Ais =0Aas =0Astro= ng =0Aas =0Aours =0Ai =0Ashould =0Alook =0Aforward =0Afor =0Anumer =0A1
= > =0Aor
> =0A> =0Anumber =0A3 =0Adepending =0Aof =0Ayour =0Anee= ds. =0A(More =0Aa =0ACompass =0Athan =0AiBatis =0Aissue)
> =0A> = =0A> =0AGreetings
> =0A> =0A> =0ACarlos =0Ade =0ALuna
>= ; =0A> =0A>
> =0A> =0A> =0A----- =0AMensaje =0Aoriginal = =0A----
> =0A> =0A> =0ADe: =0Anch
> =0A> =0A>
&g= t; =0A> =0A> =0APara: =0Auser-java@ibatis.apache.o= rg; =0Alarry.meadors@gmail.com
> =0A> =0A> = =0AEnviado: =0Ajueves, =0A13 =0Ade =0Amarzo, =0A2008 =0A9:37:04
> =0A= > =0A> =0AAsunto: =0ARe: =0Aselect =0A* =0Acausing =0A" =0AOutOfMemor= yError: =0AJava =0Aheap =0Aspace"
> =0A> =0A>
> =0A> = =0A>
> =0A> =0A> =0ASorry. =0AYes, =0AI =0Aneed =0Aall =0Ath= e =0Adata =0Aat =0Aone =0Atime, =0Abecause =0Athe =0ACompass =0AAPI
>= =0Aseems
> =0A> =0Ato =0Abe =0Ameant =0Ain =0Athat =0Away.
>= ; =0A> =0A>
> =0A> =0A> =0ALarry =0AMeadors =0Awrote: =0A= OK, =0Aso =0Athe =0Aanswer =0Ato =0Athe =0Aquestion =0Ais...[ =0Ayes, =0Ai<= br>> =0Aneed
> =0A> =0Aall =0Athe =0Adata =0Aat
> =0A>= =0A>
> =0A> =0A>
> =0A> =0A>
> =0A> = =0A> =0Aone =0Atime =0A| =0Ano, =0Ai =0Ado =0Anot =0Aneed =0Aall =0Aof = =0Athe =0Adata =0Aat =0Aone =0Atime =0A].
> =0A> =0A>
> = =0A> =0A> =0ALarry
> =0A> =0A>
> =0A> =0A>> =0A> =0A> =0AOn =0AThu, =0AMar =0A13, =0A2008 =0Aat =0A9:04 =0A= AM, =0Anch =0Awrote:
> =0A> =0A> =0A>
> =0A> =0A>= ; =0A> =0AHi!
> =0A> =0A> =0A>
> =0A> =0A> = =0A> =0AI'm =0Adoing =0Atests =0Aboth =0Aunder =0Atomcat =0Aand =0Ajetty= . =0AI =0Aset =0A1024MB =0Aof =0Aheap
> =0Aspace
> =0A> =0Ai= n
> =0A> =0A> =0A> =0Aboth =0Acases. =0AI =0Aam =0Ausing =0A= a =0Aprofiler =0Ato =0Alook =0Ainto =0Awhat's =0Ahappening =0Aduring
>= ; =0A> =0A> =0A> =0Aexecution =0Aof =0Athe =0Aindexing =0Aprocess = =0Aand =0AI =0Acan =0Atell =0Athat's =0Athe =0Aexact
> =0Aamount
&= gt; =0A> =0Ait
> =0A> =0A> =0A> =0Adisplays =0Afor =0Athe= =0AVM.
> =0A> =0A> =0A>
> =0A> =0A> =0A> =0A= As =0AI =0Adescribe =0Ain =0Athe =0ACompass =0Auser =0Aforum =0A(see =0Alin= k =0Abellow), =0AI'm
> =0Aindexing
> =0A> =0Aa
> =0A&g= t; =0A> =0A> =0AWikipedia =0Adump =0Aof =0Aabout =0A650000-675000 =0A= records. =0AThat's =0Asort =0Aof =0A1-1.5GB
> =0Aof
> =0A> = =0A> =0A> =0Adata.
> =0A> =0A> =0A>
> =0A> = =0A> =0A> =0AHibernate =0Acan =0Ahandle =0Aall =0Athat =0Adata, =0AI = =0Aguess =0Ait's =0Ausing =0Asome =0Asort =0Aof
> =0A> =0A> =0A= > =0Apagination. =0AiBatis =0Aseems =0Ato =0Adivide =0Athe =0Aselect =0A= statement =0Ainto =0Aseveral
> =0A> =0Aselect
> =0A> =0A&= gt; =0A> =0Astatements, =0Abut =0Ait =0Aalso =0Aseems =0Aeach =0Aof =0As= uch =0Astatements =0Asurpasses =0Athe
> =0A> =0Alimit =0Aof
>= ; =0A> =0A> =0A> =0A1024MB =0Awhen =0Atrying =0Ato =0Aindex =0Aonl= y =0A300000 =0Aof =0Athe =0Adatabase =0Arecords.
> =0A> =0A> = =0A>
> =0A> =0A> =0A> =0AThank =0Ayou
> =0A> =0A= > =0A>
> =0A> =0A> =0A>
> =0A> =0A> =0A>= ;
> =0A> =0A> =0A> =0AChris =0ALamey =0Awrote:
> =0A&g= t; =0A> =0A> =0AWhat =0Ais =0Ayou =0AJVM =0Aheap =0Asize =0Aset =0Ato= =0Aand =0Ahow =0Amuch =0Adata =0Ais =0Areturned =0Aby
> =0Aselect> =0A> =0A*
> =0A> =0A> =0A> =0Afrom =0Atable?
>= ; =0A> =0A> =0A>
> =0A> =0A> =0A> =0AIf =0Ayou're = =0Atrying =0Ato =0Apull =0Aback =0A1G =0Aworth =0Aof =0Adata =0Ainto =0Aa = =0AJVM =0Awith =0Aa =0Aheap
> =0A> =0Asize
> =0A> =0A>= =0A> =0Aset =0Ato =0A64M, =0Ayou =0Awill =0Ahit =0Athe =0Aheap =0Alimit= =0Apretty =0Aquick.
> =0A> =0A> =0A>
> =0A> =0A>= ; =0A>
> =0A> =0A> =0A> =0A-----Original =0AMessage-----<= br>> =0A> =0A> =0A> =0AFrom: =0Anch =0A[mailto:u= nderscore_dot@yahoo..com]
> =0A> =0A> =0A> =0ASent: =0AT= hu =0A3/13/2008 =0A8:03 =0AAM
> =0A> =0A> =0A> =0ATo: =0Auser-java@ibatis.apache.org
> =0A> =0A> =0A= > =0ASubject: =0Aselect =0A* =0Acausing =0A" =0AOutOfMemoryError: =0AJav= a =0Aheap =0Aspace"
> =0A> =0A> =0A>
> =0A> =0A>= =0A>
> =0A> =0A> =0A> =0AHi, =0Aeverybody.
> =0A&g= t; =0A> =0A> =0AI'm =0Atesting =0Athe =0ACompass =0Asearch =0Aengine = =0Aand, =0Ain =0Aparticular, =0Ahow =0Ato =0Aindex
> =0Aa
> =0A= > =0Abig
> =0A> =0A> =0A> =0Aset =0Aof =0Adocuments =0Afr= om =0Aa =0Atable =0Ain =0Aa =0AMySQL =0Adatabase. =0AIn =0Aorder =0Ato =0Ad= o =0Athis
> =0AI
> =0A> =0A> =0A> =0Aissue =0Aa =0A"se= lect =0A* =0Afrom =0Atable_name" =0Ausing =0AiBatis, =0Abut =0Athis =0Aseem= s =0Ato =0Abe
> =0A> =0A> =0A> =0Acausing =0Athe =0Aapplicat= ion =0Ato =0Ause =0Aall =0Aheap =0Aspace =0Aavailable.
> =0A> =0A&= gt; =0A> =0AI =0Aadded =0Aa =0Anew =0Aentry =0Ainto =0ACompass =0Auser = =0Aforum =0Awith =0Adetails:
> =0A> =0A> =0A> =0Ahttp://forum.compass-project.org/thread.jspa?threadID=3D215278> =0A> =0A> =0A>
> =0A> =0A> =0A> =0ACan =0Ayo= u =0Afigure =0Aout =0Awhy =0Ais =0Athis =0Ahappening?
> =0A> =0A&g= t; =0A>
> =0A> =0A> =0A> =0AMany =0Athanks
> =0A>= ; =0A> =0A>
> =0A> =0A> =0A>
> =0A> =0A> = =0A> =0A---------------------------------
> =0A> =0A> =0A>= ; =0ABe =0Aa =0Abetter =0Afriend, =0Anewshound, =0Aand =0Aknow-it-all =0Awi= th =0AYahoo! =0AMobile. =0ATry
> =0A> =0Ait
> =0A> =0A>= ; =0A> =0Anow.
> =0A> =0A> =0A>
> =0A> =0A> = =0A>
> =0A> =0A> =0A>
> =0A> =0A> =0A>
= > =0A> =0A> =0A> =0A________________________________
> = =0A> =0A> =0A> =0ANever =0Amiss =0Aa =0Athing. =0AMake =0AYahoo = =0Ayour =0Ahomepage.
> =0A> =0A>
> =0A> =0A>
>= ; =0A> =0A> =0ANever =0Amiss =0Aa =0Athing. =0AMake =0AYahoo =0Ayour = =0Ahomepage.
> =0A> =0A>
> =0A> =0A>
> =0A>= ; =0A>
> =0A> =0A>
> =0A> =0A>
> =0A> = =0A>
> =0A> =0A>
> =0A>
> =0A________________= ____________________________________________________________________
>= ; =0A> =0A> =0A=A1Capacidad =0Ailimitada =0Ade =0Aalmacenamiento =0Ae= n =0Atu =0Acorreo!
> =0A> =0A> =0ANo =0Ate =0Apreocupes =0Am=E1= s =0Apor =0Ael =0Aespacio =0Ade =0Atu =0Acuenta =0Acon =0ACorreo =0AYahoo!:=
> =0A> =0A> =0Ahttp://correo.espanol.yahoo.com/
> =0A> =0A>= ;
> =0A> =0A>
> =0A> =0A>
> =0A> =0A>> =0A> =0A>
> =0A> =0A>
> =0A> =0A>
&= gt; =0A> =0A> =0A________________________________
> =0A> =0A= Never =0Amiss =0Aa =0Athing. =0AMake =0AYahoo =0Ayour =0Ahomepage.
> = =0A>
> =0A>
> =0A>
> =0A>
> =0A>&nbs= p; =0A________________________________
> =0A> =0ANever =0Amiss =0A= a =0Athing. =0AMake =0AYahoo =0Ayour =0Ahomepage.
>
>
>&n= bsp; =0A________________________________
>
> =0ABe =0Aa =0Abett= er =0Afriend, =0Anewshound, =0Aand =0Aknow-it-all =0Awith =0AYahoo! =0AMobi= le. =0ATry =0Ait
> =0Anow.


=0A
Looking for last minute shopping deals? =0AFind them fast with Yahoo! Search. --0-2041359295-1205701594=:45308--