ibatis-user-java mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlos de Luna Saenz <cdelunasa...@yahoo.com.mx>
Subject Re: select * causing " OutOfMemoryError: Java heap space"
Date Thu, 13 Mar 2008 16:15:27 GMT
Since we have a similar trouble around here i must say that you have lot of options when going
to compass..
The first one is to index "object by object" by hand, the second one is to use the Gps wich
will use the complete bunch of data and make subindexes (making parts of your "main index"
with diferent "maps" for each subindex in your gpsDriver.
if your indexing is as strong as ours i should look forward for numer 1 or number 3 depending
of your needs. (More a Compass than iBatis issue)
Greetings
Carlos de Luna

----- Mensaje original ----
De: nch <underscore_dot@yahoo.com>
Para: user-java@ibatis.apache.org; larry.meadors@gmail.com
Enviado: jueves, 13 de marzo, 2008 9:37:04
Asunto: Re: select * causing " OutOfMemoryError: Java heap space"


Sorry. Yes, I need all the data at one time, because the Compass API seems to be meant in
that way.

Larry Meadors <larry.meadors@gmail.com> wrote: OK, so the answer to the question is...[
yes, i need all the data at
one time | no, i do not need all of the data at one time ].

Larry


On Thu, Mar 13, 2008 at 9:04 AM, nch  wrote:
>
> Hi!
>
> I'm doing tests both under tomcat and jetty. I set 1024MB of heap space in
> both cases. I am using a profiler to look into what's happening during
> execution of the indexing process and I can tell that's the exact amount it
> displays for the VM.
>
> As I describe in the Compass user forum (see link bellow), I'm indexing a
> Wikipedia dump of about 650000-675000 records. That's sort of 1-1.5GB  of
> data.
>
> Hibernate can handle all that data, I guess it's using some sort of
> pagination. iBatis seems to divide the select statement into several select
> statements, but it also seems each of such statements surpasses the limit of
> 1024MB when trying to index only 300000 of the database records.
>
> Thank you
>
>
>
> Chris Lamey  wrote:
>  What is you JVM heap size set to and how much data is returned by select *
> from table?
>
> If you're trying to pull back 1G worth of data into a JVM with a heap size
> set to 64M, you will hit the heap limit pretty quick.
>
>
> -----Original Message-----
> From: nch [mailto:underscore_dot@yahoo..com]
> Sent: Thu 3/13/2008 8:03 AM
> To: user-java@ibatis.apache.org
> Subject: select * causing " OutOfMemoryError: Java heap space"
>
>
> Hi,  everybody.
> I'm testing the Compass search engine and, in particular, how to index a big
> set of documents from a table in a MySQL database. In order to do this I
> issue a "select * from table_name" using iBatis, but this seems to be
> causing the application to use all heap space available.
> I added a new entry into Compass user forum with details:
> http://forum.compass-project.org/thread.jspa?threadID=215278
>
> Can you figure out why is this happening?
>
> Many thanks
>
>
> ---------------------------------
> Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it
> now.
>
>
>
>
>  ________________________________
> Never miss a thing. Make Yahoo your homepage.

         
Never miss a thing.   Make Yahoo your homepage.  






      ____________________________________________________________________________________
¡Capacidad ilimitada de almacenamiento en tu correo!
No te preocupes más por el espacio de tu cuenta con Correo Yahoo!:                      
http://correo.espanol.yahoo.com/


Mime
View raw message