ibatis-user-java mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Larry Meadors" <larry.mead...@gmail.com>
Subject Re: select * causing " OutOfMemoryError: Java heap space"
Date Thu, 13 Mar 2008 15:20:18 GMT
OK, so the answer to the question is...[ yes, i need all the data at
one time | no, i do not need all of the data at one time ].


On Thu, Mar 13, 2008 at 9:04 AM, nch <underscore_dot@yahoo.com> wrote:
> Hi!
> I'm doing tests both under tomcat and jetty. I set 1024MB of heap space in
> both cases. I am using a profiler to look into what's happening during
> execution of the indexing process and I can tell that's the exact amount it
> displays for the VM.
> As I describe in the Compass user forum (see link bellow), I'm indexing a
> Wikipedia dump of about 650000-675000 records. That's sort of 1-1.5GB of
> data.
> Hibernate can handle all that data, I guess it's using some sort of
> pagination. iBatis seems to divide the select statement into several select
> statements, but it also seems each of such statements surpasses the limit of
> 1024MB when trying to index only 300000 of the database records.
> Thank you
> Chris Lamey <clamey@localmatters.com> wrote:
>  What is you JVM heap size set to and how much data is returned by select *
> from table?
> If you're trying to pull back 1G worth of data into a JVM with a heap size
> set to 64M, you will hit the heap limit pretty quick.
> -----Original Message-----
> From: nch [mailto:underscore_dot@yahoo.com]
> Sent: Thu 3/13/2008 8:03 AM
> To: user-java@ibatis.apache.org
> Subject: select * causing " OutOfMemoryError: Java heap space"
> Hi, everybody.
> I'm testing the Compass search engine and, in particular, how to index a big
> set of documents from a table in a MySQL database. In order to do this I
> issue a "select * from table_name" using iBatis, but this seems to be
> causing the application to use all heap space available.
> I added a new entry into Compass user forum with details:
> http://forum.compass-project.org/thread.jspa?threadID=215278
> Can you figure out why is this happening?
> Many thanks
> ---------------------------------
> Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it
> now.
>  ________________________________
> Never miss a thing. Make Yahoo your homepage.

View raw message