lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Schetinin" <ascheti...@entopia.com>
Subject RE: 1.4.3 and 64bit support? out of memory??
Date Thu, 09 Mar 2006 07:04:06 GMT
Hi,

That's somewhat strange, if I remember correctly the index size was 6
Gb, wasn't it? 
I saw posts from people working with tens of Gb indexes. And we worked
with index of 8 Gb in 32-bit JVM (on Windows 2000) with as little as 700
Mb of max memory allowed to JVM.

Are there too many documents/terms/segments the index contains? 
Did you try opening the same index with Luke? Allow to Luke 1-2 Gb and
give it a try...
Is it possible that the index is corrupted by some reason and there is
invalid number of term entires like 0x7effffff in the dictionary?

Best Regards,

Andrew


 



-----Original Message-----
From: zzzzz shalev [mailto:zzzzz_shalev@yahoo.com] 
Sent: Thursday, March 09, 2006 12:02 AM
To: java-user@lucene.apache.org
Subject: Re: 1.4.3 and 64bit support? out of memory??

hey chris,
   
  i will check and let you know just to make sure,
   
  basically i see the OS allocating memory (up to about 4GB) while
loading the indexes to memory and then crashing on the TermInfosReader
class.  what i noticed was that the crash occured when lucene tried to
create a Term array with the following code
   
  new Term[indexSize]
   
  i assume, since this is an array java was trying to allocate
consecutive blocks in memory and this is hard to find , even in a 16 GB
RAM machine, especially since (if im not mistaken) indexSize here is the
termEnum size (which in my case is rather large)
   
  i will get back to you about the one liner, if you have any other
thoughts id be extremely happy to hear them as this problem is a Major
road block 
   
  thanks a million
   
  

Chris Hostetter <hossman_lucene@fucit.org> wrote:
  
: i am recieving the following stack trace:
:
: JVMDUMP013I Processed Dump Event "uncaught", detail
"java/lang/OutOfMemoryError".
: Exception in thread "main" java.lang.OutOfMemoryError
: at
org.apache.lucene.index.TermInfosReader.readIndex(TermInfosReader.java:8
2)

is it possible that parts of your application are eating up all of the
heap in your JVM before this exception is encountered? Possibly by
opening a the index many times without closing it?

More specifically, if you write a 4 line app that does nothing by open
your index and then close it again, do you get an OOM? ...

public class Main {
public static void main(String[] args) throws Exception { Searcher s =
new IndexSearcher("/your/index/path");
s.close();
}
}



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org



		
---------------------------------
Yahoo! Mail
Bring photos to life! New PhotoMail  makes sharing a breeze. 


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message