lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Noll <dan...@nuix.com.au>
Subject Re: 1.4.3 and 64bit support? out of memory??
Date Wed, 08 Mar 2006 23:09:44 GMT
zzzzz shalev wrote:
> hey chris,
> 
> i will check and let you know just to make sure,
> 
> basically i see the OS allocating memory (up to about 4GB) while
> loading the indexes to memory and then crashing on the
> TermInfosReader class.  what i noticed was that the crash occured
> when lucene tried to create a Term array with the following code
> 
> new Term[indexSize]
> 
> i assume, since this is an array java was trying to allocate
> consecutive blocks in memory and this is hard to find , even in a 16
> GB RAM machine, especially since (if im not mistaken) indexSize here
> is the termEnum size (which in my case is rather large)

That's not exactly how memory works.  When a program looks to allocate a 
chunk of memory, the chunk is allocated from the virtual memory space.

In the case of Windows XP on a 32-bit machine, the maximum contiguous 
virtual memory is somewhere just below 2GB in a best-case scenario 
(usually it's more like 1.5GB) regardless of the amount of physical RAM.

In the case of a 64-bit machine, though, the virtual memory space is 
much, much larger than your 16GB of RAM, so there should be no problem 
allocating ridiculous amounts of memory (or for that matter, memory 
mapping ridiculously large files to a byte buffer.)

It wasn't mentioned explicitly so it's probably worth checking... you 
are using the 64-bit JVM, right?  If you were still using the 32-bit 
JVM, that would certainly exhibit this sort of behaviour.

Daniel


-- 
Daniel Noll

Nuix Pty Ltd
Suite 79, 89 Jones St, Ultimo NSW 2007, Australia        Ph: +61 2 9280 0699
Web: http://www.nuix.com.au/                            Fax: +61 2 9212 6902

This message is intended only for the named recipient. If you are not
the intended recipient you are notified that disclosing, copying,
distributing or taking any action in reliance on the contents of this
message or attachment is strictly prohibited.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message