Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: local policy)
Content-class: urn:content-classes:message
Subject: RE: 1.4.3 and 64bit support? out of memory??
Date: Thu, 9 Mar 2006 09:04:06 +0200
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Message-ID: <39B0FB508E5D7540ACA5AD57225E150D3920C9@xmail.me.corp.entopia.com>
Thread-Topic: 1.4.3 and 64bit support? out of memory??
Thread-Index: AcZC/AWMhoepUagHRjyhF9VMDpm/VQASur1g
From: "Andrew Schetinin" <aschetinin@entopia.com>
To: <java-user@lucene.apache.org>

Hi,

That's somewhat strange, if I remember correctly the index size was 6
Gb, wasn't it?=20
I saw posts from people working with tens of Gb indexes. And we worked
with index of 8 Gb in 32-bit JVM (on Windows 2000) with as little as 700
Mb of max memory allowed to JVM.

Are there too many documents/terms/segments the index contains?=20
Did you try opening the same index with Luke? Allow to Luke 1-2 Gb and
give it a try...
Is it possible that the index is corrupted by some reason and there is
invalid number of term entires like 0x7effffff in the dictionary?

Best Regards,

Andrew


=20


-----Original Message-----
From: zzzzz shalev [mailto:zzzzz_shalev@yahoo.com]=20
Sent: Thursday, March 09, 2006 12:02 AM
To: java-user@lucene.apache.org
Subject: Re: 1.4.3 and 64bit support? out of memory??

hey chris,
  =20
  i will check and let you know just to make sure,
  =20
  basically i see the OS allocating memory (up to about 4GB) while
loading the indexes to memory and then crashing on the TermInfosReader
class.  what i noticed was that the crash occured when lucene tried to
create a Term array with the following code
  =20
  new Term[indexSize]
  =20
  i assume, since this is an array java was trying to allocate
consecutive blocks in memory and this is hard to find , even in a 16 GB
RAM machine, especially since (if im not mistaken) indexSize here is the
termEnum size (which in my case is rather large)
  =20
  i will get back to you about the one liner, if you have any other
thoughts id be extremely happy to hear them as this problem is a Major
road block=20
  =20
  thanks a million
  =20
 =20

Chris Hostetter <hossman_lucene@fucit.org> wrote:
 =20
: i am recieving the following stack trace:
:
: JVMDUMP013I Processed Dump Event "uncaught", detail
"java/lang/OutOfMemoryError".
: Exception in thread "main" java.lang.OutOfMemoryError
: at
org.apache.lucene.index.TermInfosReader.readIndex(TermInfosReader.java:8
2)

is it possible that parts of your application are eating up all of the
heap in your JVM before this exception is encountered? Possibly by
opening a the index many times without closing it?

More specifically, if you write a 4 line app that does nothing by open
your index and then close it again, do you get an OOM? ...

public class Main {
public static void main(String[] args) throws Exception { Searcher s =3D
new IndexSearcher("/your/index/path");
s.close();
}
}


-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


	=09
---------------------------------
Yahoo! Mail
Bring photos to life! New PhotoMail  makes sharing a breeze.=20


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org