Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of Fang_Li@emc.com designates
 128.222.32.20 as permitted sender)
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-2"
Content-Transfer-Encoding: quoted-printable
Subject: RE: exception to open a large index Insufficient system resources
 exist
Date: Tue, 1 Sep 2009 22:49:22 -0400
Message-ID: <247ECD1AB4ED914A80AE4873CFC230A45069F4@CORPUSMX110B.corp.emc.com>
In-Reply-To: <471B926324114432B1D4E5B9F054F93F@VEGA>
Thread-Topic: exception to open a large index Insufficient system resources
 exist
thread-index: Acoq6oECiSqANxdpSp2cvd2nxttaBgAAH1RQAAB3yNAAInlHcA==
References: <247ECD1AB4ED914A80AE4873CFC230A4506845@CORPUSMX110B.corp.emc.com>
 <2ffb6d060909010256ye0a3ca9h44dd5d85fc339260@mail.gmail.com>
 <247ECD1AB4ED914A80AE4873CFC230A4506860@CORPUSMX110B.corp.emc.com>
 <471B926324114432B1D4E5B9F054F93F@VEGA>
From: <Fang_Li@emc.com>
To: <java-user@lucene.apache.org>

32 bit JVM, 1.3G allocated heap size, Lucene 2.4.1.

In my option, this exception should not be caused by out of memory or =
out of system file handle because different exception should be thrown =
for these two cases.

Any hint?

Thanks,
Fang, Li

-----Original Message-----
From: Uwe Schindler [mailto:uwe@thetaphi.de]=20
Sent: Tuesday, September 01, 2009 6:14 PM
To: java-user@lucene.apache.org
Subject: RE: exception to open a large index Insufficient system =
resources exist

Which Lucene version, 64 bit JVM?

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Fang_Li@emc.com [mailto:Fang_Li@emc.com]
> Sent: Tuesday, September 01, 2009 12:04 PM
> To: java-user@lucene.apache.org
> Subject: RE: exception to open a large index Insufficient system =
resources
> exist
>=20
> We are running on Windows 2003 Enterprise Edition with NTFS file =
system on
> a local disc. JDK version is 1.5.0.12.
>=20
> The problem was discussed before and there is no clear solution =
confirmed.
>=20
> Thanks.
>=20
> -----Original Message-----
> From: Danil =DEORIN [mailto:torindan@gmail.com]
> Sent: Tuesday, September 01, 2009 5:56 PM
> To: java-user@lucene.apache.org
> Subject: Re: exception to open a large index Insufficient system =
resources
> exist
>=20
> There should be no problem with large segments.
> Please describe OS, FileSystem and JDK you are running on.
>=20
> There might be some problems with file >2Gb on Win32/FAT, or in some
> ancient Linuxes.
>=20
> On Tue, Sep 1, 2009 at 12:37, <Fang_Li@emc.com> wrote:
> > I met a problem to open an index bigger than 8GB and the following
> > exception was thrown. There is a segment which is bigger than 4GB
> > already. After searching internet, it is said that not using =
compound
> > index may solve the problem.
> >
> > The same exception was thrown when merging with another index =
happens.
> > If the problem is caused by the big segment, we can adjust Lucene
> > parameters to control the segment size.
> > Anyone knows what's the cause for this exception? As reproducing the
> > problem will take a long time, your idea will be save us a lot of
> > effort.
> >
> > java.io.IOException: Insufficient system resources exist to complete =
the
> > requested service
> > =A0 =A0 =A0 =A0at java.io.RandomAccessFile.readBytes(Native Method)
> > =A0 =A0 =A0 =A0at =
java.io.RandomAccessFile.read(RandomAccessFile.java:315)
> > =A0 =A0 =A0 =A0at
> > =
org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirector
> > y.java:596)
> > =A0 =A0 =A0 =A0at
> > =
org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.jav
> > a:157)
> > =A0 =A0 =A0 =A0at
> > =
org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.j
> > ava:38)
> > =A0 =A0 =A0 =A0at
> > org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:78)
> > =A0 =A0 =A0 =A0at
> > =
org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.jav
> > a:73)
> > =A0 =A0 =A0 =A0at
> > =
org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321)
> > =A0 =A0 =A0 =A0at
> > org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
> > =A0 =A0 =A0 =A0at
> > org.apache.lucene.index.SegmentReader.get(SegmentReader.java:228)
> > =A0 =A0 =A0 =A0at
> > =
org.apache.lucene.index.MultiSegmentReader.<init>(MultiSegmentReader.jav
> > a:55)
> > =A0 =A0 =A0 =A0at
> > =
org.apache.lucene.index.DirectoryIndexReader$1.doBody(DirectoryIndexRead
> > er.java:104)
> > =A0 =A0 =A0 =A0at
> > =
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.j
> > ava:653)
> > =A0 =A0 =A0 =A0at
> > =
org.apache.lucene.index.DirectoryIndexReader.open(DirectoryIndexReader.j
> > ava:115)
> > =A0 =A0 =A0 =A0at
> > org.apache.lucene.index.IndexReader.open(IndexReader.java:316)
> > =A0 =A0 =A0 =A0at
> > org.apache.lucene.index.IndexReader.open(IndexReader.java:227)
> >
> > Thanks,
> > -Li
> >
> >
> > =
---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>=20
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>=20


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org