Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Date: Fri, 23 Dec 2016 18:26:30 +0000
From: Uwe Schindler <uwe@thetaphi.de>
To: java-user@lucene.apache.org,Sadaf <Sadaf.Aslam@tpsgc-pwgsc.gc.ca>
Subject: Re: OOM error
User-Agent: K-9 Mail for Android
In-Reply-To: <1482513290841-4311073.post@n3.nabble.com>
References: <1482513290841-4311073.post@n3.nabble.com>
Message-ID: <917D6CDB-C1BC-4C48-8244-7C03E77070FF@thetaphi.de>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="----0O80W5Q1TJFPJL7L4DAML6YVATNPLC"
Content-Transfer-Encoding: 7bit
archived-at: Fri, 23 Dec 2016 18:26:56 -0000

------0O80W5Q1TJFPJL7L4DAML6YVATNPLC
Content-Type: text/plain;
 charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hi,

For sorting you should add the fields as DocValues type in addition to ind=
exed fields (if you also query on them)=2E That ist the problem=2E But for =
that you should update your Lucene version=2E

For that reason Lucene 6 no longer has FieldCache, which emulates DocValue=
s loading everything to heap, which is horrible for StringFields=2E

Uwe

Am 23=2E Dezember 2016 18:14:50 MEZ schrieb Sadaf <Sadaf=2EAslam@tpsgc-pwg=
sc=2Egc=2Eca>:
>Hi,=20
>
>This is the index we are using:
>Number of fields: 355=2E=20
>Number of documents: 225 thousand=2E=20
>Number of terms: 5522 thousand=2E=20
>The index size is around 800MB=2E
>TermInfos index divisor: 1
>Index format: Lucene 4=2E0=2E=20
>We are getting Java OutOfMemory error with searches=2E We are using a
>heap
>size of 1GB=2E We are not able to increase our heap size=2E
>Looking at the HeapDump, we have two suspects: FieldCacheImpl and
>FieldCacheImpl$BinaryDocValuesImpl=2E
>The searches we are doing have a lot of sorts=2E The sorting is being
>done on
>string fields=2E I don=E2=80=99t have much experience with OutOfMemory er=
rors or
>with
>Lucene=2E What should I try to do?
>-	Should I try to play with the swappiness on the server=2E Currently set
>to
>60?
>-	We are creating a new IndexSearcher for each search? Should I just
>have
>one? If I do that, will my results be updated as new documents are
>being
>added to the index?
>-	If you think that sorting on StringFields is the problem can you give
>me
>some pointers on what the usual suspects are?
>-      Should I insist that we need more Heap?
>Thanks, (I will not be checking my email until new year=2E Wishing
>everyone
>here Happy Holidays)
>Sadaf
>
>
>
>--
>View this message in context:
>http://lucene=2E472066=2En3=2Enabble=2Ecom/OOM-error-tp4311073=2Ehtml
>Sent from the Lucene - Java Users mailing list archive at Nabble=2Ecom=2E
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: java-user-unsubscribe@lucene=2Eapache=2Eorg
>For additional commands, e-mail: java-user-help@lucene=2Eapache=2Eorg

--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www=2Ethetaphi=2Ede
------0O80W5Q1TJFPJL7L4DAML6YVATNPLC--