Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (herse.apache.org: local policy)
Message-ID: <462D044F.9020802@sirma.bg>
Date: Mon, 23 Apr 2007 22:09:03 +0300
From: Ivan Vasilev <ivasilev@sirma.bg>
User-Agent: Thunderbird 1.5.0.10 (Windows/20070221)
MIME-Version: 1.0
To: java-user@lucene.apache.org
Subject: Re: Out of memory exception for big indexes
References: <46162A72.50400@sirma.bg>
 <359a92830704060730l76d38d22g44be77aa3bff3845@mail.gmail.com>
 <Pine.LNX.4.58.0704061101020.15801@hal.rescomp.berkeley.edu>
 <872e2d490704061203k6923dbd2rd6ffe0117a46837@mail.gmail.com>
 <f36128690704072203i3ecf944bidbbb8769681b1ea4@mail.gmail.com>
 <814127511.20070408213229@gmail.com>
In-Reply-To: <814127511.20070408213229@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Hi All,
THANK YOU FOR YOUR HELP :)
I put this problem in the forum but I had no chance to work on it last 
week unfurtunately...
So now I tested the Artem's patch but the results show:
1) speed is very slow compare with the usage without patch
2) There are not very big differences of memory usage (I tested till now 
only with relativly small indexes - less than 1 GB and less than 1 mil 
docs because the when using with 20-40 GB indexes I had to wait more 
than 5 mins what is practically usless).

So I have doubts if I use the patch correctly. I do just what is 
described in Artem's letter:

AV> You can include StoredFieldSortFactory class source file into your sources and
AV> then use StoredFieldSortFactory.create(sortFieldName, sortDescending) to get
AV> Sort object for sorting query.
AV> StoredFieldSortFactory source file can be extracted from LUCENE-769 patch or
AV> from sharehound sources: http://sharehound.cvs.sourceforge.net/*checkout*/sharehound/jNetCrawler/src/java/org/apache/lucene/search/StoredFieldSortFactory.java


What I am wondering about is that in the patch commetns 
(https://issues.apache.org/jira/browse/LUCENE-769) I see that there is 
written that patch solves the problem by using WeakHashMap, but actually 
in the downloaded StoredFieldSortFactory.java  file there is not used 
WeakHashMap. Another thing: In the comments in Lucene-769 issue there is 
mentioned something about classes like: WeakDocumentsCache and 
DocCachingIndexReader but I did not found them in Lucene source code 
neither as classes in StoredFieldSortFactory.java. So my questions are:
1. Is it enought to include the file StoredFieldSortFactory.java in the 
source code or there are also other classes that I have to douwnload and 
include?
2. Have I to use this DocCachingIndexReader instead of Reader that I 
currently use in cases when I expect OOMException and will use this patch?

Thanks to all once again :),
Ivan

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org