Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: neutral (asf.osuosl.org: local policy)
Date: Sun, 10 Jul 2005 16:15:30 -0700 (PDT)
From: Chris Hostetter <hossman_lucene@fucit.org>
Sender: hossman@hal.rescomp.berkeley.edu
To: java-user@lucene.apache.org
Subject: Re: Index Partitioning ( was Re: Search deadlocking under load)
In-Reply-To: <73809823-CE46-406E-A142-6C4C0EA1E0E4@aconex.com>
Message-ID: <Pine.LNX.4.58.0507101605250.22002@hal.rescomp.berkeley.edu>
References: <20050708193944.61757.qmail@web31112.mail.mud.yahoo.com>
 <73809823-CE46-406E-A142-6C4C0EA1E0E4@aconex.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII


: Nathan's point about pooling Searchers is something that we also
: addressed by a LRU cache mechanism.  In testing we also found that

Generally speaking, you only ever need one active Searcher, which all of
your threads should be able to use.  (Of course, Nathan says that in his
code base, doing this causes his JVM to freeze up, but I've never seen
this myself).

I say one "active" Searcher because it might make sense in your
application to open a new searcher after new documents have been added, do
some searches on that new Searcher to "warm" FieldCache for ssorting, and
then close the old searcher and make the new Searcher available to all of
your query threads.

: However his 2nd point is interesting that creating a new index each
: time eventually suffered OutOfMemory (even though he's closing them)
: is a worry.   Is this because an IndexSearcher can be closed, but the
: underlying IndexReader is not automatically closed?

As I understand it, the general rule is: if you call IndexReader.open, you
better call .close() on that reader.  If you construct and IndexSearcher
using a Directory or a path, then calling .close() on the searcher will
take care of closing the reader -- but if your code look like this...

    Searcher s = new IndexSearcher(IndexReader.open(foo))

...then you are screwed, because nothing will ever close that reader and
free it's resources.


-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org