lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: Lock handling and Lucene 1.9 / 2.0
Date Tue, 14 Sep 2004 11:11:47 GMT
It sounds like this is the source of your problem.  Reuse those
IndexReaders or IndexSearchers and you'll avoid the waits you are
talking about, as Christoph already pointed out.  How you implement the
logic for caching IndexReaders/Searchers is up to you.

Otis
P.S.
It sounds like you took the time to figure out how some of the Lucene
internals work.  The best way to persuade -dev list subscribers that
doing something one way is better than doing it the current way is by
providing a clean diff against CVS HEAD, attached to an entry in
Bugzilla.  There are now several active Lucene developers and
contributors who will look at your suggestion and apply the patch, if
it improves the code.



--- Pete Lewis <pete@uptima.co.uk> wrote:

> Hi Christoph
> 
> We are in a cluster running under Bea Weblogic.
> 
> We have a static API to the search component from the portlets.
> 
> We therefore need to open an indexreader per request.
> 
> Cheers
> Pete
> 
> ----- Original Message ----- 
> From: "Christoph Goller" <goller@detego-software.de>
> To: "Lucene Developers List" <lucene-dev@jakarta.apache.org>
> Sent: Tuesday, September 14, 2004 10:55 AM
> Subject: Re: Lock handling and Lucene 1.9 / 2.0
> 
> 
> > Pete Lewis wrote:
> > > Hi Christoph
> > >
> > > If we stand back a second and ask why we have commit locks when
> searching?
> > >
> > > The answer comes down to handling the FSDirectory - where the
> methods
> used
> > > are not j2ee compliant.
> > >
> > > We could open another can of worms and say why does the
> indexreader
> delete -
> > > but I won't go into that argument again here.....
> > >
> > > The bottom line is that we need the ability to search without
> waiting on
> a
> > > commit lock.
> >
> > You have this ability already !!!!!!!!!!!!!!!!!!!!!!!!!
> >
> > The FSDirectory is where the problems lie.  We could hack the
> > > code to make it work for our particular application; however what
> I've
> been
> > > trying to get across is the need to have a method that will give
> users
> the
> > > capability to just search (not delete) without waiting upon the
> commit
> lock,
> > > that will be j2ee compliant, and that will be appropriate
> clustered
> > > implementations - and that this should be a candidate for Lucene
> 1.9 /
> 2.0.
> > >
> > > You say that it shouldn't take long to wait.  A 1 sec spin lock
> per
> index
> > > per process is an eternity when trying to scale for potentially
> thousands of
> > > users.
> >
> > I have to admit that I am not an expert in j2ee compliancy. But I
> would
> like
> > to learn about it. If a database (I consder Lucene as a database)
> really
> has
> > to be initialized for every read-access, than there is a problem
> with j2ee
> > compliancy. I cannot believe that this is really true.
> >
> > LET ME STATE AGAIN: You should not open a new IndexReader for every
> > search/query. If you do so you definitely have a performance
> problem
> > independently from synchronization!!!!!!!!! Opening an IndexReader
> is
> > much more expensive than any individual query/search.
> >
> > You need a commit.lock for opening an IndexReader not because
> IndexReader
> > has delete functionality. You need it because if there is some
> process
> > writing to your index, your index may be in an inconsistent state.
> An
> existing
> > commit.lock indicates such an inconsistent state. Therefore, every
> writer
> needs
> > a commit.lock while committing, and every reader needs a
> commit.lock while
> > opening an index.
> >
> > Christoph
> >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
> >
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


Mime
View raw message