lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pete Lewis" <p...@uptima.co.uk>
Subject Re: Lock handling and Lucene 1.9 / 2.0
Date Tue, 14 Sep 2004 19:43:21 GMT
Hi Otis

The only way to continually use the same IndexReader would be to use a
stateful session bean (frowned upon by J2EE Container writers), and then
we'd see the same problem cross-machine - remember that the indexes that we
use are stored once, not replicated across the four servers.

We'll come up with a fix ourselves.

I thought that it might be a good candidate for Lucene 2 as the FSDirectory
code is horrible and non-J2EE compliant.

Is there any place for making suggestions for future development tasks?
I've seen the Lucene 2 whiteboard and wondered how the items on the list
were derived.

Cheers

Pete Lewis


----- Original Message ----- 
From: "Otis Gospodnetic" <otis_gospodnetic@yahoo.com>
To: "Lucene Developers List" <lucene-dev@jakarta.apache.org>
Sent: Tuesday, September 14, 2004 12:11 PM
Subject: Re: Lock handling and Lucene 1.9 / 2.0


> It sounds like this is the source of your problem.  Reuse those
> IndexReaders or IndexSearchers and you'll avoid the waits you are
> talking about, as Christoph already pointed out.  How you implement the
> logic for caching IndexReaders/Searchers is up to you.
>
> Otis
> P.S.
> It sounds like you took the time to figure out how some of the Lucene
> internals work.  The best way to persuade -dev list subscribers that
> doing something one way is better than doing it the current way is by
> providing a clean diff against CVS HEAD, attached to an entry in
> Bugzilla.  There are now several active Lucene developers and
> contributors who will look at your suggestion and apply the patch, if
> it improves the code.
>
>
>
> --- Pete Lewis <pete@uptima.co.uk> wrote:
>
> > Hi Christoph
> >
> > We are in a cluster running under Bea Weblogic.
> >
> > We have a static API to the search component from the portlets.
> >
> > We therefore need to open an indexreader per request.
> >
> > Cheers
> > Pete
> >
> > ----- Original Message ----- 
> > From: "Christoph Goller" <goller@detego-software.de>
> > To: "Lucene Developers List" <lucene-dev@jakarta.apache.org>
> > Sent: Tuesday, September 14, 2004 10:55 AM
> > Subject: Re: Lock handling and Lucene 1.9 / 2.0
> >
> >
> > > Pete Lewis wrote:
> > > > Hi Christoph
> > > >
> > > > If we stand back a second and ask why we have commit locks when
> > searching?
> > > >
> > > > The answer comes down to handling the FSDirectory - where the
> > methods
> > used
> > > > are not j2ee compliant.
> > > >
> > > > We could open another can of worms and say why does the
> > indexreader
> > delete -
> > > > but I won't go into that argument again here.....
> > > >
> > > > The bottom line is that we need the ability to search without
> > waiting on
> > a
> > > > commit lock.
> > >
> > > You have this ability already !!!!!!!!!!!!!!!!!!!!!!!!!
> > >
> > > The FSDirectory is where the problems lie.  We could hack the
> > > > code to make it work for our particular application; however what
> > I've
> > been
> > > > trying to get across is the need to have a method that will give
> > users
> > the
> > > > capability to just search (not delete) without waiting upon the
> > commit
> > lock,
> > > > that will be j2ee compliant, and that will be appropriate
> > clustered
> > > > implementations - and that this should be a candidate for Lucene
> > 1.9 /
> > 2.0.
> > > >
> > > > You say that it shouldn't take long to wait.  A 1 sec spin lock
> > per
> > index
> > > > per process is an eternity when trying to scale for potentially
> > thousands of
> > > > users.
> > >
> > > I have to admit that I am not an expert in j2ee compliancy. But I
> > would
> > like
> > > to learn about it. If a database (I consder Lucene as a database)
> > really
> > has
> > > to be initialized for every read-access, than there is a problem
> > with j2ee
> > > compliancy. I cannot believe that this is really true.
> > >
> > > LET ME STATE AGAIN: You should not open a new IndexReader for every
> > > search/query. If you do so you definitely have a performance
> > problem
> > > independently from synchronization!!!!!!!!! Opening an IndexReader
> > is
> > > much more expensive than any individual query/search.
> > >
> > > You need a commit.lock for opening an IndexReader not because
> > IndexReader
> > > has delete functionality. You need it because if there is some
> > process
> > > writing to your index, your index may be in an inconsistent state.
> > An
> > existing
> > > commit.lock indicates such an inconsistent state. Therefore, every
> > writer
> > needs
> > > a commit.lock while committing, and every reader needs a
> > commit.lock while
> > > opening an index.
> > >
> > > Christoph
> > >
> > >
> > >
> > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


Mime
View raw message