lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Engels" <reng...@ix.netcom.com>
Subject RE: Lock handling and Lucene 1.9 / 2.0
Date Tue, 14 Sep 2004 21:34:41 GMT
I think the problem is Peter's bias towards Pico, and its IoC philosophy.
This coupled with a J2EE Web Environment leads to massive object creation,
something most highly scalable systems attempt to avoid.

Peter, am I safe to assume that you would ideally like to code something
like (in response to each web request).

IndexReader r = pico.getObjectInstance(IndexReader.class)

If so, this is probably not going to work in the case of Lucene, or in ANY
system that has objects that maintain a lot of state - the creation of the
state consumes far too many resources, so the proper way is to create the
state once, and share access to it.

As Doug points out, it is the same thing with database connections... they
are extremely expensive to create, so a pool of these connections is
created, and each request contends for a connection. If there are more
incoming requests than the pool has connections some requests will either
block or fail.

So create a pool of IndexReaders and have the requests contend for them...
BUT, Lucene is more efficient than database connections, in that, once the
IndexReader is created, then multiple request can share it - not block
waiting on it.

The only time you need to drop and recreate IndexReaders is if you want the
IndexReader to be able to see the "latest" data, because otherwise they will
only a snapshot of the data when they were created.

Robert


-----Original Message-----
From: Doug Cutting [mailto:cutting@apache.org]
Sent: Tuesday, September 14, 2004 4:13 PM
To: Lucene Developers List
Subject: Re: Lock handling and Lucene 1.9 / 2.0


Pete Lewis wrote:
> The only way to continually use the same IndexReader would be to use a
> stateful session bean (frowned upon by J2EE Container writers)

Can one implement DB connection pooling in J2EE?  This is analogous.
One may keep a pool of IndexReaders that are reused by subsequent
queries.  One difference is that the cache need only contain a single
IndexReader per index, rather than a DB connection pool, which typically
keeps multiple connections per DB.  Also, at checkout, the cache code
should check whether a newer version of the index is available, and, if
it is, update the cache.

If there are lots of different indexes, more than you'd like to keep
open at once, then an LRU cache might work well, implemented e.g. with
LinkedHashMap.  Such a cache might be a useful contribution to Lucene.

> I thought that it might be a good candidate for Lucene 2 as the
FSDirectory
> code is horrible and non-J2EE compliant.

Your constructive criticism is greatly appreciated!

Have a nice day,

Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


Mime
View raw message