lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <>
Subject Re: Realtime Search for Social Networks Collaboration
Date Mon, 08 Sep 2008 16:33:00 GMT

I'd also trying to make time to explore the approach of creating an  
IndexReader impl. that searches IndexWriter's RAM buffer.

I think it's quite feasible, but, it'd still have a "reopen" cost in  
that any buffered delete by term or query would have to be  
"materialiazed" into docIDs on reopen.  Though, if this somehow turns  
out to be a problem, in the future we could do this materializing  
immediately, instead of buffering, if we already have a reader open.

Flushing is somewhat tricky because any open RAM readers would then  
have to cutover to the newly flushed segment once the flush completes,  
so that the RAM buffer can be recycled for the next segment.


Jason Rutherglen wrote:

> InstantiatedIndex isn't quite realtime.  Instead a new
> InstantiatedIndex is created per transaction in Ocean and managed
> thereafter.  This however is fairly easy to build and could offer
> realtime in Lucene without adding the transaction logging.  It would
> be good to find out what scope is acceptable for a Lucene core version
> of realtime.  Perhaps this basic feature set is good enough.
> On Mon, Sep 8, 2008 at 10:23 AM, Mark Miller <>  
> wrote:
>> Ning Li wrote:
>>> I agree with Otis that the first step for Lucene is probably to
>>> support real-time
>>> search. The instantiated index in contrib seems to be something  
>>> close..
>> Maybe we should start fleshing out what we want in realtime search  
>> on the
>> wiki?
>> Could it be as simple as making InstantiatedIndex realtime (allow
>> writes/read at same time?). Then you could search over your  
>> IndexReader as
>> well as the InstantiatedIndex. Writes go to both the Writer and the
>> InstantiatedIndex. Nothing is actually permanent until the true  
>> commit, but
>> stuff is visible pretty fast...a new IndexReader view starts a fresh
>> InstantiedIndex...
>> Jasons realtime patch is still pretty large...would be nice if we  
>> could
>> accomplish this with as few changes as possible...
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message