lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ning Li" <ning.li...@gmail.com>
Subject Re: Realtime Search for Social Networks Collaboration
Date Tue, 09 Sep 2008 15:42:54 GMT
On Mon, Sep 8, 2008 at 4:23 PM, Yonik Seeley <yonik@apache.org> wrote:
>> I thought an index reader which supports real-time search no longer
>> maintains a static view of an index?
>
> It seems advantageous to just make it really cheap to get a new view
> of the index (if you do it for every search, t amounts to the same
> thing, right?)

Sounds like these light-weight views of the index are backed up by
something dynamic, right?


> Quite a bit of code in Lucene assumes a static view of
> the Index I think (even IndexSearcher), and it's nice to have a stable
> index view for the duration of a single request.

Agree.


On Tue, Sep 9, 2008 at 10:02 AM, Yonik Seeley <yonik@apache.org> wrote:
> Yeah, I think the underlying RandomAccessFile might do the right
> thing, but IndexInput isn't required to see any changes on the fly
> (and current implementations don't) so at a minimum it would be a
> change of IndexInput semantics.  Maybe there would need to be a
> refresh() function added, or we would need to require a specific
> Directory impl?
>
> OR, if all writes are append-only, perhaps we don't ever need to
> invalidate the read buffer and would just need to remove the current
> logic that caches the file length and then let the underlying
> RandomAccessFile do the EOF checking.

We cannot assume it's always RandomAccessFile, can we?
So we may have to flush after writing each document. Even so,
this may not be sufficient for some FS such as HDFS... Is it
reasonable in this case to keep in memory everything including
stored fields and term vectors?


Cheers,
Ning

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message