lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ning Li" <ning.li...@gmail.com>
Subject Re: Realtime Search for Social Networks Collaboration
Date Tue, 09 Sep 2008 18:40:58 GMT
>>> Even so,
>>> this may not be sufficient for some FS such as HDFS... Is it
>>> reasonable in this case to keep in memory everything including
>>> stored fields and term vectors?
>>
>> We could maybe do something like a proxy IndexInput/IndexOutput that
>> would allow updating the read buffer from the writer buffer.
>
> Does HDFS disallow a reader from reading a file that's still open for
> append?

HDFS allows that. "A reader is guaranteed to be able to read data that
was 'flushed' before the reader opened the file." However, it may not
see the latest appends (after open) even if they are flushed. Yonik's
comments below also apply in this case.

> Right, but it would be nice to not have to open a new IndexInput for
> each snapshot... opening a file is not a quick operation.

Cheers,
Ning

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message