lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: IndexWriter: setRAMBufferSizeMB
Date Fri, 08 Feb 2008 20:00:58 GMT

It's complicated.

In 2.3, you can use IW.flush to write docs to disk.  But that method  
will be deprecated in 2.4 and replaced with commit.  Or, you can close.

If application (jvm) dies or killed, the index will be fine but won't  
have any un-flushed buffered docs.

If machine dies (os crashes, power cord pulled) then there is a real  
risk that the index will become corrupt.  This is because Lucene has  
never explicitly sync()'d the files to ensure they are on stable  
storage.  LUCENE-1044 fixes that (adds syncs).

Mike

<spring@gmx.eu> wrote:

> OK, so there is nothing in 2.3 besides IndexWriter.close to ensure  
> that the
> docs are written to disk and that the index will survive an  
> application /
> machine death?
>
>> -----Original Message-----
>> From: Michael McCandless [mailto:lucene@mikemccandless.com]
>> Sent: Freitag, 8. Februar 2008 19:34
>> To: java-user@lucene.apache.org
>> Subject: Re: IndexWriter: setRAMBufferSizeMB
>>
>> Well ... every time the RAM buffer is full, a new segment is flushed
>> to the Directory, but that is not necessarily a "commit" in that
>> an IndexReader would see the new segment, nor, that the segment would
>> survive if the machine suddenly crashed.
>>
>> You should't rely on when specifically IndexWriter makes its changes
>> visible to readers.  The best way to be sure is to close the writer.
>>
>> There is work underway now, in this issue:
>>
>>    https://issues.apache.org/jira/browse/LUCENE-1044
>>
>> that will add an explicit "commit" call, which you would use to 1)
>> make the changes visible to readers, and 2) sync the index such that
>> if the machine crashed (after commit returns) then your changes as of
>> the commit will survive.  But it's not committed yet ... it will  
>> be in
>> 2.4.
>>
>> One way for a reader to check if a new commit has happened is to
>> call the isCurrent method.  Maybe that helps?
>>
>> Mike
>>
>> <spring@gmx.eu> wrote:
>>
>>> Hi,
>>>
>>> if I understand this property correctly every time the ram buffer
>>> is full it
>>> gets automaticaly written to disk. Something like a commit in a
>>> database.
>>> Thus if my application dies, all docs in the buffer get lost. Right?
>>>
>>> If so, is there any event/callback etc. which informs my
>>> application that
>>> such a commit happend?
>>>
>>> Thank you.
>>>
>>>
>>>
>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message