lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <spr...@gmx.eu>
Subject RE: IndexWriter: setRAMBufferSizeMB
Date Sun, 10 Feb 2008 13:10:32 GMT
Thank you.
So I will call flush in 2.3 (and may lose data when machine dies) and
commit() in 2.4+ (here a sync() will save the data).

> -----Original Message-----
> From: Michael McCandless [mailto:lucene@mikemccandless.com] 
> Sent: Freitag, 8. Februar 2008 21:01
> To: java-user@lucene.apache.org
> Subject: Re: IndexWriter: setRAMBufferSizeMB
> 
> 
> It's complicated.
> 
> In 2.3, you can use IW.flush to write docs to disk.  But that method  
> will be deprecated in 2.4 and replaced with commit.  Or, you 
> can close.
> 
> If application (jvm) dies or killed, the index will be fine 
> but won't  
> have any un-flushed buffered docs.
> 
> If machine dies (os crashes, power cord pulled) then there is a real  
> risk that the index will become corrupt.  This is because Lucene has  
> never explicitly sync()'d the files to ensure they are on stable  
> storage.  LUCENE-1044 fixes that (adds syncs).
> 
> Mike
> 
> <spring@gmx.eu> wrote:
> 
> > OK, so there is nothing in 2.3 besides IndexWriter.close to ensure  
> > that the
> > docs are written to disk and that the index will survive an  
> > application /
> > machine death?
> >
> >> -----Original Message-----
> >> From: Michael McCandless [mailto:lucene@mikemccandless.com]
> >> Sent: Freitag, 8. Februar 2008 19:34
> >> To: java-user@lucene.apache.org
> >> Subject: Re: IndexWriter: setRAMBufferSizeMB
> >>
> >> Well ... every time the RAM buffer is full, a new segment 
> is flushed
> >> to the Directory, but that is not necessarily a "commit" in that
> >> an IndexReader would see the new segment, nor, that the 
> segment would
> >> survive if the machine suddenly crashed.
> >>
> >> You should't rely on when specifically IndexWriter makes 
> its changes
> >> visible to readers.  The best way to be sure is to close 
> the writer.
> >>
> >> There is work underway now, in this issue:
> >>
> >>    https://issues.apache.org/jira/browse/LUCENE-1044
> >>
> >> that will add an explicit "commit" call, which you would use to 1)
> >> make the changes visible to readers, and 2) sync the index 
> such that
> >> if the machine crashed (after commit returns) then your 
> changes as of
> >> the commit will survive.  But it's not committed yet ... it will  
> >> be in
> >> 2.4.
> >>
> >> One way for a reader to check if a new commit has happened is to
> >> call the isCurrent method.  Maybe that helps?
> >>
> >> Mike
> >>
> >> <spring@gmx.eu> wrote:
> >>
> >>> Hi,
> >>>
> >>> if I understand this property correctly every time the ram buffer
> >>> is full it
> >>> gets automaticaly written to disk. Something like a commit in a
> >>> database.
> >>> Thus if my application dies, all docs in the buffer get 
> lost. Right?
> >>>
> >>> If so, is there any event/callback etc. which informs my
> >>> application that
> >>> such a commit happend?
> >>>
> >>> Thank you.
> >>>
> >>>
> >>>
> >> 
> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >>> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>>
> >>
> >>
> >> 
> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >
> >
> > 
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message