lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pablo Guerrero <sir...@gmail.com>
Subject Re: Lucene reliability as primary store
Date Sat, 23 Mar 2013 08:13:21 GMT
Thanks Simon,

I'll see if I can implement some kind of transaction log to avoid
committing every change.

Cheers,
Pablo


On Fri, Mar 22, 2013 at 8:13 PM, Simon Willnauer
<simon.willnauer@gmail.com>wrote:

> On Fri, Mar 22, 2013 at 2:00 PM, Pablo Guerrero <siriux@gmail.com> wrote:
> > Hi all,
> >
> > I'm evaluating using Lucene for some data that would not be stored
> anywhere
> > else, and I'm concerned about reliabilty. Having a database storing the
> > data in addition to Lucene would be a problem, and I want to know if
> Lucene
> > is reliable enough.
> >
> > Reading this article,
> > http://blog.mikemccandless.com/2012/03/transactional-lucene.html I think
> > that all committed data would be safe (at least as safe as in, for
> example,
> > MySQL on the same machine) in the event of JVM crash or system crash. Is
> > that true?
>
> yes that is true. Yet, a commit in Lucene is still pretty expensive,
> apps like ElasticSearch or Solr us a Journal / TranactionLog to
> overcome this.
>
> >
> > As an example, if I have an index with some data already committed, A,
> and
> > the JVM crashes during a commit of data B, could the index be corrupted,
> or
> > will just ignore B? If it's corrupted, will CheckIndex be able to
> recover,
> > at least all data in A? Will it be also true in the case of a power
> > shutdown, where the OS buffers are lost, but there is no disk corruption?
>
> unless there is a bug, the index will not be corrupted and B is
> ignored / lost. CheckIndex will not be able to recover your lost docs
> it will only delete broken segments if you ask it to do so. Once you
> commit and lucene returned successfully you should also survice a
> power outage. If you disk is broken then your index will likely be
> broken too.
>
> simon
> >
> > Thank you in advance,
> > Pablo
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message