lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacob Rhoden <>
Subject Re: Lucene as a primary datastore
Date Wed, 20 Jan 2010 22:09:26 GMT
In the same way that you should take regular exports/dumps of your mysql databases, you could
have the same strategy with lucene.

As long as you have code that can export your data that runs daily, and code that can rebuild
your index from that data, In the event of a problem the most you will loose is up to 24 hours
of data yes?

The whole concept of using lucene as the data store has also been on my mind, simply because
I have some systems where the lucene index is simply a copy of all of the mysql data, makes
me wonder why I even bother with the mysql part (:

On 20/01/2010, at 9:30 PM, Chris Harris wrote:

> I don't do a lot of work with straight Lucene right now, but I do use
> Solr, and from time to time the Lucene index inside my master Solr
> server gets corrupted; in particular, some of the Lucene segment files
> that are still in use somehow get deleted, resulting in Lucene
> throwing FileNotFoundExceptions. Once this happens, I have to either
> rebuild the whole index, or else run the Lucene CheckIndex tool in
> "fix" mode, which renders the index operable again, but at the expense
> of throwing away some of the data. This happens rarely, and I haven't
> been able to diagnose it yet. In the meantime, though, I find it
> somewhat reassuring to know that my source data is in a SQL table.
> I don't know that this experience is relevant to you; my problem could
> come from a variety of sources outside Lucene, including a potential
> bug in Solr, and user error on my part. All the same, perhaps it would
> be worth searching the mailing list archives for FileNotFound, to see
> what else comes up?
> On Tue, Jan 19, 2010 at 7:58 PM, Guido Bartolucci
> <> wrote:
>> I know that the primary use case for Lucene is as an index of data
>> that can be reconstructed (e.g., from a relational database or from
>> spidering your corporate intranet).
>> But, I'm curious if anyone uses Lucene as their primary datastore for
>> their gold data. Is it good enough?
>> Would anyone consider (or do people already) store data in Lucene
>> that, if it was lost, would destroy their business? And no, I'm not
>> suggesting that you don't back up this data, I'm just curious if there
>> are problems with using Lucene in this way. Are there subtle
>> corruptions that might show up in Lucene that wouldn't show up in
>> Oracle or MySQL?
>> I'm considering using Lucene in this way but I haven't been able to
>> find any documentation describing this use case. Are there any studies
>> of Lucene vs MySQL running for N years comparing the corruptions and
>> recovery times?
>> Am I just ignorant and scared of Lucene and too trusting of Oracle and MySQL?
>> Thanks.
>> -guido.
>> (BTW, I did find a similar question asked back in 2007 in the archives
>> but it doesn't really answer my question)
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

Kind regards,
Jacob Rhoden

Information Technology Services,
The University of Melbourne

Phone: +61 3 8344 2884
Mobile: +61 4 1095 7575

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message