lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ganesh" <>
Subject Re: Lucene as a primary datastore
Date Wed, 20 Jan 2010 06:13:21 GMT
We have data in compound files and we use Lucene as primary database. Its working great and
much faster with millions of records. The only issue, I face is with sorting. Lucene sorting
consumes good amount of memory. I don't know much about the MySQL/PostgreSQL database, and
how they behave with millions of records but i guess their sorting memory consumption would
be less.  

It would be great, If Lucene has the ability to do backups / replication. I don't know how
to modify/use the solr script.  


----- Original Message ----- 
From: "Otis Gospodnetic" <>
To: <>; <>
Sent: Wednesday, January 20, 2010 10:45 AM
Subject: Re: Lucene as a primary datastore

> You are not alone, Guido.  It's a good question.  In my experience, Lucene is as stable
as MySQL/PostgreSQL in terms of its ability to hold your data and not corrupt it.  Of course,
even with the most expensive databases, you'd want to make backups.  The same goes with Lucene.
 Nowadays, one way people make "backups" is via replication. :)  Solr users thus often get
backups for free, as do people who put copies of their data on file systems like HDFS, which
tend to have replication turned on.
> Otis
> --
> Sematext -- -- Solr - Lucene - Nutch
> ----- Original Message ----
>> From: Guido Bartolucci <>
>> To:
>> Sent: Tue, January 19, 2010 10:58:36 PM
>> Subject: Lucene as a primary datastore
>> I know that the primary use case for Lucene is as an index of data
>> that can be reconstructed (e.g., from a relational database or from
>> spidering your corporate intranet).
>> But, I'm curious if anyone uses Lucene as their primary datastore for
>> their gold data. Is it good enough?
>> Would anyone consider (or do people already) store data in Lucene
>> that, if it was lost, would destroy their business? And no, I'm not
>> suggesting that you don't back up this data, I'm just curious if there
>> are problems with using Lucene in this way. Are there subtle
>> corruptions that might show up in Lucene that wouldn't show up in
>> Oracle or MySQL?
>> I'm considering using Lucene in this way but I haven't been able to
>> find any documentation describing this use case. Are there any studies
>> of Lucene vs MySQL running for N years comparing the corruptions and
>> recovery times?
>> Am I just ignorant and scared of Lucene and too trusting of Oracle and MySQL?
>> Thanks.
>> -guido.
>> (BTW, I did find a similar question asked back in 2007 in the archives
>> but it doesn't really answer my question)
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:
Send instant messages to your online friends 

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message