lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ganesh" <emailg...@yahoo.co.in>
Subject Re: Multiple index vs Single Index
Date Thu, 23 Feb 2012 11:00:12 GMT
Thanks. 

The reason i have gone for multiple index is, I have more updates for current date and deletes
in the older date. Now i am planning to use Single index. I think i should use forceMergeDelete
to merge the deletes. 

Do you optimize your index? How you handle millions of docs in the index.

Regards
Ganesh

----- Original Message ----- 
From: "Ian Lea" <ian.lea@gmail.com>
To: <java-user@lucene.apache.org>
Sent: Thursday, February 23, 2012 3:40 PM
Subject: Re: Multiple index vs Single Index


Millions of docs in a single index is definitely OK.  If it was my
system I'd willingly trade slightly slower indexing for simplicity and
ease of use.  If it works and is fast enough, job done.


--
Ian.


On Thu, Feb 23, 2012 at 7:31 AM, Ganesh <emailgane@yahoo.co.in> wrote:
> Hello all,
>
> This debate we might have had more frequently in the group. Yet one more time, i want
to clarify.
>
> I was using multiple indexes (per week one index) with previous versions of Lucene (2.4
- 3.0.3). The performance was really good for incremental indexing. I used to optimize once
per day. The only disadvantage is, when data get piled up to 1 year and when i try to reindex,
it is throwing too many open files exception.
>
> I have seen couple of comments in the group to use Single Index / per machine and not
use multiple index. The suggesstion is, not to use optimize for large index. This works fine.
I am able to index more than 58 million records. The search performance is really good but
index performance is little slow then in my previous model.
>
> Previously, I have choosen multiple indexes model beacause i need to do optimize and
it was failing for the large index. I thought to break in to multiple small index with same
schema. Now optimization is discourged and the name has been changed to forceMerge to discourge
its use.
>
> I just want to confirm, Is Single Index would be good for millions of records?
>
> Regards
> Ganesh
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message