lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ganesh" <>
Subject Re: Single searcher vs Multi Searcher
Date Mon, 06 Oct 2008 04:36:47 GMT
Hello Anshum,

My index is growing 1 million documents per day. Initially i planned to have 
a single database but the sorting of one or more fields consumes more RAM. 
Whether sharding the index would also consume the same.

My application should co-exist with other application of my product and my 
app could get 1 GB of RAM. Search speed is fine but i need to display the 
result in the sorted order.

I thought to keep 7 days of documents in one index and create one more after 
the 7 days. After 30 days the first index may get deleted. I need to keep 
the documents in the index DB for 30 days. My Index DB is in HDD.

I want to the pros and cons of sharding. I think maintance of the DB becomes 

It would be very much helpful, if you share some of your thoughts.


----- Original Message ----- 
From: "Anshum" <>
To: <>
Sent: Friday, October 03, 2008 9:48 PM
Subject: Re: Single searcher vs Multi Searcher

> Hi Ganesh,
> I have experimented with sharded indexes and they seem to benefit 
> me(atleast
> in my case). I would like to know a few things before I answer your
> question:
> 1. Do you have a reasonable criteria ( a calculated one) to shard the
> indexes?
> 2. How do you plan to split the index? Is it going to be document based
> (which I guess it should be as otherwise you would have to build a 
> complete
> distributed system)
> 3. Do you plan to put your indexes on the RAM or on (physically) seperate
> HDDs?
> Though all said and done, sharded indexes are a good approach, if done the
> right way.
> --
> Anshum Gupta
> Naukri Labs!
> The facts expressed here belong to everybody, the opinions to me. The
> distinction is yours to draw............
> On Fri, Oct 3, 2008 at 3:01 PM, Ganesh <> wrote:
>> Hello all,
>> My indexing is growing by 1 million records per day and the memory
>> consumption of the searcher object is quite high.
>> There are different opinion in the groups. Few suggest to use single
>> database and few to use sharding. My Database has 10 million records now 
>> and
>> it might go till 30 million or more. I plan to shard the index. but
>> Multisearcher will give me benifit.
>> Regards
>> Ganesh
>> Send instant messages to your online friends 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:

Send instant messages to your online friends 

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message