lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sethi, Parampreet" <parampreet.se...@teamaol.com>
Subject Re: Doubt Regarding Shards Index
Date Tue, 03 Jan 2012 20:38:57 GMT
Hi Suneel,

I have implemented Solr sharding in one of my projects where data was of
the order of 1 billion documents and my queries were throwing Out of
memory exception because of huge index. Here are my views:

- Have identical Solr server setups for each shard with same schema.

1. We need to create handler for shards in solrconfig.xml ?
- In my case, I did not add any handlers in solrconfig.xml for sharding.

2. Index will be different for each shards instance means we need to break
data in part to create index for each instance or index will be same?
- yes, Index needs to be broken into each shard instance. I used
creation_date field in my case to divide the data by years into each shard
(for example all documents with year 2007 will go to shard 1 and 2008 will
go to shard 2 and so on) and similarly while writing the data look for the
same field and index the corresponding shard.

3. How i will recognize which instance return the result ?
- Once you know how data is divided, you can easily figure out which shard
is serving. 

I have put some of my analysis on this blog post
http://www.params.me/2010/04/working-with-solr.html. Hope it helps!

Best,
Param
http://params.me



On 1/3/12 2:02 PM, "Suneel" <pandey.suneel@gmail.com> wrote:

>I am using solr My index becomes too large I want to implement shards
>concept
>but i have some doubt. i searched a lot but not found satisfied result.
>
>1. We need to create handler for shards in solrconfig.xml ?
>
>2. Index will be different for each shards instance means we need to break
>data in part to create index for each instance or index will be same?
>
>3. How i will recognize which instance return the result ?
>
>
>Please provide me above details this will be very helpful for me.
>
>Thanks & Regards
>Suneel Pandey 
>
>--
>View this message in context:
>http://lucene.472066.n3.nabble.com/Doubt-Regarding-Shards-Index-tp3629964p
>3629964.html
>Sent from the Solr - User mailing list archive at Nabble.com.


Mime
View raw message