lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Susheel Kumar <susheel2...@gmail.com>
Subject Re: Shard vs Replica
Date Wed, 06 Jul 2016 22:30:27 GMT
To understand shard & replica, let's first understand what is sharding and
why it is needed.

Sharding -  Assume your index grows large that it doesn't fit into a single
machine (for e.g. your index size is 80GB and your machine is 64GB in which
case index won't fit into memory).  Now to get better performance either
you increase your RAM or have another machine with similar configuration
and you divide (aka partitioning) your index into two and have each 40GB
index fit into two machines.    So your complete index = Index1 + Index2
(each 40GB).  These index1 and index2 etc. are called shards.  and
depending on how big is your index and machines resources, you may plan to
have N shards.  To do a complete search, the search has to be performed on
all the shards.

 Hope that clarifies and explains why sharding and what is shard.

Replication - Assume one of the above machine goes down and now you won't
be able to search on complete index since half of the copy/data is not
available. To avoid this single point of failure, you can create replica
(which is copy of shard) on either existing machines (or have another new
machines depending on requirements)

Machine 1 = Shard1 +  Copy of Shard2 (Shard2_Replica1)
Machine 2 = Shard2 +  Copy of Shard1 (Shard1_Replica1)

So you create replica to avoid single point of failure and also to serve
higher queries per second (in case if replica gets created on another
machines)

Hope that clarifies.

Thanks,
Susheel



On Wed, Jul 6, 2016 at 5:32 PM, John Doe <mailinglists001@gmail.com> wrote:

> Hey,
>
> I have have the same question on freenode channel , people answered me ,
> but I believe that I still got doubts. Just because I never had approach to
> such data store technologies before it makes me hardly understand what is
> exactly is replica and shard in solr. I believe once I understand what
> exactly are these two, then I would be able to see the difference.
>
> According to English dictionary replica is exact copy of something, which
> sounds like a true to me, but what is shard then here and how is it
> connected with all this context ? Can someone explain this in brief and
> give some examples ?
>
> Thank you in advance
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message