Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
MIME-Version: 1.0
Date: Wed, 23 Sep 2015 15:08:16 +0100
Message-ID: 
 <CAB-fSbzFhb9gOgZAHtOo4zY182Hh4cwYuFx+cBGxnkyO6LH0sA@mail.gmail.com>
Subject: Risk of Over Sharding
From: Alessandro Benedetti <benedetti.alex85@gmail.com>
To: "solr-user@lucene.apache.org" <solr-user@lucene.apache.org>
Content-Type: multipart/alternative; boundary=001a113d17b863cac105206aa5b8

--001a113d17b863cac105206aa5b8
Content-Type: text/plain; charset=UTF-8

It's quite common to hear about the benefit of sharding.
Until we reach the I/O bound on our machines, sharding is likely to reduce
the query time.
Furthermore working on smaller indexes will make the single searches faster
on the smaller nodes.
But what about the other way around ?
What if we actually shard much more than needed ?
Are we going to see also an increase in the query time ( due to the
overhead of query distribution and aggregation of results ? )

Example : 100.000.000 docs across 16 shards .
Wouldn't be more effective to have 4 or 8 shards maximum ?
I suspect that prototyping is the right answer, but do we have any general
suggestions strongly motivated  ?
Any resource to study ?

Cheers

-- 
--------------------------

Benedetti Alessandro
Visiting card - http://about.me/alessandro_benedetti
Blog - http://alexbenedetti.blogspot.co.uk

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

--001a113d17b863cac105206aa5b8--