lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manohar Sripada <manohar...@gmail.com>
Subject How to select the correct number of Shards in SolrCloud
Date Fri, 16 Jan 2015 05:58:59 GMT
Hi All,

My Setup is as follows. There are 16 nodes in my SolrCloud and 4 CPU cores
on each Solr Node VM. Each having 64 GB of RAM, out of which I have
allocated 32 GB to Solr. I have a collection which contains around 100
million Docs, which I created with 64 shards, replication factor 2, and 8
shards per node. Each shard is getting around 1.6 Million Documents.

The reason I have created 64 Shards is there are 4 CPU cores on each VM;
while querying I can make use of all the CPU cores. On an average, Solr
QTime is around 500ms here.

Last time to my other discussion, Erick suggested that I might be over
sharding, So, I tried reducing the number of shards to 32 and then 16. To
my surprise, it started performing better. It came down to 300 ms (for 32
shards) and 100 ms (for 16 shards). I haven't tested with filters and
facets yet here. But, the simple search queries had shown lot of
improvement.

So, how come the less number of shards performing better?? Is it because
there are less number of posting lists to search on OR less merges that are
happening? And how to determine the correct number of shards?

Thanks,
Manohar

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message