lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: Solr 4.1 Solr Cloud Shard Structure
Date Fri, 01 Mar 2013 01:07:36 GMT

On Feb 28, 2013, at 7:55 PM, Walter Underwood <wunder@wunderwood.org> wrote:

> 100 shards on a node will almost certainly be slow

I think it depends on some things - with one of the largest of those things being your hardware.
Many have found that you can get much better performance out of super concurrent, beefy hardware
using more cores on a single node. So there will be some give and take that are tough jump
to conclusions about. Slower at 100, I would assume yes, slow, depends.

One thing that will happen is that you will require a lot more threads…

You would want some pretty beefy hardware.

But you don't have to do 100 either. That should just be a rough starting number. At some
point you have to reindex into a new cluster if you keep growing. Or consider shard splitting
if its feasible (and becomes available). You can only over shard so much.

So perhaps you do 50 or whatever. It will be faster than you think I imagine. My main concern
is the number of threads - might want to mess with Xss to minimize their ram usage at least.

- Mark
Mime
View raw message