lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shalin Shekhar Mangar <shalinman...@gmail.com>
Subject Re: Autoscaling and inactive shards
Date Tue, 12 Jun 2018 06:10:49 GMT
Hi Jan,

Comments inline:

On Tue, Jun 12, 2018 at 2:19 AM Jan Høydahl <jan.asf@cominvent.com> wrote:

> Hi
>
> I'm trying to have Autoscaling move a shard to another node after manually
> splitting.
> We have two nodes, one has a shard1 and the other node is empty.
>
> After SPLITSHARD you have
>
> * shard1 (inactive)
> * shard1_0
> * shard1_1
>
> For autoscaling we have the {"minimize" : "cores"} cluster preference
> active. Because of that I'd expect that Autoscaling would suggest to move
> e.g. shard1_1 to the other (empty) node, but it doesn't. Then I create a
> rule just to test {"cores": "<2", "node": "#ANY"}, but still no
> suggestions. Not until I delete the inactive shard1, then it suggests to
> move one of the two remaining shards to the other node.
>
> So my two questions are
> 1. Is it by design that inactive shards "count" wrt #cores?
>    I understand that it consumes disk but it is not active otherwise,
>    so one could argue that it should not be counted in core/replica rules?
>

Today, inactive slices also count towards the number of cores -- though
technically correct, it is probably an oversight.


> 2. Why is there no suggestion to move a shard due to the "minimize cores"
> reference itself?
>

The /autoscaling/suggestions end point only suggests if there are policy
violations. Preferences such as minimize:cores are more of a sorting order
so they aren't really being violated. After you add the rule, the framework
still cannot give a suggestion that satisfies your rule. This is because
even if shard1_1 is moved to node2, node1 still has shard1 and shard1_0. So
the system ends up not suggesting anything. You should get a suggestion if
you add a third node to the cluster though.

Also see SOLR-11997 <https://issues.apache.org/jira/browse/SOLR-11997> which
will tell users that a suggestion could not be returned because we cannot
satisfy the policy. There are a slew of other improvements to suggestions
planned that will return suggestions even when there are no policy
violations.


>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
>

-- 
Regards,
Shalin Shekhar Mangar.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message