helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhen Zhang <zzh...@linkedin.com>
Subject Re: Composite rebalancer
Date Sat, 18 Jan 2014 19:35:33 GMT
Agree. Rebalancer isolation seems necessary for multi-tenancy clusters. We could probably tag
rebalancers with resource names, and for example, if the pipeline is triggered by resource-A's
ideal-state change, there is no need to invoke rebalancers not related to resource-A?

From: kishore g <g.kishore@gmail.com<mailto:g.kishore@gmail.com>>
Reply-To: "user@helix.apache.org<mailto:user@helix.apache.org>" <user@helix.apache.org<mailto:user@helix.apache.org>>
Date: Saturday, January 18, 2014 11:21 AM
To: "dev@helix.apache.org<mailto:dev@helix.apache.org>" <dev@helix.apache.org<mailto:dev@helix.apache.org>>,
"user@helix.apache.org<mailto:user@helix.apache.org>" <user@helix.apache.org<mailto:user@helix.apache.org>>
Subject: Composite rebalancer

Currently in each stage we iterate over each resource and compute the idealstate, the calculation
of one idealstate is independent of other resource. Even though nothing is stopping a rebalancer
for reasource A to read the computed idealstate of another resource B its not implicit.

I think it will be a good idea to have concept of composite rebalancer.

For example the rebalancer interface that computes the idealstate looks like this currently.
  public ResourceAssignment computeResourceMapping(RebalancerConfig rebalancerConfig,
      ResourceAssignment prevAssignment, Cluster cluster, ResourceCurrentState currentState);

we can add a compositerebalancer like
  public Map<ResourceId,ResourceAssignment> computeResourceMapping(RebalancerConfig
      Map<ResourceId,ResourceAssignment> prevAssignments, Cluster cluster, Map<ResourceId,ResourceCurrentState>

In the pipeline, if we see the rebalancer type is compositerebalancer then we can invoke computeResourceMapping
once for all resources that are configured to use this specific rebalancer.


Kishore G

View raw message