helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ming Fang <mingf...@mac.com>
Subject Re: Specifying partition placement using AUTO
Date Tue, 19 Feb 2013 04:58:17 GMT
Thanks Kishore's help I was able to get this working.

I just wanted to point out a couple of things in hopes of helping others.

1-I had to add the MasterSlave model def like this
      admin.addStateModelDef(CLUSTER_NAME, "MasterSlave", new StateModelDefinition(
              new StateModelConfigGenerator().generateConfigForMasterSlave()));

At first I thought MasterSlave was a special model that was already registered by default.
 Maybe it should be.

2-I was able to use AUTO mode with a JSON file.  The documentation suggests that this was
only possible with CUSTOMIZED mode, but it appears to work for AUTO too.
I prefer using a JSON file to create the ideal state.

Finally thanks to the Helix team for giving us an excellent product.

On Feb 14, 2013, at 12:14 PM, kishore g <g.kishore@gmail.com> wrote:

> Hi Ming,
> There is an example https://git-wip-us.apache.org/repos/asf?p=incubator-helix.git;a=blob;f=helix-core/src/main/java/org/apache/helix/examples/IdealStateExample.java;hb=437eb42e.

> admin.addResource(clusterName, resourceName, numPartitions, stateModelName, "AUTO");
> // rebalance resource "TestDB" using 3 replicas
> admin.rebalance(clusterName, resourceName, numReplica);
> This will automatically do the partition placement among the nodes in the cluster. We
use a mix of consistent hashing and crush http://www.ssrc.ucsc.edu/Papers/weil-sc06.pdf to
do the placement.
> But i think you need a way to come up with your own logic of assigning the replica set
for a partition. You can find an example for that in BaseStageTest. See setupIdealState https://git-wip-us.apache.org/repos/asf?p=incubator-helix.git;a=blob;f=helix-core/src/test/java/org/apache/helix/controller/stages/BaseStageTest.java;h=ed54e6facdc17b40bd8eeb56c4cfaa5ad39ea91b;hb=437eb42e
> Here is the code snippet.http://pastebin.com/e2LXtq46.
> Hope this helps
> Thanks,
> Kishore G
> On Thu, Feb 14, 2013 at 8:03 AM, Ming Fang <mingfang@mac.com> wrote:
> Can someone point me to an example of how to specify partition placement when using AUTO
> The link here http://helix.incubator.apache.org/Features.html says it's possible but
I was not able to find any example or unit test that does this.
> Thanks
> --ming

View raw message