helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kishore g <g.kish...@gmail.com>
Subject Re: Specifying partition placement using AUTO
Date Tue, 19 Feb 2013 16:31:56 GMT
Glad it worked for you and thanks for the suggestions. MasterSlave,
LeaderStandby, OnlineOffline does come by default. If you create the
cluster using the command line tool, then those state models are created by
default. But if you use helix admin api, it probably does not. There should
probably be an api to add default state
models.

You are right, you can use JSON file option to upload for any mode. Will
update the docs accordingly.


Thanks,
Kishore G



On Mon, Feb 18, 2013 at 8:58 PM, Ming Fang <mingfang@mac.com> wrote:

> Thanks Kishore's help I was able to get this working.
>
> I just wanted to point out a couple of things in hopes of helping others.
>
> 1-I had to add the MasterSlave model def like this
>       admin.addStateModelDef(CLUSTER_NAME, "MasterSlave", new
> StateModelDefinition(
>               new
> StateModelConfigGenerator().generateConfigForMasterSlave()));
>
> At first I thought MasterSlave was a special model that was already
> registered by default.  Maybe it should be.
>
> 2-I was able to use AUTO mode with a JSON file.  The documentation
> suggests that this was only possible with CUSTOMIZED mode, but it appears
> to work for AUTO too.
> I prefer using a JSON file to create the ideal state.
>
> Finally thanks to the Helix team for giving us an excellent product.
>
> On Feb 14, 2013, at 12:14 PM, kishore g <g.kishore@gmail.com> wrote:
>
> Hi Ming,
>
> There is an example
> https://git-wip-us.apache.org/repos/asf?p=incubator-helix.git;a=blob;f=helix-core/src/main/java/org/apache/helix/examples/IdealStateExample.java;hb=437eb42e
> .
>
> admin.addResource(clusterName, resourceName, numPartitions, stateModelName, "AUTO");
> // rebalance resource "TestDB" using 3 replicas
> admin.rebalance(clusterName, resourceName, numReplica);
>
> This will automatically do the partition placement among the nodes in the
> cluster. We use a mix of consistent hashing and crush
> http://www.ssrc.ucsc.edu/Papers/weil-sc06.pdf to do the placement.
>
> But i think you need a way to come up with your own logic of assigning the
> replica set for a partition. You can find an example for that in
> BaseStageTest. See setupIdealState
> https://git-wip-us.apache.org/repos/asf?p=incubator-helix.git;a=blob;f=helix-core/src/test/java/org/apache/helix/controller/stages/BaseStageTest.java;h=ed54e6facdc17b40bd8eeb56c4cfaa5ad39ea91b;hb=437eb42e
>
> Here is the code snippet.http://pastebin.com/e2LXtq46.
>
> Hope this helps
>
> Thanks,
> Kishore G
>
>
>
>
>
>
>
> On Thu, Feb 14, 2013 at 8:03 AM, Ming Fang <mingfang@mac.com> wrote:
>
>> Can someone point me to an example of how to specify partition placement
>> when using AUTO mode?
>> The link here http://helix.incubator.apache.org/Features.html says it's
>> possible but I was not able to find any example or unit test that does this.
>> Thanks
>> --ming
>
>
>
>

Mime
View raw message