helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kishore g <g.kish...@gmail.com>
Subject Re: Using partitions in Helix
Date Tue, 16 Apr 2013 17:04:19 GMT
Hi Ivan,

Thanks for the question. You can chose one of the two.

1. Use HelixAdmin and create your own mapping of partition to key range
Basically you will have 5 partitions { resource_0, resource_1, resource_2,
resource_3, resource_4}, then you can have config for each partition which
tells the key range it is associated with.
{resource_0=k1start} {resource_1=k2start}.......

This works fine initially but when you further split your key ranges the
partition numbering becomes confusing and difficult to map.

2. Set your own idealstate

In Helix partition is just a logical name. So you can always generate an
idealstate with partition names as follows.

{resource_k1start, resource_k2start, resource_k3start} or even
{resource_k1start-k1end}.. to make the mapping explicit.

This is easier to understand and reason about. External view then becomes
the routing table.

Currently Helix does not understand the concept of a partition being split,
you will have to program it yourself by changing the idealstate during
splits. We have a jira for this and we are working on the initial design.
Your inputs will be valuable. I will try to add more details to the JIRA.
This is an important feature and we plan to support it as a first class
citizen in Helix so that one can benefit from other features like
throttling ( for example, one could say that dont do more than X splits
across cluster, node etc)


Kishore G

On Tue, Apr 16, 2013 at 6:36 AM, Ivan Kelly <ivank@apache.org> wrote:

> Hi,
> I'm sketching out the design of a prototype key value store which I'd
> like to use helix in. However, I'm unclear of how to use partitions in
> helix.
> So, the number of partitions for a particular resource is defined when
> I call #addResource from helix admin. Lets say I want to split my key
> space into 5 partitions. How do I define what the boundaries for these
> partitions are? Do I have to create my own mapping of key range to
> partition? Or do I have to provide my own implementation of
> IdealState?
> Thanks
> -Ivan

View raw message