helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kishore g <g.kish...@gmail.com>
Subject Re: Resource Buckets
Date Wed, 03 Apr 2013 14:23:35 GMT
Hi Vinayak,

It useful when there you have large number of partitions. It is used to
split the znodes in zookeeper. Helix by default uses one znode to represent
all partitions in IdealState,CurrentState, ExternalView. This reduces the
number of watches and also allows us to improve performance. How ever
zookeeper has a size limit of 1mb per znode and when we have large number
of partitions we can split the Idealstate,CurrentState, ExternalView into
multiple znodes.

For example if you have 10k partitions, you can set bucket size to 100 and
we create 100 znodes each with 0-99,100-199 etc

Thanks
Kishore G


On Wed, Apr 3, 2013 at 12:47 AM, Vinayak Borkar <vborky@yahoo.com> wrote:

> Hi,
>
>
> The addResource call has a variant that lets users set a bucket size. What
> is a bucket? When would I use this feature? I could not find any mention of
> this other than in the Javadoc of that method that says "... where bucket
> size > 1".
>
>
>
> Thanks,
> Vinayak
>

Mime
View raw message