helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Varun Sharma <va...@pinterest.com>
Subject Re: Specifying Bucket size when adding resource with IDEAL_STATE
Date Sat, 22 Nov 2014 21:26:22 GMT
I got hold of the EXTERNALVIEW znode for 500 partitions and it worked out
to 80kB. So, I am using 1 bucket per 1000 partitions ~ 160Kb. There are a
handful of cases where we need upto 4K partitions - hence I wanna use 4
buckets for that usecase.

On Sat, Nov 22, 2014 at 12:59 PM, Zhen Zhang <nehzgnahz@gmail.com> wrote:

> Hi Varun,
>
> You can refer to this example:
>
> https://github.com/apache/helix/blob/helix-0.6.x/helix-core/src/test/java/org/apache/helix/integration/TestBucketizedResource.java
>
> The accessor handles bucket so HelixAdmin#setResourceIdealState should do
> the work also.
> May I know how many partitions you have and how large is your ideal state?
>
> Thanks,
> Zhen
>
>
> On Sat, Nov 22, 2014 at 11:53 AM, kishore g <g.kishore@gmail.com> wrote:
>
>> I dint realize that you had buckets. Let me check again
>>
>> On Sat, Nov 22, 2014 at 11:52 AM, kishore g <g.kishore@gmail.com> wrote:
>>
>>>
>>> http://helix.apache.org/javadocs/0.6.3/reference/org/apache/helix/HelixAdmin.html
>>>  public abstract void setResourceIdealState (String
>>> <http://download.oracle.com/javase/6/docs/api/index.html?java/lang/String.html>
>>> clusterName, String
>>> <http://download.oracle.com/javase/6/docs/api/index.html?java/lang/String.html>
>>> resourceName, IdealState
>>> <http://helix.apache.org/javadocs/0.6.3/reference/org/apache/helix/model/IdealState.html>
>>> idealState)
>>>
>>> Set ideal state for a resource.
>>>
>>> That should work.
>>>
>>>
>>> On Sat, Nov 22, 2014 at 11:26 AM, Varun Sharma <varun@pinterest.com>
>>> wrote:
>>>
>>>> How do I add a resource with an ideal state and buckets since my
>>>> resource has extremely large number of partitions ? I can't find an
>>>> addResource call in HelixAdmin to do the same ?
>>>>
>>>> Thanks
>>>> Varun
>>>>
>>>> On Fri, Nov 21, 2014 at 8:07 PM, Kanak Biscuitwala <kanak.b@hotmail.com
>>>> > wrote:
>>>>
>>>>> Hi Varun,
>>>>>
>>>>> Adding a resource is equivalent to adding an ideal state. So if you
>>>>> just create your own initial ideal state and set it, you've effectively
>>>>> created a resource with the properties that you want.
>>>>>
>>>>> Kanak
>>>>>
>>>>> ________________________________
>>>>> > Date: Fri, 21 Nov 2014 17:08:36 -0800
>>>>> > Subject: Re: Specifying Bucket size when adding resource with
>>>>> IDEAL_STATE
>>>>> > From: varun@pinterest.com
>>>>> > To: user@helix.apache.org
>>>>> >
>>>>> > Or should I break this up into 2 calls - one to add the resource
and
>>>>> > then another to set the ideal states ?
>>>>> >
>>>>> > On Fri, Nov 21, 2014 at 5:03 PM, Varun Sharma
>>>>> > <varun@pinterest.com<mailto:varun@pinterest.com>> wrote:
>>>>> > Looking at the HelixAdmin API - I do not see an API for adding a
>>>>> > resource with a particular IdealState and the number of buckets
to
>>>>> use
>>>>> > (znodes).
>>>>> >
>>>>> > Thanks
>>>>> > Varun
>>>>> >
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message