helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kishore g <g.kish...@gmail.com>
Subject Re: Specifying Bucket size when adding resource with IDEAL_STATE
Date Sat, 22 Nov 2014 21:34:38 GMT
We should add some docs for large number of partitions.
Varun, you might also want to enable group message feature to for faster
state transitions.
On Nov 22, 2014 1:28 PM, "Varun Sharma" <varun@pinterest.com> wrote:

> I got hold of the EXTERNALVIEW znode for 500 partitions and it worked out
> to 80kB. So, I am using 1 bucket per 1000 partitions ~ 160Kb. There are a
> handful of cases where we need upto 4K partitions - hence I wanna use 4
> buckets for that usecase.
>
> On Sat, Nov 22, 2014 at 12:59 PM, Zhen Zhang <nehzgnahz@gmail.com> wrote:
>
>> Hi Varun,
>>
>> You can refer to this example:
>>
>> https://github.com/apache/helix/blob/helix-0.6.x/helix-core/src/test/java/org/apache/helix/integration/TestBucketizedResource.java
>>
>> The accessor handles bucket so HelixAdmin#setResourceIdealState should do
>> the work also.
>> May I know how many partitions you have and how large is your ideal state?
>>
>> Thanks,
>> Zhen
>>
>>
>> On Sat, Nov 22, 2014 at 11:53 AM, kishore g <g.kishore@gmail.com> wrote:
>>
>>> I dint realize that you had buckets. Let me check again
>>>
>>> On Sat, Nov 22, 2014 at 11:52 AM, kishore g <g.kishore@gmail.com> wrote:
>>>
>>>>
>>>> http://helix.apache.org/javadocs/0.6.3/reference/org/apache/helix/HelixAdmin.html
>>>>  public abstract void setResourceIdealState (String
>>>> <http://download.oracle.com/javase/6/docs/api/index.html?java/lang/String.html>
>>>> clusterName, String
>>>> <http://download.oracle.com/javase/6/docs/api/index.html?java/lang/String.html>
>>>> resourceName, IdealState
>>>> <http://helix.apache.org/javadocs/0.6.3/reference/org/apache/helix/model/IdealState.html>
>>>> idealState)
>>>>
>>>> Set ideal state for a resource.
>>>>
>>>> That should work.
>>>>
>>>>
>>>> On Sat, Nov 22, 2014 at 11:26 AM, Varun Sharma <varun@pinterest.com>
>>>> wrote:
>>>>
>>>>> How do I add a resource with an ideal state and buckets since my
>>>>> resource has extremely large number of partitions ? I can't find an
>>>>> addResource call in HelixAdmin to do the same ?
>>>>>
>>>>> Thanks
>>>>> Varun
>>>>>
>>>>> On Fri, Nov 21, 2014 at 8:07 PM, Kanak Biscuitwala <
>>>>> kanak.b@hotmail.com> wrote:
>>>>>
>>>>>> Hi Varun,
>>>>>>
>>>>>> Adding a resource is equivalent to adding an ideal state. So if you
>>>>>> just create your own initial ideal state and set it, you've effectively
>>>>>> created a resource with the properties that you want.
>>>>>>
>>>>>> Kanak
>>>>>>
>>>>>> ________________________________
>>>>>> > Date: Fri, 21 Nov 2014 17:08:36 -0800
>>>>>> > Subject: Re: Specifying Bucket size when adding resource with
>>>>>> IDEAL_STATE
>>>>>> > From: varun@pinterest.com
>>>>>> > To: user@helix.apache.org
>>>>>> >
>>>>>> > Or should I break this up into 2 calls - one to add the resource
and
>>>>>> > then another to set the ideal states ?
>>>>>> >
>>>>>> > On Fri, Nov 21, 2014 at 5:03 PM, Varun Sharma
>>>>>> > <varun@pinterest.com<mailto:varun@pinterest.com>>
wrote:
>>>>>> > Looking at the HelixAdmin API - I do not see an API for adding
a
>>>>>> > resource with a particular IdealState and the number of buckets
to
>>>>>> use
>>>>>> > (znodes).
>>>>>> >
>>>>>> > Thanks
>>>>>> > Varun
>>>>>> >
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message