helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Varun Sharma <va...@pinterest.com>
Subject Re: Specifying Bucket size when adding resource with IDEAL_STATE
Date Mon, 24 Nov 2014 17:28:31 GMT
How can I enable the group message feature ?

Thanks
Varun

On Sat, Nov 22, 2014 at 1:34 PM, kishore g <g.kishore@gmail.com> wrote:

> We should add some docs for large number of partitions.
> Varun, you might also want to enable group message feature to for faster
> state transitions.
> On Nov 22, 2014 1:28 PM, "Varun Sharma" <varun@pinterest.com> wrote:
>
>> I got hold of the EXTERNALVIEW znode for 500 partitions and it worked out
>> to 80kB. So, I am using 1 bucket per 1000 partitions ~ 160Kb. There are a
>> handful of cases where we need upto 4K partitions - hence I wanna use 4
>> buckets for that usecase.
>>
>> On Sat, Nov 22, 2014 at 12:59 PM, Zhen Zhang <nehzgnahz@gmail.com> wrote:
>>
>>> Hi Varun,
>>>
>>> You can refer to this example:
>>>
>>> https://github.com/apache/helix/blob/helix-0.6.x/helix-core/src/test/java/org/apache/helix/integration/TestBucketizedResource.java
>>>
>>> The accessor handles bucket so HelixAdmin#setResourceIdealState should
>>> do the work also.
>>> May I know how many partitions you have and how large is your ideal
>>> state?
>>>
>>> Thanks,
>>> Zhen
>>>
>>>
>>> On Sat, Nov 22, 2014 at 11:53 AM, kishore g <g.kishore@gmail.com> wrote:
>>>
>>>> I dint realize that you had buckets. Let me check again
>>>>
>>>> On Sat, Nov 22, 2014 at 11:52 AM, kishore g <g.kishore@gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>> http://helix.apache.org/javadocs/0.6.3/reference/org/apache/helix/HelixAdmin.html
>>>>>  public abstract void setResourceIdealState (String
>>>>> <http://download.oracle.com/javase/6/docs/api/index.html?java/lang/String.html>
>>>>> clusterName, String
>>>>> <http://download.oracle.com/javase/6/docs/api/index.html?java/lang/String.html>
>>>>> resourceName, IdealState
>>>>> <http://helix.apache.org/javadocs/0.6.3/reference/org/apache/helix/model/IdealState.html>
>>>>> idealState)
>>>>>
>>>>> Set ideal state for a resource.
>>>>>
>>>>> That should work.
>>>>>
>>>>>
>>>>> On Sat, Nov 22, 2014 at 11:26 AM, Varun Sharma <varun@pinterest.com>
>>>>> wrote:
>>>>>
>>>>>> How do I add a resource with an ideal state and buckets since my
>>>>>> resource has extremely large number of partitions ? I can't find
an
>>>>>> addResource call in HelixAdmin to do the same ?
>>>>>>
>>>>>> Thanks
>>>>>> Varun
>>>>>>
>>>>>> On Fri, Nov 21, 2014 at 8:07 PM, Kanak Biscuitwala <
>>>>>> kanak.b@hotmail.com> wrote:
>>>>>>
>>>>>>> Hi Varun,
>>>>>>>
>>>>>>> Adding a resource is equivalent to adding an ideal state. So
if you
>>>>>>> just create your own initial ideal state and set it, you've effectively
>>>>>>> created a resource with the properties that you want.
>>>>>>>
>>>>>>> Kanak
>>>>>>>
>>>>>>> ________________________________
>>>>>>> > Date: Fri, 21 Nov 2014 17:08:36 -0800
>>>>>>> > Subject: Re: Specifying Bucket size when adding resource
with
>>>>>>> IDEAL_STATE
>>>>>>> > From: varun@pinterest.com
>>>>>>> > To: user@helix.apache.org
>>>>>>> >
>>>>>>> > Or should I break this up into 2 calls - one to add the
resource
>>>>>>> and
>>>>>>> > then another to set the ideal states ?
>>>>>>> >
>>>>>>> > On Fri, Nov 21, 2014 at 5:03 PM, Varun Sharma
>>>>>>> > <varun@pinterest.com<mailto:varun@pinterest.com>>
wrote:
>>>>>>> > Looking at the HelixAdmin API - I do not see an API for
adding a
>>>>>>> > resource with a particular IdealState and the number of
buckets to
>>>>>>> use
>>>>>>> > (znodes).
>>>>>>> >
>>>>>>> > Thanks
>>>>>>> > Varun
>>>>>>> >
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>

Mime
View raw message