helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Varun Sharma <va...@pinterest.com>
Subject Re: Bucketized resources not working in Helix 0.6.4
Date Mon, 23 Feb 2015 19:48:58 GMT
Note that we had to prefix each partition with the resource name to make
the partitions unique.

On Mon, Feb 23, 2015 at 11:48 AM, Varun Sharma <varun@pinterest.com> wrote:

> Here is the latest error I am getting:
>
> 2015-02-23 19:44:48,153 [controller-pool-0]
> (TerrapinControllerServiceImpl.java:129) WARN  Resource creation failed for
> $terrapin$data$pinpin_board_join$1424377884722, rolling back.
>
> *java.lang.IllegalArgumentException: Could NOT parse partition#
> (join$1424377884722$0) in $terrapin$data$pinpin_board_join$1424377884722$0*
>
> * at
> org.apache.helix.ZNRecordBucketizer.getBucketName(ZNRecordBucketizer.java:72)*
>
> * at
> org.apache.helix.ZNRecordBucketizer.bucketize(ZNRecordBucketizer.java:105)*
>
> at
> org.apache.helix.manager.zk.ZKHelixDataAccessor.setProperty(ZKHelixDataAccessor.java:110)
>
> at
> org.apache.helix.manager.zk.ZKHelixAdmin.setResourceIdealState(ZKHelixAdmin.java:686)
>
> On Mon, Feb 23, 2015 at 10:40 AM, Zhen Zhang <zzhang@linkedin.com> wrote:
>
>>  Not really. The bucketize logic just creates child znode under the
>> original ideal state, and each of the child znode holds a subset of the
>> total partitions.
>>  ------------------------------
>> *From:* Varun Sharma [varun@pinterest.com]
>> *Sent:* Monday, February 23, 2015 10:37 AM
>>
>> *To:* user@helix.apache.org
>> *Subject:* Re: Bucketized resources not working in Helix 0.6.4
>>
>>   One other question - would bucketizing add anything to the partition
>> name - just worried about any special symbols (we use _ and $) causing
>> issues there ?
>>
>> On Mon, Feb 23, 2015 at 10:35 AM, Zhen Zhang <zzhang@linkedin.com> wrote:
>>
>>>  Hi Varun,
>>>
>>>  HelixAdmin#addResource was initially used for adding an empty resource
>>> only, followed by HelixAdmin#rebalance, which bucketize the ideal state, so
>>> the bucketize logic is missing from addResource. We should definitely fix
>>> this. We are also considering merging addResource and rebalance, since
>>> separating them are confusing. For now you may use setResourceIdealState as
>>> a workaround.
>>>
>>>  Thanks,
>>> Zhen
>>>
>>>  ------------------------------
>>> *From:* Varun Sharma [varun@pinterest.com]
>>> *Sent:* Monday, February 23, 2015 10:20 AM
>>> *To:* user@helix.apache.org
>>> *Subject:* Re: Bucketized resources not working in Helix 0.6.4
>>>
>>>    Thanks, Zhen.
>>>
>>>  What about just calling addResource with an idealState with the bucket
>>> size - is that expected to not work on its own ?
>>>
>>>  Varun
>>>
>>> On Mon, Feb 23, 2015 at 10:18 AM, Zhen Zhang <zzhang@linkedin.com>
>>> wrote:
>>>
>>>> Hi Varun,
>>>>
>>>> Did you do the following?
>>>> HelixAdmin#addResource() // set bucket size
>>>> HelixAdmin#setResourceIdealState() // no bucket size
>>>>
>>>> When you set the ideal state, you still need set the bucket size in the
>>>> ideal state. This value will be used to bucketize the ideal state.
>>>>
>>>> Thanks,
>>>> Jason
>>>>
>>>> ________________________________________
>>>> From: Varun Sharma [varun@pinterest.com]
>>>> Sent: Monday, February 23, 2015 1:51 AM
>>>> To: user@helix.apache.org
>>>> Subject: Bucketized resources not working in Helix 0.6.4
>>>>
>>>> Hi,
>>>>
>>>> I am trying to bucketize a resource on Helix 0.6.4 - however, am unable
>>>> to do so. I am setting the bucketSize in the idealState and using
>>>> addResource() and setResourceIdealState and it ends up complaining that ZK
>>>> size of 1M has been exceeded.
>>>>
>>>> I also tried using the addResource() api which creates an empty ideal
>>>> state with a valid bucket size. However, when I tried to set the ideal
>>>> state on that bucket (this ideal state did not have the bucket size set),
>>>> it failed as well.
>>>>
>>>> Whats the right way to get bucketizing to work ?
>>>>
>>>> Thanks !
>>>> Varun
>>>>
>>>
>>>
>>
>

Mime
View raw message