helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Varun Sharma <va...@pinterest.com>
Subject Re: Bucketized resources not working in Helix 0.6.4
Date Mon, 23 Feb 2015 19:48:31 GMT
Here is the latest error I am getting:

2015-02-23 19:44:48,153 [controller-pool-0]
(TerrapinControllerServiceImpl.java:129) WARN  Resource creation failed for
$terrapin$data$pinpin_board_join$1424377884722, rolling back.

*java.lang.IllegalArgumentException: Could NOT parse partition#
(join$1424377884722$0) in $terrapin$data$pinpin_board_join$1424377884722$0*

* at
org.apache.helix.ZNRecordBucketizer.getBucketName(ZNRecordBucketizer.java:72)*

* at
org.apache.helix.ZNRecordBucketizer.bucketize(ZNRecordBucketizer.java:105)*

at
org.apache.helix.manager.zk.ZKHelixDataAccessor.setProperty(ZKHelixDataAccessor.java:110)

at
org.apache.helix.manager.zk.ZKHelixAdmin.setResourceIdealState(ZKHelixAdmin.java:686)

On Mon, Feb 23, 2015 at 10:40 AM, Zhen Zhang <zzhang@linkedin.com> wrote:

>  Not really. The bucketize logic just creates child znode under the
> original ideal state, and each of the child znode holds a subset of the
> total partitions.
>  ------------------------------
> *From:* Varun Sharma [varun@pinterest.com]
> *Sent:* Monday, February 23, 2015 10:37 AM
>
> *To:* user@helix.apache.org
> *Subject:* Re: Bucketized resources not working in Helix 0.6.4
>
>   One other question - would bucketizing add anything to the partition
> name - just worried about any special symbols (we use _ and $) causing
> issues there ?
>
> On Mon, Feb 23, 2015 at 10:35 AM, Zhen Zhang <zzhang@linkedin.com> wrote:
>
>>  Hi Varun,
>>
>>  HelixAdmin#addResource was initially used for adding an empty resource
>> only, followed by HelixAdmin#rebalance, which bucketize the ideal state, so
>> the bucketize logic is missing from addResource. We should definitely fix
>> this. We are also considering merging addResource and rebalance, since
>> separating them are confusing. For now you may use setResourceIdealState as
>> a workaround.
>>
>>  Thanks,
>> Zhen
>>
>>  ------------------------------
>> *From:* Varun Sharma [varun@pinterest.com]
>> *Sent:* Monday, February 23, 2015 10:20 AM
>> *To:* user@helix.apache.org
>> *Subject:* Re: Bucketized resources not working in Helix 0.6.4
>>
>>    Thanks, Zhen.
>>
>>  What about just calling addResource with an idealState with the bucket
>> size - is that expected to not work on its own ?
>>
>>  Varun
>>
>> On Mon, Feb 23, 2015 at 10:18 AM, Zhen Zhang <zzhang@linkedin.com> wrote:
>>
>>> Hi Varun,
>>>
>>> Did you do the following?
>>> HelixAdmin#addResource() // set bucket size
>>> HelixAdmin#setResourceIdealState() // no bucket size
>>>
>>> When you set the ideal state, you still need set the bucket size in the
>>> ideal state. This value will be used to bucketize the ideal state.
>>>
>>> Thanks,
>>> Jason
>>>
>>> ________________________________________
>>> From: Varun Sharma [varun@pinterest.com]
>>> Sent: Monday, February 23, 2015 1:51 AM
>>> To: user@helix.apache.org
>>> Subject: Bucketized resources not working in Helix 0.6.4
>>>
>>> Hi,
>>>
>>> I am trying to bucketize a resource on Helix 0.6.4 - however, am unable
>>> to do so. I am setting the bucketSize in the idealState and using
>>> addResource() and setResourceIdealState and it ends up complaining that ZK
>>> size of 1M has been exceeded.
>>>
>>> I also tried using the addResource() api which creates an empty ideal
>>> state with a valid bucket size. However, when I tried to set the ideal
>>> state on that bucket (this ideal state did not have the bucket size set),
>>> it failed as well.
>>>
>>> Whats the right way to get bucketizing to work ?
>>>
>>> Thanks !
>>> Varun
>>>
>>
>>
>

Mime
View raw message