helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhen Zhang <zzh...@linkedin.com>
Subject RE: Cutting a release 0.6.5 tonight
Date Thu, 12 Mar 2015 20:24:43 GMT
Hi Varun,

It's a backward incompatible change. It's intend to fix the problem that different resources
can't have the same partition names. Simply adding a resource name as the first parameter
to your stateModelFactory#createNewStateModel(resourceName, partitionName) should be fine.

Thanks,
Jason
________________________________
From: Varun Sharma [varun@pinterest.com]
Sent: Thursday, March 12, 2015 1:18 PM
To: user@helix.apache.org
Subject: Re: Cutting a release 0.6.5 tonight

Kishore, I will be testing it out today. Btw, did you also look at the issue with change of
Abstract method signature createNewStateModel in StateModelFactory ?

On Thu, Mar 12, 2015 at 9:58 AM, kishore g <g.kishore@gmail.com<mailto:g.kishore@gmail.com>>
wrote:
Jason and I have checked in the changes needed. I am not sure if TestSchedulerMsg is fixed.

Last call for any other changes needed.

Varun, it will be great if you can test the bug fixes for bucketing before we cut the release.

thanks,
Kishore G

On Wed, Mar 11, 2015 at 5:42 PM, kishore g <g.kishore@gmail.com<mailto:g.kishore@gmail.com>>
wrote:
Thanks Lei, I fixed the first two.

Here is what caused the failure. I added the code to copy all simple fields from IS to EV
when we update the ExternalView in controller. This is to enable compression in ExternalView
if its set in IS. I could have copied only enableCompression variable but I thought its good
to have partition number/replica etc in ExternalView as well. Let me know if you foresee any
problem in this. Only thing I could think of is IdealState is deleted in which case I copy
the simplefields from existing externalview.

I dont understand why testschedulermsg test cases are failing. Uncommenting my code does not
help either.


On Wed, Mar 11, 2015 at 4:06 PM, Kanak Biscuitwala <kanak.b@hotmail.com<mailto:kanak.b@hotmail.com>>
wrote:
The first two are concerning. I don't think the scheduler message test failures should block
releases.

----------------------------------------
> From: lxia@linkedin.com.INVALID
> To: dev@helix.apache.org<mailto:dev@helix.apache.org>
> CC: user@helix.apache.org<mailto:user@helix.apache.org>
> Subject: RE: Cutting a release 0.6.5 tonight
> Date: Wed, 11 Mar 2015 21:25:15 +0000
>
> The test failed are:
>
> org.apache.helix.integration.TestExternalViewUpdates.testExternalViewUpdates
> org.apache.helix.integration.TestEnableCompression.testEnableCompressionResource
> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsg3
> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsg4
> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsgContraints
> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsgUsingQueue
>
> I got these from my local build too. (mvn clean install package on helix-0.6.x)
>
>
>
> Thanks
> Lei
>
> --
>
> Lei Xia
> Software Engineer
> Data Infrastructure/Distributed Data Systems/Nuage
> LinkedIn
>
> lxia@linkedin.com<mailto:lxia@linkedin.com>
> www.linkedin.com/in/lxia1<http://www.linkedin.com/in/lxia1>
>
> ________________________________________
> From: kishore g [g.kishore@gmail.com<mailto:g.kishore@gmail.com>]
> Sent: Wednesday, March 11, 2015 2:03 PM
> To: dev@helix.apache.org<mailto:dev@helix.apache.org>
> Cc: user@helix.apache.org<mailto:user@helix.apache.org>
> Subject: Re: Cutting a release 0.6.5 tonight
>
> Hi Lei,
>
> Can you point to the failures?
>
> thanks,
> Kishore G
>
> On Wed, Mar 11, 2015 at 1:19 PM, Lei Xia <lxia@linkedin.com.invalid> wrote:
>
>> Hi, Kishore
>>
>> I saw there are regression test failures from last two recent commits
>> on 0.6.x branch, running from both local box and Linkedin's hudson jobs.
>> Are we going to fix them before the release?
>>
>>
>> Thanks
>> Lei
>>
>> --
>>
>> Lei Xia
>> Software Engineer
>> Data Infrastructure/Distributed Data Systems/Nuage
>> LinkedIn
>>
>> lxia@linkedin.com<mailto:lxia@linkedin.com>
>> www.linkedin.com/in/lxia1<http://www.linkedin.com/in/lxia1>
>>
>> ________________________________________
>> From: kishore g [g.kishore@gmail.com<mailto:g.kishore@gmail.com>]
>> Sent: Wednesday, March 11, 2015 12:04 PM
>> To: user@helix.apache.org<mailto:user@helix.apache.org>
>> Cc: dev@helix.apache.org<mailto:dev@helix.apache.org>
>> Subject: Re: Cutting a release 0.6.5 tonight
>>
>> Will write a test case for migration and document the steps
>> On Mar 11, 2015 11:37 AM, "Varun Sharma" <varun@pinterest.com<mailto:varun@pinterest.com>>
wrote:
>>
>>> What would be the migration path from non-compressed buckets to
>> compressed
>>> non bucket resources ? It seems even the CURRENTSTATES are being bucketed
>>> in this case, I thought that was not expected with bucketing. Does the
>>> controller read these current states appropriately ? To migrate, it seems
>>> that we would need to also rewrite the CURRENT STATES ?
>>>
>>> On Wed, Mar 11, 2015 at 10:02 AM, kishore g <g.kishore@gmail.com<mailto:g.kishore@gmail.com>>
wrote:
>>>
>>>> Hi,
>>>>
>>>> I will work with Jason to cut a 0.6.5 release tonight.
>>>>
>>>> The new thing I added is to enableCompression while storing data in
>>>> Zookeeper, this allows us to go up to 100k partitions per resource
>> without
>>>> having to use bucketing feature. We also fixed few bugs with bucketed
>>>> resource just in case some one needs it.
>>>>
>>>> The property store api needs some changes, I plan to get it in today.
>>>>
>>>> Let me know if you need any other changes to be included. Are there any
>>>> changes that went into 0.7.x branch that we need to merge it back in to
>>>> 0.6.x ?
>>>>
>>>> thanks,
>>>> Kishore G
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>





Mime
View raw message