helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Varun Sharma <va...@pinterest.com>
Subject Re: Cutting a release 0.6.5 tonight
Date Thu, 12 Mar 2015 23:08:49 GMT
I just deployed this to a server node and when the current states node is
moved over - the bucket nodes are no longer present - the node has executed
dropped transitions for all the nodes. Is it expected to not contain any
buckets ?

On Thu, Mar 12, 2015 at 1:33 PM, Zhen Zhang <zzhang@linkedin.com> wrote:

>  Should be no problem. Controller and participant are compatible.
>
>  ------------------------------
> *From:* Varun Sharma [varun@pinterest.com]
> *Sent:* Thursday, March 12, 2015 1:29 PM
> *To:* user@helix.apache.org
>
> *Subject:* Re: Cutting a release 0.6.5 tonight
>
>   Just to confirm - it wont be a problem if the controller is running
> 0.6.5 and participant runs 0.6.4 for a while due to this ?
>
> On Thu, Mar 12, 2015 at 1:25 PM, kishore g <g.kishore@gmail.com> wrote:
>
>> Yes, I will think of a way to make it backwards compatible.
>>
>> Thanks
>> Kishore G
>>  On Mar 12, 2015 1:21 PM, "Varun Sharma" <varun@pinterest.com> wrote:
>>
>>> Kishore, I will be testing it out today. Btw, did you also look at the
>>> issue with change of Abstract method signature createNewStateModel in
>>> StateModelFactory ?
>>>
>>> On Thu, Mar 12, 2015 at 9:58 AM, kishore g <g.kishore@gmail.com> wrote:
>>>
>>>> Jason and I have checked in the changes needed. I am not sure if
>>>> TestSchedulerMsg is fixed.
>>>>
>>>>  Last call for any other changes needed.
>>>>
>>>>  Varun, it will be great if you can test the bug fixes for bucketing
>>>> before we cut the release.
>>>>
>>>>  thanks,
>>>> Kishore G
>>>>
>>>> On Wed, Mar 11, 2015 at 5:42 PM, kishore g <g.kishore@gmail.com> wrote:
>>>>
>>>>> Thanks Lei, I fixed the first two.
>>>>>
>>>>>  Here is what caused the failure. I added the code to copy all simple
>>>>> fields from IS to EV when we update the ExternalView in controller. This
is
>>>>> to enable compression in ExternalView if its set in IS. I could have
copied
>>>>> only enableCompression variable but I thought its good to have partition
>>>>> number/replica etc in ExternalView as well. Let me know if you foresee
any
>>>>> problem in this. Only thing I could think of is IdealState is deleted
in
>>>>> which case I copy the simplefields from existing externalview.
>>>>>
>>>>>  I dont understand why testschedulermsg test cases are failing.
>>>>> Uncommenting my code does not help either.
>>>>>
>>>>>
>>>>> On Wed, Mar 11, 2015 at 4:06 PM, Kanak Biscuitwala <
>>>>> kanak.b@hotmail.com> wrote:
>>>>>
>>>>>> The first two are concerning. I don't think the scheduler message
>>>>>> test failures should block releases.
>>>>>>
>>>>>> ----------------------------------------
>>>>>> > From: lxia@linkedin.com.INVALID
>>>>>> > To: dev@helix.apache.org
>>>>>> > CC: user@helix.apache.org
>>>>>> > Subject: RE: Cutting a release 0.6.5 tonight
>>>>>> > Date: Wed, 11 Mar 2015 21:25:15 +0000
>>>>>>  >
>>>>>> > The test failed are:
>>>>>> >
>>>>>> >
>>>>>> org.apache.helix.integration.TestExternalViewUpdates.testExternalViewUpdates
>>>>>> >
>>>>>> org.apache.helix.integration.TestEnableCompression.testEnableCompressionResource
>>>>>> > org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsg3
>>>>>> > org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsg4
>>>>>> >
>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsgContraints
>>>>>> >
>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsgUsingQueue
>>>>>> >
>>>>>> > I got these from my local build too. (mvn clean install package
on
>>>>>> helix-0.6.x)
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > Thanks
>>>>>> > Lei
>>>>>> >
>>>>>> > --
>>>>>> >
>>>>>> > Lei Xia
>>>>>> > Software Engineer
>>>>>> > Data Infrastructure/Distributed Data Systems/Nuage
>>>>>> > LinkedIn
>>>>>> >
>>>>>> > lxia@linkedin.com
>>>>>> > www.linkedin.com/in/lxia1
>>>>>> >
>>>>>> > ________________________________________
>>>>>> > From: kishore g [g.kishore@gmail.com]
>>>>>> > Sent: Wednesday, March 11, 2015 2:03 PM
>>>>>> > To: dev@helix.apache.org
>>>>>> > Cc: user@helix.apache.org
>>>>>> > Subject: Re: Cutting a release 0.6.5 tonight
>>>>>> >
>>>>>> > Hi Lei,
>>>>>> >
>>>>>> > Can you point to the failures?
>>>>>> >
>>>>>> > thanks,
>>>>>> > Kishore G
>>>>>> >
>>>>>> > On Wed, Mar 11, 2015 at 1:19 PM, Lei Xia <lxia@linkedin.com.invalid>
>>>>>> wrote:
>>>>>> >
>>>>>> >> Hi, Kishore
>>>>>> >>
>>>>>> >> I saw there are regression test failures from last two recent
>>>>>> commits
>>>>>> >> on 0.6.x branch, running from both local box and Linkedin's
hudson
>>>>>> jobs.
>>>>>> >> Are we going to fix them before the release?
>>>>>> >>
>>>>>> >>
>>>>>> >> Thanks
>>>>>> >> Lei
>>>>>> >>
>>>>>> >> --
>>>>>> >>
>>>>>> >> Lei Xia
>>>>>> >> Software Engineer
>>>>>> >> Data Infrastructure/Distributed Data Systems/Nuage
>>>>>> >> LinkedIn
>>>>>> >>
>>>>>> >> lxia@linkedin.com
>>>>>> >> www.linkedin.com/in/lxia1
>>>>>> >>
>>>>>> >> ________________________________________
>>>>>> >> From: kishore g [g.kishore@gmail.com]
>>>>>> >> Sent: Wednesday, March 11, 2015 12:04 PM
>>>>>> >> To: user@helix.apache.org
>>>>>> >> Cc: dev@helix.apache.org
>>>>>> >> Subject: Re: Cutting a release 0.6.5 tonight
>>>>>> >>
>>>>>> >> Will write a test case for migration and document the steps
>>>>>> >> On Mar 11, 2015 11:37 AM, "Varun Sharma" <varun@pinterest.com>
>>>>>> wrote:
>>>>>> >>
>>>>>> >>> What would be the migration path from non-compressed
buckets to
>>>>>> >> compressed
>>>>>> >>> non bucket resources ? It seems even the CURRENTSTATES
are being
>>>>>> bucketed
>>>>>> >>> in this case, I thought that was not expected with bucketing.
>>>>>> Does the
>>>>>> >>> controller read these current states appropriately ?
To migrate,
>>>>>> it seems
>>>>>> >>> that we would need to also rewrite the CURRENT STATES
?
>>>>>> >>>
>>>>>> >>> On Wed, Mar 11, 2015 at 10:02 AM, kishore g <g.kishore@gmail.com>
>>>>>> wrote:
>>>>>> >>>
>>>>>> >>>> Hi,
>>>>>> >>>>
>>>>>> >>>> I will work with Jason to cut a 0.6.5 release tonight.
>>>>>> >>>>
>>>>>> >>>> The new thing I added is to enableCompression while
storing data
>>>>>> in
>>>>>> >>>> Zookeeper, this allows us to go up to 100k partitions
per
>>>>>> resource
>>>>>> >> without
>>>>>> >>>> having to use bucketing feature. We also fixed few
bugs with
>>>>>> bucketed
>>>>>> >>>> resource just in case some one needs it.
>>>>>> >>>>
>>>>>> >>>> The property store api needs some changes, I plan
to get it in
>>>>>> today.
>>>>>> >>>>
>>>>>> >>>> Let me know if you need any other changes to be
included. Are
>>>>>> there any
>>>>>> >>>> changes that went into 0.7.x branch that we need
to merge it
>>>>>> back in to
>>>>>> >>>> 0.6.x ?
>>>>>> >>>>
>>>>>> >>>> thanks,
>>>>>> >>>> Kishore G
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>
>>>>>> >>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>

Mime
View raw message