helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Varun Sharma <va...@pinterest.com>
Subject Re: Cutting a release 0.6.5 tonight
Date Fri, 13 Mar 2015 00:24:22 GMT
I tested it out using a resource with 8192 partitions and 8 buckets and it
worked out just fine - (disable and drop).

On Thu, Mar 12, 2015 at 4:08 PM, Varun Sharma <varun@pinterest.com> wrote:

> I just deployed this to a server node and when the current states node is
> moved over - the bucket nodes are no longer present - the node has executed
> dropped transitions for all the nodes. Is it expected to not contain any
> buckets ?
>
> On Thu, Mar 12, 2015 at 1:33 PM, Zhen Zhang <zzhang@linkedin.com> wrote:
>
>>  Should be no problem. Controller and participant are compatible.
>>
>>  ------------------------------
>> *From:* Varun Sharma [varun@pinterest.com]
>> *Sent:* Thursday, March 12, 2015 1:29 PM
>> *To:* user@helix.apache.org
>>
>> *Subject:* Re: Cutting a release 0.6.5 tonight
>>
>>   Just to confirm - it wont be a problem if the controller is running
>> 0.6.5 and participant runs 0.6.4 for a while due to this ?
>>
>> On Thu, Mar 12, 2015 at 1:25 PM, kishore g <g.kishore@gmail.com> wrote:
>>
>>> Yes, I will think of a way to make it backwards compatible.
>>>
>>> Thanks
>>> Kishore G
>>>  On Mar 12, 2015 1:21 PM, "Varun Sharma" <varun@pinterest.com> wrote:
>>>
>>>> Kishore, I will be testing it out today. Btw, did you also look at the
>>>> issue with change of Abstract method signature createNewStateModel in
>>>> StateModelFactory ?
>>>>
>>>> On Thu, Mar 12, 2015 at 9:58 AM, kishore g <g.kishore@gmail.com> wrote:
>>>>
>>>>> Jason and I have checked in the changes needed. I am not sure if
>>>>> TestSchedulerMsg is fixed.
>>>>>
>>>>>  Last call for any other changes needed.
>>>>>
>>>>>  Varun, it will be great if you can test the bug fixes for bucketing
>>>>> before we cut the release.
>>>>>
>>>>>  thanks,
>>>>> Kishore G
>>>>>
>>>>> On Wed, Mar 11, 2015 at 5:42 PM, kishore g <g.kishore@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Thanks Lei, I fixed the first two.
>>>>>>
>>>>>>  Here is what caused the failure. I added the code to copy all
>>>>>> simple fields from IS to EV when we update the ExternalView in controller.
>>>>>> This is to enable compression in ExternalView if its set in IS. I
could
>>>>>> have copied only enableCompression variable but I thought its good
to have
>>>>>> partition number/replica etc in ExternalView as well. Let me know
if you
>>>>>> foresee any problem in this. Only thing I could think of is IdealState
is
>>>>>> deleted in which case I copy the simplefields from existing externalview.
>>>>>>
>>>>>>  I dont understand why testschedulermsg test cases are failing.
>>>>>> Uncommenting my code does not help either.
>>>>>>
>>>>>>
>>>>>> On Wed, Mar 11, 2015 at 4:06 PM, Kanak Biscuitwala <
>>>>>> kanak.b@hotmail.com> wrote:
>>>>>>
>>>>>>> The first two are concerning. I don't think the scheduler message
>>>>>>> test failures should block releases.
>>>>>>>
>>>>>>> ----------------------------------------
>>>>>>> > From: lxia@linkedin.com.INVALID
>>>>>>> > To: dev@helix.apache.org
>>>>>>> > CC: user@helix.apache.org
>>>>>>> > Subject: RE: Cutting a release 0.6.5 tonight
>>>>>>> > Date: Wed, 11 Mar 2015 21:25:15 +0000
>>>>>>>  >
>>>>>>> > The test failed are:
>>>>>>> >
>>>>>>> >
>>>>>>> org.apache.helix.integration.TestExternalViewUpdates.testExternalViewUpdates
>>>>>>> >
>>>>>>> org.apache.helix.integration.TestEnableCompression.testEnableCompressionResource
>>>>>>> > org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsg3
>>>>>>> > org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsg4
>>>>>>> >
>>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsgContraints
>>>>>>> >
>>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsgUsingQueue
>>>>>>> >
>>>>>>> > I got these from my local build too. (mvn clean install
package on
>>>>>>> helix-0.6.x)
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> > Thanks
>>>>>>> > Lei
>>>>>>> >
>>>>>>> > --
>>>>>>> >
>>>>>>> > Lei Xia
>>>>>>> > Software Engineer
>>>>>>> > Data Infrastructure/Distributed Data Systems/Nuage
>>>>>>> > LinkedIn
>>>>>>> >
>>>>>>> > lxia@linkedin.com
>>>>>>> > www.linkedin.com/in/lxia1
>>>>>>> >
>>>>>>> > ________________________________________
>>>>>>> > From: kishore g [g.kishore@gmail.com]
>>>>>>> > Sent: Wednesday, March 11, 2015 2:03 PM
>>>>>>> > To: dev@helix.apache.org
>>>>>>> > Cc: user@helix.apache.org
>>>>>>> > Subject: Re: Cutting a release 0.6.5 tonight
>>>>>>> >
>>>>>>> > Hi Lei,
>>>>>>> >
>>>>>>> > Can you point to the failures?
>>>>>>> >
>>>>>>> > thanks,
>>>>>>> > Kishore G
>>>>>>> >
>>>>>>> > On Wed, Mar 11, 2015 at 1:19 PM, Lei Xia <lxia@linkedin.com.invalid>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> >> Hi, Kishore
>>>>>>> >>
>>>>>>> >> I saw there are regression test failures from last two
recent
>>>>>>> commits
>>>>>>> >> on 0.6.x branch, running from both local box and Linkedin's
>>>>>>> hudson jobs.
>>>>>>> >> Are we going to fix them before the release?
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> Thanks
>>>>>>> >> Lei
>>>>>>> >>
>>>>>>> >> --
>>>>>>> >>
>>>>>>> >> Lei Xia
>>>>>>> >> Software Engineer
>>>>>>> >> Data Infrastructure/Distributed Data Systems/Nuage
>>>>>>> >> LinkedIn
>>>>>>> >>
>>>>>>> >> lxia@linkedin.com
>>>>>>> >> www.linkedin.com/in/lxia1
>>>>>>> >>
>>>>>>> >> ________________________________________
>>>>>>> >> From: kishore g [g.kishore@gmail.com]
>>>>>>> >> Sent: Wednesday, March 11, 2015 12:04 PM
>>>>>>> >> To: user@helix.apache.org
>>>>>>> >> Cc: dev@helix.apache.org
>>>>>>> >> Subject: Re: Cutting a release 0.6.5 tonight
>>>>>>> >>
>>>>>>> >> Will write a test case for migration and document the
steps
>>>>>>> >> On Mar 11, 2015 11:37 AM, "Varun Sharma" <varun@pinterest.com>
>>>>>>> wrote:
>>>>>>> >>
>>>>>>> >>> What would be the migration path from non-compressed
buckets to
>>>>>>> >> compressed
>>>>>>> >>> non bucket resources ? It seems even the CURRENTSTATES
are being
>>>>>>> bucketed
>>>>>>> >>> in this case, I thought that was not expected with
bucketing.
>>>>>>> Does the
>>>>>>> >>> controller read these current states appropriately
? To migrate,
>>>>>>> it seems
>>>>>>> >>> that we would need to also rewrite the CURRENT STATES
?
>>>>>>> >>>
>>>>>>> >>> On Wed, Mar 11, 2015 at 10:02 AM, kishore g <g.kishore@gmail.com>
>>>>>>> wrote:
>>>>>>> >>>
>>>>>>> >>>> Hi,
>>>>>>> >>>>
>>>>>>> >>>> I will work with Jason to cut a 0.6.5 release
tonight.
>>>>>>> >>>>
>>>>>>> >>>> The new thing I added is to enableCompression
while storing
>>>>>>> data in
>>>>>>> >>>> Zookeeper, this allows us to go up to 100k partitions
per
>>>>>>> resource
>>>>>>> >> without
>>>>>>> >>>> having to use bucketing feature. We also fixed
few bugs with
>>>>>>> bucketed
>>>>>>> >>>> resource just in case some one needs it.
>>>>>>> >>>>
>>>>>>> >>>> The property store api needs some changes, I
plan to get it in
>>>>>>> today.
>>>>>>> >>>>
>>>>>>> >>>> Let me know if you need any other changes to
be included. Are
>>>>>>> there any
>>>>>>> >>>> changes that went into 0.7.x branch that we
need to merge it
>>>>>>> back in to
>>>>>>> >>>> 0.6.x ?
>>>>>>> >>>>
>>>>>>> >>>> thanks,
>>>>>>> >>>> Kishore G
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>
>>>>>>> >>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>
>

Mime
View raw message