helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kishore g <g.kish...@gmail.com>
Subject Re: Cutting a release 0.6.5 tonight
Date Mon, 16 Mar 2015 02:04:19 GMT
Update: we have fixed all issues but I am seeing some tests that are flaky
and fail.

Is there a way to turn of running tests while making the release ( I know
this is bad, but I will verify that failed tests pass when run again, which
is the case most of the time).

On Thu, Mar 12, 2015 at 7:55 PM, kishore g <g.kishore@gmail.com> wrote:

> Thanks for the stack trace, let me try to reproduce this.
>
> On Thu, Mar 12, 2015 at 7:53 PM, Varun Sharma <varun@pinterest.com> wrote:
>
>> Here is the relevant trace. Essentially, the following happens -
>> externalview is deleted by controller since no more nodes are reporting it
>> in the current state. After removing the rogue node, controller tries to
>> still subscribe to the zk data/child change and we get a NoNode exception.
>> The RoutingTableProvider receives a FINALIZE notification and we see the
>> "Resetting the RoutingTableProvider" notification. I hit this the very
>> first time I got a bucketed resource deleted correctly in 0.6.5 so it
>> should be reproducible.
>>
>> *2015-03-13 00:10:30,172 [Thread-7] (ExternalViewComputeStage.java:161)
>> INFO  Remove externalView for resource:
>> $terrapin$data$pinpin_board_join$1424982892102*
>>
>> 2015-03-13 00:10:30,190 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:352) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102/$terrapin$data$pinpin_board_join$1424982892102_p911-p1821,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,190 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:358) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe child-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102/$terrapin$data$pinpin_board_join$1424982892102_p911-p1821,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,193 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:139) INFO  16 START:INVOKE
>> /mainoffline_a/EXTERNALVIEW
>> listener:com.pinterest.terrapin.controller.TerrapinRoutingTableProvider
>>
>> 2015-03-13 00:10:30,193 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:223) INFO  terrapinmainoffline-a-namenode001_9090
>> subscribes child-change. path: /mainoffline_a/EXTERNALVIEW, listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> *2015-03-13 00:10:30,207 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:223) INFO  terrapinmainoffline-a-namenode001_9090
>> subscribes child-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331*
>>
>> 2015-03-13 00:10:30,217 [Thread-7] (ExternalViewComputeStage.java:167)
>> INFO  END ExternalViewComputeStage.process(). took: 229 ms
>>
>> 2015-03-13 00:10:30,217 [Thread-7] (GenericHelixController.java:301)
>> INFO  END: Invoking controller pipeline for event: currentStateChange, took
>> 1082 ms
>>
>> 2015-03-13 00:10:30,218 [Thread-7] (GenericHelixController.java:288)
>> INFO  START: Invoking controller pipeline for event: messageChange
>>
>> 2015-03-13 00:10:30,218 [Thread-7] (ReadClusterDataStage.java:47) INFO
>> START ReadClusterDataStage.process()
>>
>> *2015-03-13 00:10:30,218 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:304) WARN  fail to subscribe child/data change. path:
>> /mainoffline_a/EXTERNALVIEW, listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331*
>>
>> *org.I0Itec.zkclient.exception.ZkNoNodeException:
>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
>> NoNode for
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102*
>>
>>         at
>> org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:47)
>>
>>         at
>> org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685)
>>
>> at org.apache.helix.manager.zk.ZkClient.getChildren(ZkClient.java:209)
>>
>>         at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:409)
>>
>>         at
>> org.apache.helix.manager.zk.CallbackHandler.subscribeForChanges(CallbackHandler.java:279)
>>
>>         at
>> org.apache.helix.manager.zk.CallbackHandler.invoke(CallbackHandler.java:202)
>>
>>         at
>> org.apache.helix.manager.zk.CallbackHandler.handleChildChange(CallbackHandler.java:391)
>>
>>         at org.I0Itec.zkclient.ZkClient$7.run(ZkClient.java:568)
>>
>>         at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71)
>>
>> Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
>> KeeperErrorCode = NoNode for
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102
>>
>>         at
>> org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
>>
>>         at
>> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
>>
>>         at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1249)
>>
>>         at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1277)
>>
>>         at
>> org.I0Itec.zkclient.ZkConnection.getChildren(ZkConnection.java:99)
>>
>>         at org.apache.helix.manager.zk.ZkClient$3.call(ZkClient.java:212)
>>
>>         at org.apache.helix.manager.zk.ZkClient$3.call(ZkClient.java:209)
>>
>>         at
>> org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
>>
>>         ... 7 more
>>
>> 2015-03-13 00:10:30,218 [Thread-7] (ClusterDataCache.java:83) INFO
>> START: ClusterDataCache.refresh()
>>
>> 2015-03-13 00:10:30,347 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:214) INFO  16 END:INVOKE /mainoffline_a/EXTERNALVIEW
>> listener:com.pinterest.terrapin.controller.TerrapinRoutingTableProvider
>> Took: 154ms
>>
>> 2015-03-13 00:10:30,348 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:352) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102/$terrapin$data$pinpin_board_join$1424982892102_p5466-p6376,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,348 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:358) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe child-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102/$terrapin$data$pinpin_board_join$1424982892102_p5466-p6376,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,350 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:352) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102/$terrapin$data$pinpin_board_join$1424982892102_p3644-p4554,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,350 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:358) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe child-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102/$terrapin$data$pinpin_board_join$1424982892102_p3644-p4554,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,351 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:352) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102/$terrapin$data$pinpin_board_join$1424982892102_p2733-p3643,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,351 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:358) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe child-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1424982892102/$terrapin$data$pinpin_board_join$1424982892102_p2733-p3643,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,352 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:139) INFO  16 START:INVOKE
>> /mainoffline_a/EXTERNALVIEW
>> listener:com.pinterest.terrapin.controller.TerrapinRoutingTableProvider
>>
>> 2015-03-13 00:10:30,352 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:227) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe child-change. path: /mainoffline_a/EXTERNALVIEW, listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,362 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:227) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe child-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,362 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,367 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p7288-p8198,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,367 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p2733-p3643,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,367 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p5466-p6376,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,367 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p3644-p4554,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,367 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p0-p910,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,368 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p911-p1821,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,368 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p6377-p7287,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,368 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p6377-p7287,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,368 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p4555-p5465,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,368 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_board_join$1426126779804/$terrapin$data$pinpin_board_join$1426126779804_p1822-p2732,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,368 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:227) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe child-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_pin_join$1424997461905,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,368 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_pin_join$1424997461905,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,369 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_pin_join$1424997461905/$terrapin$data$pinpin_pin_join$1424997461905_p683-p1365,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,369 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_pin_join$1424997461905/$terrapin$data$pinpin_pin_join$1424997461905_p1366-p2048,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,369 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$pinpin_pin_join$1424997461905/$terrapin$data$pinpin_pin_join$1424997461905_p0-p682,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb73312015-03-13
>> 00:10:30,369 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$popular_pins$1422378222676,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,369 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$recommendation_p2p_fullsig$1422422699753,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> 2015-03-13 00:10:30,370 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (CallbackHandler.java:244) INFO  terrapinmainoffline-a-namenode001_9090
>> unsubscribe data-change. path:
>> /mainoffline_a/EXTERNALVIEW/$terrapin$data$u2p_from_fb_friends$1422438987857,
>> listener:
>> com.pinterest.terrapin.controller.TerrapinRoutingTableProvider@36eb7331
>>
>> *2015-03-13 00:10:30,439 [ZkClient-EventThread-16-terrapinzk001a:2181]
>> (RoutingTableProvider.java:99) INFO  Resetting the routing table.*
>>
>> On Thu, Mar 12, 2015 at 7:22 PM, Varun Sharma <varun@pinterest.com>
>> wrote:
>>
>>> Sorry spoke too soon - I saw the "Resetting the RoutingTableProvider"
>>> bug again which caused the RoutingTableProvider to stop receiving updates.
>>>
>>> Varun
>>>
>>> On Thu, Mar 12, 2015 at 5:24 PM, Varun Sharma <varun@pinterest.com>
>>> wrote:
>>>
>>>> I tested it out using a resource with 8192 partitions and 8 buckets and
>>>> it worked out just fine - (disable and drop).
>>>>
>>>> On Thu, Mar 12, 2015 at 4:08 PM, Varun Sharma <varun@pinterest.com>
>>>> wrote:
>>>>
>>>>> I just deployed this to a server node and when the current states node
>>>>> is moved over - the bucket nodes are no longer present - the node has
>>>>> executed dropped transitions for all the nodes. Is it expected to not
>>>>> contain any buckets ?
>>>>>
>>>>> On Thu, Mar 12, 2015 at 1:33 PM, Zhen Zhang <zzhang@linkedin.com>
>>>>> wrote:
>>>>>
>>>>>>  Should be no problem. Controller and participant are compatible.
>>>>>>
>>>>>>  ------------------------------
>>>>>> *From:* Varun Sharma [varun@pinterest.com]
>>>>>> *Sent:* Thursday, March 12, 2015 1:29 PM
>>>>>> *To:* user@helix.apache.org
>>>>>>
>>>>>> *Subject:* Re: Cutting a release 0.6.5 tonight
>>>>>>
>>>>>>   Just to confirm - it wont be a problem if the controller is
>>>>>> running 0.6.5 and participant runs 0.6.4 for a while due to this
?
>>>>>>
>>>>>> On Thu, Mar 12, 2015 at 1:25 PM, kishore g <g.kishore@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Yes, I will think of a way to make it backwards compatible.
>>>>>>>
>>>>>>> Thanks
>>>>>>> Kishore G
>>>>>>>  On Mar 12, 2015 1:21 PM, "Varun Sharma" <varun@pinterest.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Kishore, I will be testing it out today. Btw, did you also
look at
>>>>>>>> the issue with change of Abstract method signature createNewStateModel
in
>>>>>>>> StateModelFactory ?
>>>>>>>>
>>>>>>>> On Thu, Mar 12, 2015 at 9:58 AM, kishore g <g.kishore@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Jason and I have checked in the changes needed. I am
not sure if
>>>>>>>>> TestSchedulerMsg is fixed.
>>>>>>>>>
>>>>>>>>>  Last call for any other changes needed.
>>>>>>>>>
>>>>>>>>>  Varun, it will be great if you can test the bug fixes
for
>>>>>>>>> bucketing before we cut the release.
>>>>>>>>>
>>>>>>>>>  thanks,
>>>>>>>>> Kishore G
>>>>>>>>>
>>>>>>>>> On Wed, Mar 11, 2015 at 5:42 PM, kishore g <g.kishore@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks Lei, I fixed the first two.
>>>>>>>>>>
>>>>>>>>>>  Here is what caused the failure. I added the code
to copy all
>>>>>>>>>> simple fields from IS to EV when we update the ExternalView
in controller.
>>>>>>>>>> This is to enable compression in ExternalView if
its set in IS. I could
>>>>>>>>>> have copied only enableCompression variable but I
thought its good to have
>>>>>>>>>> partition number/replica etc in ExternalView as well.
Let me know if you
>>>>>>>>>> foresee any problem in this. Only thing I could think
of is IdealState is
>>>>>>>>>> deleted in which case I copy the simplefields from
existing externalview.
>>>>>>>>>>
>>>>>>>>>>  I dont understand why testschedulermsg test cases
are failing.
>>>>>>>>>> Uncommenting my code does not help either.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 11, 2015 at 4:06 PM, Kanak Biscuitwala
<
>>>>>>>>>> kanak.b@hotmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> The first two are concerning. I don't think the
scheduler
>>>>>>>>>>> message test failures should block releases.
>>>>>>>>>>>
>>>>>>>>>>> ----------------------------------------
>>>>>>>>>>> > From: lxia@linkedin.com.INVALID
>>>>>>>>>>> > To: dev@helix.apache.org
>>>>>>>>>>> > CC: user@helix.apache.org
>>>>>>>>>>> > Subject: RE: Cutting a release 0.6.5 tonight
>>>>>>>>>>> > Date: Wed, 11 Mar 2015 21:25:15 +0000
>>>>>>>>>>>  >
>>>>>>>>>>> > The test failed are:
>>>>>>>>>>> >
>>>>>>>>>>> >
>>>>>>>>>>> org.apache.helix.integration.TestExternalViewUpdates.testExternalViewUpdates
>>>>>>>>>>> >
>>>>>>>>>>> org.apache.helix.integration.TestEnableCompression.testEnableCompressionResource
>>>>>>>>>>> >
>>>>>>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsg3
>>>>>>>>>>> >
>>>>>>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsg4
>>>>>>>>>>> >
>>>>>>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsgContraints
>>>>>>>>>>> >
>>>>>>>>>>> org.apache.helix.integration.TestSchedulerMessage.testSchedulerMsgUsingQueue
>>>>>>>>>>> >
>>>>>>>>>>> > I got these from my local build too. (mvn
clean install
>>>>>>>>>>> package on helix-0.6.x)
>>>>>>>>>>> >
>>>>>>>>>>> >
>>>>>>>>>>> >
>>>>>>>>>>> > Thanks
>>>>>>>>>>> > Lei
>>>>>>>>>>> >
>>>>>>>>>>> > --
>>>>>>>>>>> >
>>>>>>>>>>> > Lei Xia
>>>>>>>>>>> > Software Engineer
>>>>>>>>>>> > Data Infrastructure/Distributed Data Systems/Nuage
>>>>>>>>>>> > LinkedIn
>>>>>>>>>>> >
>>>>>>>>>>> > lxia@linkedin.com
>>>>>>>>>>> > www.linkedin.com/in/lxia1
>>>>>>>>>>> >
>>>>>>>>>>> > ________________________________________
>>>>>>>>>>> > From: kishore g [g.kishore@gmail.com]
>>>>>>>>>>> > Sent: Wednesday, March 11, 2015 2:03 PM
>>>>>>>>>>> > To: dev@helix.apache.org
>>>>>>>>>>> > Cc: user@helix.apache.org
>>>>>>>>>>> > Subject: Re: Cutting a release 0.6.5 tonight
>>>>>>>>>>> >
>>>>>>>>>>> > Hi Lei,
>>>>>>>>>>> >
>>>>>>>>>>> > Can you point to the failures?
>>>>>>>>>>> >
>>>>>>>>>>> > thanks,
>>>>>>>>>>> > Kishore G
>>>>>>>>>>> >
>>>>>>>>>>> > On Wed, Mar 11, 2015 at 1:19 PM, Lei Xia
>>>>>>>>>>> <lxia@linkedin.com.invalid> wrote:
>>>>>>>>>>> >
>>>>>>>>>>> >> Hi, Kishore
>>>>>>>>>>> >>
>>>>>>>>>>> >> I saw there are regression test failures
from last two recent
>>>>>>>>>>> commits
>>>>>>>>>>> >> on 0.6.x branch, running from both local
box and Linkedin's
>>>>>>>>>>> hudson jobs.
>>>>>>>>>>> >> Are we going to fix them before the
release?
>>>>>>>>>>> >>
>>>>>>>>>>> >>
>>>>>>>>>>> >> Thanks
>>>>>>>>>>> >> Lei
>>>>>>>>>>> >>
>>>>>>>>>>> >> --
>>>>>>>>>>> >>
>>>>>>>>>>> >> Lei Xia
>>>>>>>>>>> >> Software Engineer
>>>>>>>>>>> >> Data Infrastructure/Distributed Data
Systems/Nuage
>>>>>>>>>>> >> LinkedIn
>>>>>>>>>>> >>
>>>>>>>>>>> >> lxia@linkedin.com
>>>>>>>>>>> >> www.linkedin.com/in/lxia1
>>>>>>>>>>> >>
>>>>>>>>>>> >> ________________________________________
>>>>>>>>>>> >> From: kishore g [g.kishore@gmail.com]
>>>>>>>>>>> >> Sent: Wednesday, March 11, 2015 12:04
PM
>>>>>>>>>>> >> To: user@helix.apache.org
>>>>>>>>>>> >> Cc: dev@helix.apache.org
>>>>>>>>>>> >> Subject: Re: Cutting a release 0.6.5
tonight
>>>>>>>>>>> >>
>>>>>>>>>>> >> Will write a test case for migration
and document the steps
>>>>>>>>>>> >> On Mar 11, 2015 11:37 AM, "Varun Sharma"
<varun@pinterest.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> >>
>>>>>>>>>>> >>> What would be the migration path
from non-compressed buckets
>>>>>>>>>>> to
>>>>>>>>>>> >> compressed
>>>>>>>>>>> >>> non bucket resources ? It seems
even the CURRENTSTATES are
>>>>>>>>>>> being bucketed
>>>>>>>>>>> >>> in this case, I thought that was
not expected with
>>>>>>>>>>> bucketing. Does the
>>>>>>>>>>> >>> controller read these current states
appropriately ? To
>>>>>>>>>>> migrate, it seems
>>>>>>>>>>> >>> that we would need to also rewrite
the CURRENT STATES ?
>>>>>>>>>>> >>>
>>>>>>>>>>> >>> On Wed, Mar 11, 2015 at 10:02 AM,
kishore g <
>>>>>>>>>>> g.kishore@gmail.com> wrote:
>>>>>>>>>>> >>>
>>>>>>>>>>> >>>> Hi,
>>>>>>>>>>> >>>>
>>>>>>>>>>> >>>> I will work with Jason to cut
a 0.6.5 release tonight.
>>>>>>>>>>> >>>>
>>>>>>>>>>> >>>> The new thing I added is to
enableCompression while storing
>>>>>>>>>>> data in
>>>>>>>>>>> >>>> Zookeeper, this allows us to
go up to 100k partitions per
>>>>>>>>>>> resource
>>>>>>>>>>> >> without
>>>>>>>>>>> >>>> having to use bucketing feature.
We also fixed few bugs
>>>>>>>>>>> with bucketed
>>>>>>>>>>> >>>> resource just in case some one
needs it.
>>>>>>>>>>> >>>>
>>>>>>>>>>> >>>> The property store api needs
some changes, I plan to get it
>>>>>>>>>>> in today.
>>>>>>>>>>> >>>>
>>>>>>>>>>> >>>> Let me know if you need any
other changes to be included.
>>>>>>>>>>> Are there any
>>>>>>>>>>> >>>> changes that went into 0.7.x
branch that we need to merge
>>>>>>>>>>> it back in to
>>>>>>>>>>> >>>> 0.6.x ?
>>>>>>>>>>> >>>>
>>>>>>>>>>> >>>> thanks,
>>>>>>>>>>> >>>> Kishore G
>>>>>>>>>>> >>>>
>>>>>>>>>>> >>>>
>>>>>>>>>>> >>>>
>>>>>>>>>>> >>>>
>>>>>>>>>>> >>>>
>>>>>>>>>>> >>>>
>>>>>>>>>>> >>>
>>>>>>>>>>> >>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message