hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (Naga)" <garlanaganarasi...@huawei.com>
Subject RE: 2.7.2 release plan
Date Tue, 03 Nov 2015 21:30:42 GMT
Thanks Wangda, for looking into the list,

If the changes are riskier then hoping to get 2.8 earlier :)



+ Naga

________________________________
From: Wangda Tan [wheeleast@gmail.com]
Sent: Monday, November 02, 2015 13:54
To: Naganarasimha G R (Naga)
Cc: yarn-dev@hadoop.apache.org; Tsuyoshi Ozawa; Vinod Vavilapalli; hdfs-dev@hadoop.apache.org;
common-dev@hadoop.apache.org; Vinod Kumar Vavilapalli
Subject: Re: 2.7.2 release plan

HI Naga and Vinod/Tsuyoshi/Karthik,

Looked at this list, IIRC, some of them are 70k+ patch, I'm afraid the changes number is too
many and risky for a minor release. Issues besides YARN-3136 are more or less change web UI
/ REST API, and they look more like enhancements instead of bug fixes.

I marked YARN-3136 to 2.7.2-candidate, and I suggest to delay other changes to 2.8.0 release.

Thoughts?

Regards,
Wangda


On Wed, Oct 28, 2015 at 11:56 PM, Naganarasimha G R (Naga) <garlanaganarasimha@huawei.com<mailto:garlanaganarasimha@huawei.com>>
wrote:
Thanks for sharing this important viewpoint.

This sub list of NodeLabels jiras what i have selected is doing minimal modifications to the
core code but tries to increase the usability of NodeLabels and fix some bugs or add missing
necessary features
Other additional features which  were done for NodeLabels like Distributed Scheduling or Delegated
Centralized are all not included.
May be Wangda could be better judge to further scrutinize the list and select from them or
even add to them
My intention here is to only make the Node Labels more usable as there has been long delay
for 2.8 and not heard of any approximate dates for it.

Regards,
+ Naga


________________________________________
From: Karthik Kambatla [kasha@cloudera.com<mailto:kasha@cloudera.com>]
Sent: Thursday, October 29, 2015 04:04
To: yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org>
Cc: Tsuyoshi Ozawa; Vinod Vavilapalli; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>;
common-dev@hadoop.apache.org<mailto:common-dev@hadoop.apache.org>; Vinod Kumar Vavilapalli;
Wangda Tan
Subject: Re: 2.7.2 release plan

I would like for us to make sure later maintenance releases are more stable
than previous ones. IMO, increasing stability is more important than the
timing of a release.

Will adding all the patches in 2.7.3 reduce the stability going from 2.7.2
to 2.7.3? If yes, can we just leave them for 2.8.0?

On Wed, Oct 28, 2015 at 12:07 PM, Naganarasimha G R (Naga) <
garlanaganarasimha@huawei.com<mailto:garlanaganarasimha@huawei.com>> wrote:

> Yes Vinod & Tsuyoshi,
>
> Within a week merging them would be difficult. I can start backporting
> them after 2.7.2 so that it can be ported to 2.7.3 faster, also shall i
> apply  2.7.3-candidate labels to them ?
>
> + Naga
> ______________________________
> From: Tsuyoshi Ozawa [ozawa@apache.org<mailto:ozawa@apache.org>]
> Sent: Wednesday, October 28, 2015 23:13
> To: Vinod Vavilapalli
> Cc: yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>;
> common-dev@hadoop.apache.org<mailto:common-dev@hadoop.apache.org>; Vinod Kumar
Vavilapalli; Wangda Tan;
> Tsuyoshi Ozawa; Naganarasimha G R (Naga)
> Subject: Re: 2.7.2 release plan
>
> Vinod,
>
> Thank you for taking care of this. I've checked the list of changes.
> As a result, I agree that we don't have enough time to backport these
> changes into 2.7.2 by this weekend. For a fast move, it's better
> suggestion to me to backport these tickets into 2.7.3.
>
> Best,
> - Tsuyoshi
>
> On Thu, Oct 29, 2015 at 2:19 AM, Vinod Vavilapalli
> <vinodkv@hortonworks.com<mailto:vinodkv@hortonworks.com>> wrote:
> > Tsuyoshi / Wangda / Naga,
> >
> > This looks too big of a list to me if we have to cut an RC by this
> weekend per my plan.
> >
> > I’d suggest a fast move on things you think are low risk enough and punt
> everything else for next release.
> >
> > Thanks
> > +Vinod
> >
> >> On Oct 28, 2015, at 3:08 AM, Naganarasimha G R (Naga) <
> garlanaganarasimha@huawei.com<mailto:garlanaganarasimha@huawei.com>> wrote:
> >>
> >> Thanks Tsuyoshi,
> >> If required even i can pitch in  :)
> >> Additional to this we added the support in Mapreduce for labels in
> MAPREDUCE-6304,
> >>
> >> Regards,
> >> + Naga
> >> ________________________________________
> >> From: Tsuyoshi Ozawa [ozawa@apache.org<mailto:ozawa@apache.org>]
> >> Sent: Wednesday, October 28, 2015 14:28
> >> To: yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org>
> >> Cc: hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>; common-dev@hadoop.apache.org<mailto:common-dev@hadoop.apache.org>;
Vinod
> Kumar Vavilapalli; Wangda tan
> >> Subject: Re: 2.7.2 release plan
> >>
> >> Thank you for reporting, Naganarasimha.
> >> Vinod and Wangda, I will help you to backport these changes.
> >>
> >> Best,
> >> - Tsuyoshi
> >>
> >> On Wed, Oct 28, 2015 at 2:57 PM, Naganarasimha G R (Naga)
> >> <garlanaganarasimha@huawei.com<mailto:garlanaganarasimha@huawei.com>>
wrote:
> >>> Hi Vinod, & Wangda
> >>>
> >>> I think it would be good to backport, following jira's related to
> NodeLabels as it will improve debug ability and usability of NodeLabels
> >>> --------------------------------
> >>> Key                     Summary
> >>> --------------------------------
> >>> YARN-4215       YARN-2492 RMNodeLabels Manager Need to verify and
> replace node labels for the only modified Node Label Mappings in the request
> >>> YARN-4162       YARN-2492 CapacityScheduler: Add resource usage by
> partition and queue capacity by partition to REST API
> >>> YARN-4140       YARN-2492 RM container allocation delayed incase of
> app submitted to Nodelabel partition
> >>> YARN-3717       YARN-2492 Expose app/am/queue's node-label-expression
> to RM web UI / CLI / REST-API
> >>> YARN-3647       YARN-2492 RMWebServices api's should use updated api
> from CommonNodeLabelsManager to get NodeLabel object
> >>> YARN-3593       YARN-2492 Add label-type and Improve
> "DEFAULT_PARTITION" in Node Labels Page
> >>> YARN-3583       YARN-2492 Support of NodeLabel object instead of plain
> String in YarnClient side.
> >>> YARN-3581       YARN-2492 Deprecate -directlyAccessNodeLabelStore in
> RMAdminCLI
> >>> YARN-3579       YARN-2492 CommonNodeLabelsManager should support
> NodeLabel instead of string label name when getting
> node-to-label/label-to-label mappings
> >>> YARN-3565       YARN-2492
> NodeHeartbeatRequest/RegisterNodeManagerRequest should use NodeLabel object
> instead of String
> >>> YARN-3521       YARN-2492 Support return structured NodeLabel objects
> in REST API
> >>> YARN-3362       YARN-2492 Add node label usage in RM CapacityScheduler
> web UI
> >>> YARN-3326       YARN-2492 Support RESTful API for getLabelsToNodes
> >>> YARN-3216       YARN-2492 Max-AM-Resource-Percentage should respect
> node labels
> >>> YARN-3136       YARN-3091 getTransferredContainers can be a bottleneck
> during AM registration
> >>>
> >>> Please inform if any support is required to backport them to 2.7.2
> >>>
> >>> Regards,
> >>> + Naga
> >>> ________________________________________
> >>> From: Kihwal Lee [kihwal@yahoo-inc.com.INVALID]
> >>> Sent: Tuesday, October 27, 2015 20:42
> >>> To: hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>;
common-dev@hadoop.apache.org<mailto:common-dev@hadoop.apache.org>
> >>> Cc: Chris Nauroth; yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org>;
> mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org>; Vinod
Kumar Vavilapalli; Ming Ma
> >>> Subject: Re: 2.7.2 release plan
> >>>
> >>> I think we need HDFS-8950 and HDFS-7725 in 2.7.2.It<http://2.7.2.It>
should be easy to
> backport/cherry-pick HDFS-7725. For HDFS-8950, it will be nice if Ming can
> chime in.
> >>> Kihwal
> >>>
> >>>      From: Tsuyoshi Ozawa <ozawa@apache.org<mailto:ozawa@apache.org>>
> >>> To: "common-dev@hadoop.apache.org<mailto:common-dev@hadoop.apache.org>"
<common-dev@hadoop.apache.org<mailto:common-dev@hadoop.apache.org>>
> >>> Cc: Chris Nauroth <cnauroth@hortonworks.com<mailto:cnauroth@hortonworks.com>>;
"
> yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org>" <yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org>>;
"
> hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>" <hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>>;
"
> mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org>" <mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org>>;
Vinod
> Kumar Vavilapalli <vinodkv@apache.org<mailto:vinodkv@apache.org>>
> >>> Sent: Tuesday, October 27, 2015 2:39 AM
> >>> Subject: Re: 2.7.2 release plan
> >>>
> >>> Vinod and Chris,
> >>>
> >>> Thanks for your reply. I'll do also backport not only bug fixes but
> >>> also documentations(I think 2.7.2 includes them). It helps users a lot.
> >>>
> >>> Best,
> >>> - Tsuyoshi
> >>>
> >>> On Tuesday, 27 October 2015, Vinod Vavilapalli <
> vinodkv@hortonworks.com<mailto:vinodkv@hortonworks.com>>
> >>> wrote:
> >>>
> >>>> +1.
> >>>>
> >>>> Thanks
> >>>> +Vinod
> >>>>
> >>>>> On Jul 16, 2015, at 8:18 AM, Chris Nauroth <cnauroth@hortonworks.com<mailto:cnauroth@hortonworks.com>
> >>>> <javascript:;>> wrote:
> >>>>>
> >>>>> I'd be comfortable with inclusion of any doc-only patch in minor
> >>>> releases.
> >>>>> There is a lot of value to end users in pushing documentation fixes
> as
> >>>>> quickly as possible, and they don't bear the same risk of
> regressions or
> >>>>> incompatibilities as code changes.
> >>>>>
> >>>>> --Chris Nauroth
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 7/16/15, 12:38 AM, "Tsuyoshi Ozawa" <ozawa@apache.org<mailto:ozawa@apache.org>
> <javascript:;>>
> >>>> wrote:
> >>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> thank you for starting the discussion about 2.7.2 release.
> >>>>>>
> >>>>>>> The focus obviously is to have blocker issues [2], bug-fixes
and
> *no*
> >>>>>> features / improvements.
> >>>>>>
> >>>>>> I've committed YARN-3170, which is an improvement of documentation.
> I
> >>>>>> thought documentation pages which can be fit into branch-2.7
can be
> >>>>>> included easily. Should I revert it?
> >>>>>>
> >>>>>>>> I need help from all committers in automatically
> >>>>>> merging in any patch that fits the above criterion into 2.7.2
> instead of
> >>>>>> only on trunk or 2.8.
> >>>>>>
> >>>>>> Sure, I'll try my best.
> >>>>>>
> >>>>>>> That way we can include not only blocker but also critical
bug
> fixes to
> >>>>>>> 2.7.2 release.
> >>>>>>
> >>>>>> As Vinod mentioned, we should also apply major bug fixes into
> >>>> branch-2.7.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> - Tsuyoshi
> >>>>>>
> >>>>>> On Thu, Jul 16, 2015 at 3:52 PM, Akira AJISAKA
> >>>>>> <ajisakaa@oss.nttdata.co.jp<mailto:ajisakaa@oss.nttdata.co.jp>
<javascript:;>> wrote:
> >>>
> >>>
> >>>>>>> Thanks Vinod for starting 2.7.2 release plan.
> >>>>>>>
> >>>>>>>> The focus obviously is to have blocker issues [2], bug-fixes
and
> *no*
> >>>>>>>> features / improvements.
> >>>>>>>
> >>>>>>> Can we adopt the plan as Karthik mentioned in "Additional
> maintenance
> >>>>>>> releases for Hadoop 2.y versions" thread? That way we can
include
> not
> >>>>>>> only
> >>>>>>> blocker but also critical bug fixes to 2.7.2 release.
> >>>>>>>
> >>>>>>> In addition, branch-2.7 is a special case. (2.7.1 is the
first
> stable
> >>>>>>> release) Therefore I'm thinking we can include major bug
fixes as
> well.
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>> Akira
> >>>>>>>
> >>>>>>>
> >>>>>>> On 7/16/15 04:13, Vinod Kumar Vavilapalli wrote:
> >>>>>>>>
> >>>>>>>> Hi all,
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Thanks everyone for the push on 2.7.1! Branch-2.7 is
now open for
> >>>>>>>> commits
> >>>>>>>> to a 2.7.2 release. JIRA also now has a 2.7.2 version
for all the
> >>>>>>>> sub-projects.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Continuing the previous 2.7.1 thread on steady maintenance
> releases
> >>>>>>>> [1],
> >>>>>>>> we
> >>>>>>>> should follow up 2.7.1 with a 2.7.2 within 4 weeks.
Earlier I
> tried a
> >>>>>>>> 2-3
> >>>>>>>> week cycle for 2.7.1, but it seems to be impractical
given the
> >>>>>>>> community
> >>>>>>>> size. So, I propose we target a release by the end for
4 weeks
> from
> >>>>>>>> now,
> >>>>>>>> starting the release close-down within 2-3 weeks.
> >>>>>>>>
> >>>>>>>> The focus obviously is to have blocker issues [2], bug-fixes
and
> *no*
> >>>>>>>> features / improvements. I need help from all committers
in
> >>>>>>>> automatically
> >>>>>>>> merging in any patch that fits the above criterion into
2.7.2
> instead
> >>>>>>>> of
> >>>>>>>> only on trunk or 2.8.
> >>>>>>>>
> >>>>>>>> Thoughts?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>>
> >>>>>>>> +Vinod
> >>>>>>>>
> >>>>>>>> [1] A 2.7.1 release to follow up 2.7.0
> >>>>>>>> http://markmail.org/message/zwzze6cqqgwq4rmw
> >>>>>>>>
> >>>>>>>> [2] 2.7.2 release blockers:
> >>>>>>>> https://issues.apache.org/jira/issues/?filter=12332867
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>>
> >>
> >
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message