hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: 2.7.3 release plan
Date Fri, 01 Apr 2016 18:32:41 GMT
As a downstream consumer of Apache Hadoop 2.7.x releases, I expect we would
patch the release to revert HDFS-8791 before pushing it out to production.
For what it's worth.


On Fri, Apr 1, 2016 at 11:23 AM, Andrew Wang <andrew.wang@cloudera.com>
wrote:

> One other thing I wanted to bring up regarding HDFS-8791, we haven't
> backported the parallel DN upgrade improvement (HDFS-8578) to branch-2.6.
> HDFS-8578 is a very important related fix since otherwise upgrade will be
> very slow.
>
> On Thu, Mar 31, 2016 at 10:35 AM, Andrew Wang <andrew.wang@cloudera.com>
> wrote:
>
> > As I expressed on HDFS-8791, I do not want to include this JIRA in a
> > maintenance release. I've only seen it crop up on a handful of our
> > customer's clusters, and large users like Twitter and Yahoo that seem to
> be
> > more affected are also the most able to patch this change in themselves.
> >
> > Layout upgrades are quite disruptive, and I don't think it's worth
> > breaking upgrade and downgrade expectations when it doesn't affect the
> (in
> > my experience) vast majority of users.
> >
> > Vinod seemed to have a similar opinion in his comment on HDFS-8791, but
> > will let him elaborate.
> >
> > Best,
> > Andrew
> >
> > On Thu, Mar 31, 2016 at 9:11 AM, Sean Busbey <busbey@cloudera.com>
> wrote:
> >
> >> As of 2 days ago, there were already 135 jiras associated with 2.7.3,
> >> if *any* of them end up introducing a regression the inclusion of
> >> HDFS-8791 means that folks will have cluster downtime in order to back
> >> things out. If that happens to any substantial number of downstream
> >> folks, or any particularly vocal downstream folks, then it is very
> >> likely we'll lose the remaining trust of operators for rolling out
> >> maintenance releases. That's a pretty steep cost.
> >>
> >> Please do not include HDFS-8791 in any 2.6.z release. Folks having to
> >> be aware that an upgrade from e.g. 2.6.5 to 2.7.2 will fail is an
> >> unreasonable burden.
> >>
> >> I agree that this fix is important, I just think we should either cut
> >> a version of 2.8 that includes it or find a way to do it that gives an
> >> operational path for rolling downgrade.
> >>
> >> On Thu, Mar 31, 2016 at 10:10 AM, Junping Du <jdu@hortonworks.com>
> wrote:
> >> > Thanks for bringing up this topic, Sean.
> >> > When I released our latest Hadoop release 2.6.4, the patch of
> HDFS-8791
> >> haven't been committed in so that's why we didn't discuss this earlier.
> >> > I remember in JIRA discussion, we treated this layout change as a
> >> Blocker bug that fixing a significant performance regression before but
> not
> >> a normal performance improvement. And I believe HDFS community already
> did
> >> their best with careful and patient to deliver the fix and other related
> >> patches (like upgrade fix in HDFS-8578). Take an example of HDFS-8578,
> you
> >> can see 30+ rounds patch review back and forth by senior committers,
> not to
> >> mention the outstanding performance test data in HDFS-8791.
> >> > I would trust our HDFS committers' judgement to land HDFS-8791 on
> >> 2.7.3. However, that needs Vinod's final confirmation who serves as RM
> for
> >> branch-2.7. In addition, I didn't see any blocker issue to bring it into
> >> 2.6.5 now.
> >> > Just my 2 cents.
> >> >
> >> > Thanks,
> >> >
> >> > Junping
> >> >
> >> > ________________________________________
> >> > From: Sean Busbey <busbey@cloudera.com>
> >> > Sent: Thursday, March 31, 2016 2:57 PM
> >> > To: hdfs-dev@hadoop.apache.org
> >> > Cc: Hadoop Common; yarn-dev@hadoop.apache.org;
> >> mapreduce-dev@hadoop.apache.org
> >> > Subject: Re: 2.7.3 release plan
> >> >
> >> > A layout change in a maintenance release sounds very risky. I saw some
> >> > discussion on the JIRA about those risks, but the consensus seemed to
> >> > be "we'll leave it up to the 2.6 and 2.7 release managers." I thought
> >> > we did RMs per release rather than per branch? No one claiming to be a
> >> > release manager ever spoke up AFAICT.
> >> >
> >> > Should this change be included? Should it go into a special 2.8
> >> > release as mentioned in the ticket?
> >> >
> >> > On Thu, Mar 31, 2016 at 1:45 AM, Akira AJISAKA
> >> > <ajisakaa@oss.nttdata.co.jp> wrote:
> >> >> Thank you Vinod!
> >> >>
> >> >> FYI: 2.7.3 will be a bit special release.
> >> >>
> >> >> HDFS-8791 bumped up the datanode layout version,
> >> >> so rolling downgrade from 2.7.3 to 2.7.[0-2]
> >> >> is impossible. We can rollback instead.
> >> >>
> >> >> https://issues.apache.org/jira/browse/HDFS-8791
> >> >>
> >>
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html
> >> >>
> >> >> Regards,
> >> >> Akira
> >> >>
> >> >>
> >> >> On 3/31/16 08:18, Vinod Kumar Vavilapalli wrote:
> >> >>>
> >> >>> Hi all,
> >> >>>
> >> >>> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go
out
> >> (which
> >> >>> did go out mid February). Got a little busy since.
> >> >>>
> >> >>> Following up the 2.7.2 maintenance release, we should work towards
a
> >> >>> 2.7.3. The focus obviously is to have blocker issues [1], bug-fixes
> >> and *no*
> >> >>> features / improvements.
> >> >>>
> >> >>> I hope to cut an RC in a week - giving enough time for outstanding
> >> blocker
> >> >>> / critical issues. Will start moving out any tickets that are not
> >> blockers
> >> >>> and/or won’t fit the timeline - there are 3 blockers and 15 critical
> >> tickets
> >> >>> outstanding as of now.
> >> >>>
> >> >>> Thanks,
> >> >>> +Vinod
> >> >>>
> >> >>> [1] 2.7.3 release blockers:
> >> >>> https://issues.apache.org/jira/issues/?filter=12335343
> >> >>>
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > busbey
> >>
> >>
> >>
> >> --
> >> busbey
> >>
> >
> >
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message