hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Wang <andrew.w...@cloudera.com>
Subject Re: 2.7.3 release plan
Date Fri, 01 Apr 2016 18:23:28 GMT
One other thing I wanted to bring up regarding HDFS-8791, we haven't
backported the parallel DN upgrade improvement (HDFS-8578) to branch-2.6.
HDFS-8578 is a very important related fix since otherwise upgrade will be
very slow.

On Thu, Mar 31, 2016 at 10:35 AM, Andrew Wang <andrew.wang@cloudera.com>
wrote:

> As I expressed on HDFS-8791, I do not want to include this JIRA in a
> maintenance release. I've only seen it crop up on a handful of our
> customer's clusters, and large users like Twitter and Yahoo that seem to be
> more affected are also the most able to patch this change in themselves.
>
> Layout upgrades are quite disruptive, and I don't think it's worth
> breaking upgrade and downgrade expectations when it doesn't affect the (in
> my experience) vast majority of users.
>
> Vinod seemed to have a similar opinion in his comment on HDFS-8791, but
> will let him elaborate.
>
> Best,
> Andrew
>
> On Thu, Mar 31, 2016 at 9:11 AM, Sean Busbey <busbey@cloudera.com> wrote:
>
>> As of 2 days ago, there were already 135 jiras associated with 2.7.3,
>> if *any* of them end up introducing a regression the inclusion of
>> HDFS-8791 means that folks will have cluster downtime in order to back
>> things out. If that happens to any substantial number of downstream
>> folks, or any particularly vocal downstream folks, then it is very
>> likely we'll lose the remaining trust of operators for rolling out
>> maintenance releases. That's a pretty steep cost.
>>
>> Please do not include HDFS-8791 in any 2.6.z release. Folks having to
>> be aware that an upgrade from e.g. 2.6.5 to 2.7.2 will fail is an
>> unreasonable burden.
>>
>> I agree that this fix is important, I just think we should either cut
>> a version of 2.8 that includes it or find a way to do it that gives an
>> operational path for rolling downgrade.
>>
>> On Thu, Mar 31, 2016 at 10:10 AM, Junping Du <jdu@hortonworks.com> wrote:
>> > Thanks for bringing up this topic, Sean.
>> > When I released our latest Hadoop release 2.6.4, the patch of HDFS-8791
>> haven't been committed in so that's why we didn't discuss this earlier.
>> > I remember in JIRA discussion, we treated this layout change as a
>> Blocker bug that fixing a significant performance regression before but not
>> a normal performance improvement. And I believe HDFS community already did
>> their best with careful and patient to deliver the fix and other related
>> patches (like upgrade fix in HDFS-8578). Take an example of HDFS-8578, you
>> can see 30+ rounds patch review back and forth by senior committers, not to
>> mention the outstanding performance test data in HDFS-8791.
>> > I would trust our HDFS committers' judgement to land HDFS-8791 on
>> 2.7.3. However, that needs Vinod's final confirmation who serves as RM for
>> branch-2.7. In addition, I didn't see any blocker issue to bring it into
>> 2.6.5 now.
>> > Just my 2 cents.
>> >
>> > Thanks,
>> >
>> > Junping
>> >
>> > ________________________________________
>> > From: Sean Busbey <busbey@cloudera.com>
>> > Sent: Thursday, March 31, 2016 2:57 PM
>> > To: hdfs-dev@hadoop.apache.org
>> > Cc: Hadoop Common; yarn-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org
>> > Subject: Re: 2.7.3 release plan
>> >
>> > A layout change in a maintenance release sounds very risky. I saw some
>> > discussion on the JIRA about those risks, but the consensus seemed to
>> > be "we'll leave it up to the 2.6 and 2.7 release managers." I thought
>> > we did RMs per release rather than per branch? No one claiming to be a
>> > release manager ever spoke up AFAICT.
>> >
>> > Should this change be included? Should it go into a special 2.8
>> > release as mentioned in the ticket?
>> >
>> > On Thu, Mar 31, 2016 at 1:45 AM, Akira AJISAKA
>> > <ajisakaa@oss.nttdata.co.jp> wrote:
>> >> Thank you Vinod!
>> >>
>> >> FYI: 2.7.3 will be a bit special release.
>> >>
>> >> HDFS-8791 bumped up the datanode layout version,
>> >> so rolling downgrade from 2.7.3 to 2.7.[0-2]
>> >> is impossible. We can rollback instead.
>> >>
>> >> https://issues.apache.org/jira/browse/HDFS-8791
>> >>
>> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html
>> >>
>> >> Regards,
>> >> Akira
>> >>
>> >>
>> >> On 3/31/16 08:18, Vinod Kumar Vavilapalli wrote:
>> >>>
>> >>> Hi all,
>> >>>
>> >>> Got nudged about 2.7.3. Was previously waiting for 2.6.4 to go out
>> (which
>> >>> did go out mid February). Got a little busy since.
>> >>>
>> >>> Following up the 2.7.2 maintenance release, we should work towards a
>> >>> 2.7.3. The focus obviously is to have blocker issues [1], bug-fixes
>> and *no*
>> >>> features / improvements.
>> >>>
>> >>> I hope to cut an RC in a week - giving enough time for outstanding
>> blocker
>> >>> / critical issues. Will start moving out any tickets that are not
>> blockers
>> >>> and/or won’t fit the timeline - there are 3 blockers and 15 critical
>> tickets
>> >>> outstanding as of now.
>> >>>
>> >>> Thanks,
>> >>> +Vinod
>> >>>
>> >>> [1] 2.7.3 release blockers:
>> >>> https://issues.apache.org/jira/issues/?filter=12335343
>> >>>
>> >>
>> >
>> >
>> >
>> > --
>> > busbey
>>
>>
>>
>> --
>> busbey
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message