Mailing-List: contact yarn-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
MIME-Version: 1.0
In-Reply-To: <62EB0A59-7039-486B-8D60-7C617D1E2A97@hortonworks.com>
References: <D0B2535D-454F-490D-ACF2-FAE5E2760596@apache.org>
 <1500582711718.13957@hortonworks.com> <1358004998.2062665.1500649275418@mail.yahoo.com>
 <1199503934.2073180.1500650233097@mail.yahoo.com> <1500661426047.38906@hortonworks.com>
 <26719281-DA04-4B63-84E0-D3CA52CEF9B5@apache.org> <1500672545427.89754@hortonworks.com>
 <1500674278486.28606@hortonworks.com> <CO1PR18MB0871E49257FFDD6E5D312171E1BA0@CO1PR18MB0871.namprd18.prod.outlook.com>
 <1500917796382.42128@hortonworks.com> <1500928876650.88617@hortonworks.com>
 <1502088277103.98878@hortonworks.com> <CO1PR18MB087183B95F3EECE45F3D83DCE19F0@CO1PR18MB0871.namprd18.prod.outlook.com>
 <1504078665025.24719@hortonworks.com> <1504261027568.37942@hortonworks.com> <62EB0A59-7039-486B-8D60-7C617D1E2A97@hortonworks.com>
From: larry mccay <lmccay@apache.org>
Date: Fri, 1 Sep 2017 13:55:24 -0400
Message-ID: <CACRbFyhi4AkYgq_9nP6PN5G1WrJyhMy_ccR_5PQ59JsXFykQCA@mail.gmail.com>
Subject: Re: Apache Hadoop 2.8.2 Release Plan
To: Steve Loughran <stevel@hortonworks.com>
Cc: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>,
	"hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>,
	"mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org>,
	"yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>
Content-Type: multipart/alternative; boundary="001a11444128344278055824776e"
archived-at: Fri, 01 Sep 2017 17:55:33 -0000

--001a11444128344278055824776e
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

If we do "fix" this in 2.8.2 we should seriously consider not doing so in
3.0.
This is a very poor practice.

I can see an argument for backward compatibility in 2.8.x line though.

On Fri, Sep 1, 2017 at 1:41 PM, Steve Loughran <stevel@hortonworks.com>
wrote:

> One thing we need to consider is
>
> HADOOP-14439: regression: secret stripping from S3x URIs breaks some
> downstream code
>
> Hadoop 2.8 has a best-effort attempt to strip out secrets from the
> toString() value of an s3a or s3n path where someone has embedded them in
> the URI; this has caused problems in some uses, specifically: when people
> use secrets this way (bad) and assume that you can round trip paths to
> string and back
>
> Should we fix this? If so, Hadoop 2.8.2 is the time to do it
>
>
> > On 1 Sep 2017, at 11:14, Junping Du <jdu@hortonworks.com> wrote:
> >
> > HADOOP-14814 get committed and HADOOP-9747 get push out to 2.8.3, so we
> are clean on blocker/critical issues now.
> > I finish practice of going through JACC report and no more incompatible
> public API changes get found between 2.8.2 and 2.7.4. Also I check commit
> history and fixed 10+ commits which are missing from branch-2.8.2 for som=
e
> reason. So, the current branch-2.8.2 should be good to go for RC stage, a=
nd
> I will kick off our first RC tomorrow.
> > In the meanwhile, please don't land any commits to branch-2.8.2 since
> now. If some issues really belong to blocker, please ping me on the JIRA
> before doing any commits. branch-2.8 is still open for landing. Thanks fo=
r
> your cooperation!
> >
> >
> > Thanks,
> >
> > Junping
> >
> > ________________________________________
> > From: Junping Du <jdu@hortonworks.com>
> > Sent: Wednesday, August 30, 2017 12:35 AM
> > To: Brahma Reddy Battula; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org
> > Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >
> > Thanks Brahma for comment on this thread. To be clear, I always update
> branch version just before RC kicking off.
> >
> > For 2.8.2 release, I don't have plan to involve big top or other
> third-party test tools. As always, we will rely on test/verify efforts fr=
om
> community especially from large deployed production cluster - as far as I
> know,  there are already several companies. like: Yahoo!, Alibaba, etc.
> already deploy 2.8 release in large production clusters for months which
> give me more confidence on 2.8.2.
> >
> >
> > Here is more update on 2.8.2 release:
> >
> > Blocker issues:
> >
> >    -  A new blocker YARN-7076 get reported and fixed by Jian He through
> last weekend.
> >
> >    -  Another new blocker - HADOOP-14814 get identified from my latest
> jdiff run against 2.7.4. The simple fix on an incompatible API change
> should get commit soon.
> >
> >
> > Critical issues:
> >
> >    -  YARN-7083 already get committed. Thanks Jason for reporting the
> issue and delivering the fix.
> >
> >    -  YARN-6091 get push out from 2.8.2 as issue is not a regression an=
d
> pending for a while.
> >
> >    -  Daryn is actively working on HADOOP-9747 for a while, and the
> patch are getting close to be committed. However, according to Daryn, the
> patch seems to cause some regression in some corner cases in secured
> environment (Storm auto tgt, etc.). May need some additional watch/review
> on this JIRA's fixes.
> >
> >
> >
> > My monitoring JACC report between 2.8.2 and 2.7.4 will get finish
> tomorrow. If everything is going smoothly, I am planning to kick off RC0
> around holiday (this weekend).
> >
> >
> >
> > Thanks,
> >
> >
> >
> > =E2=80=8BJunping
> >
> >
> >
> > ________________________________
> > From: Brahma Reddy Battula <brahmareddy.battula@hotmail.com>
> > Sent: Tuesday, August 29, 2017 8:42 AM
> > To: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.or=
g;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> > Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >
> >
> > Hi All
> >
> > Update on 2.8.2 release status
> > we are down to 3 critical issues ( YARN-6091,YARN-7083,HADOOP-9747),all
> are patch available and closer to commit.
> > Junping is closing tracking this.
> >
> > Todo:
> >
> > 1) Update pom.xml ..?  currently it's with 2.8.3
> > https://github.com/apache/hadoop/blob/branch-2.8.2/pom.xml#L21
> > <https://github.com/apache/hadoop/blob/branch-2.8.2/pom.xml#L21>2) Wiki
> is outdated, need to update the wiki..?
> > 3) As this is going to stable release,are we planing enable Big top for
> 2.8.2 testing Or Dynamometer testing (anybody from linked-in can help)..?
> >
> > @Junping Du<mailto:jdu@hortonworks.com>,Please correct me,if I am wrong=
.
> >
> >
> > --Brahma Reddy Battula
> > ________________________________
> > From: Junping Du <jdu@hortonworks.com>
> > Sent: Monday, August 7, 2017 2:44 PM
> > To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> > Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >
> > Hello community,
> >    Here is a quick update on status for 2.8.2:
> >    - We are 0 blockers now!
> >    - Still 9 critical issues, 8 of them are Patch Available and with
> actively working.
> >    For details of pending blocker/critical issues, please refer:
> https://s.apache.org/JM5x
> > Issue Navigator - ASF JIRA<https://s.apache.org/JM5x>
> > s.apache.org
> > Linked Applications. Loading=E2=80=A6 Dashboards
> >
> >
> >
> >
> >    I am planning to cut off first RC in week of Aug. 21st to give these
> critical issues a bit more time (~2 weeks) to get fixed. Let's working
> towards first production GA release of Apache Hadoop 2.8 - let me know if
> you have any thoughts or comments.
> >
> > Cheers,
> >
> > Junping
> > ________________________________________
> > From: Junping Du <jdu@hortonworks.com>
> > Sent: Monday, July 24, 2017 1:41 PM
> > To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> > Subject: Re:
> >
> > I have done the change.
> >
> > All committers,
> >
> >      2.8.2 release is supposed to be a stable/production release for
> branch-2.8. For commits to go for 2.8.2 release (only important and low
> risk bug fixes), please commit to trunk, branch-2, branch-2.8 and
> branch-2.8.2. For unimportant or high risk bug fixes/improvements, please
> commit to branch-2.8 (trunk/branch-2) only and mark JIRA fixed as 2.8.3.
> Thanks for your cooperation!
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Junping Du
> > Sent: Monday, July 24, 2017 10:36 AM
> > To: Brahma Reddy Battula; Vinod Kumar Vavilapalli
> > Cc: Kihwal Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.or=
g;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Jason Lowe;
> humbedooh@apache.org
> > Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >
> >
> > Nice catch, Brahma. Actually, this is not supposed to be happen as we
> all should know the patch should firstly get landed on branch-2.8 (as wel=
l
> as trunk, branch-2) before landed on branch-2.8.2. Anyway, we will always
> check JIRAs claim to fixed in a specific release version with commits
> actually landing on the releasing branch before kicking off RC. So, I am
> not too worry about this mistaking behaviors as it happens in every
> releases.
> >
> > If no other concerns, I will do the branch update in next 30 minutes.
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Brahma Reddy Battula <brahmareddy.battula@hotmail.com>
> > Sent: Sunday, July 23, 2017 1:50 AM
> > To: Junping Du; Vinod Kumar Vavilapalli
> > Cc: Kihwal Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.or=
g;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Jason Lowe;
> humbedooh@apache.org
> > Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >
> >
> >
> > Just executed the "git log branch-2.8 ^branch-2.8.2" found two commits
> are missed (HDFS-8312 and HADOOP-13867 ) in branch-2.8.I just pushed this
> two commits.Hope we'll not miss any commits which present in only in
> branch-2.8.2.
> >
> > ________________________________
> > From: Junping Du <jdu@hortonworks.com>
> > Sent: Saturday, July 22, 2017 5:57 AM
> > To: Vinod Kumar Vavilapalli
> > Cc: Kihwal Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.or=
g;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Jason Lowe;
> humbedooh@apache.org
> > Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >
> > Already get back from Daniel who is from ASF INFRA team, I plan to do
> following operations on next Monday morning:
> > 1. Drop current branch-2.8.2 and recut branch-2.8.2 from branch-2.8
> > 2. Drop abandoned branch-2.8.1 and rename branch-2.8.1-private to
> branch-2.8.1 where we just released 2.8.1 from.
> > I will also adjust fix version on all affected JIRA accordingly.
> >
> > If you have any concerns on above operations, please raise it before th=
e
> end of this Sunday (7/23).
> >
> >
> > Thanks,
> >
> > Junping
> >
> > ________________________________________
> > From: Junping Du <jdu@hortonworks.com>
> > Sent: Friday, July 21, 2017 2:29 PM
> > To: Vinod Kumar Vavilapalli
> > Cc: Kihwal Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.or=
g;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Jason Lowe
> > Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >
> > Make sense, just raise: https://issues.apache.org/
> jira/browse/INFRA-14669
> > [INFRA-14669] Delete several abandoned branches - ASF JIRA<
> https://issues.apache.org/jira/browse/INFRA-14669>
> > issues.apache.org
> > Linked Applications. Loading=E2=80=A6 Dashboards
> >
> >
> >
> >
> > Thanks,
> >
> > Junping
> > ________________________________________
> > From: Vinod Kumar Vavilapalli <vinodkv@apache.org>
> > Sent: Friday, July 21, 2017 12:31 PM
> > To: Junping Du
> > Cc: Kihwal Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.or=
g;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Jason Lowe
> > Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >
> > Junping,
> >
> > If we are looking at a month, I=E2=80=99d not rebranch branch-2.8.2 rig=
ht now
> given how these things go. We can just continue to commit on branch-2.8 f=
or
> now.
> >
> > I also think we should just follow up with ASF INFRA and clean up the
> branches
> > - Delete branch-2.8.2 so that we can recreate it afresh a little later.
> > - branch-2.8.1 is also stale and it should be deleted.
> branch-2.8.1-private should be renamed to branch-2.8.1
> >
> > Thanks
> > +Vinod
> >
> >> On Jul 21, 2017, at 11:23 AM, Junping Du <jdu@hortonworks.com> wrote:
> >>
> >> Thanks for suggestions, Jason and Kihwal!
> >> +1 on releasing 2.8.2 on latest branch-2.8 too. Practically, if
> branch-2.8.2 cannot be abandoned/replaced (suspect all branches are
> read-only now), I will manually merge all commits that not landed on 2.8.=
2
> yet.
> >>
> >> Thanks,
> >>
> >> Junping
> >> ________________________________________
> >> From: Jason Lowe <jlowe@yahoo-inc.com.INVALID>
> >> Sent: Friday, July 21, 2017 8:17 AM
> >> To: Kihwal Lee; Junping Du; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org
> >> Subject: Re: Apache Hadoop 2.8.2 Release Plan
> >>
> >> +1 to base the 2.8.2 release off of the more recent activity on
> branch-2.8.  Because branch-2.8.2 was cut so long ago it is missing a lot
> of fixes that are in branch-2.8.  There also are a lot of JIRAs that clai=
m
> they are fixed in 2.8.2 but are not in branch-2.8.2.  Having the 2.8.2
> release be based on recent activity in branch-2.8 would solve both of the=
se
> issues, and we'd only need to move the handful of JIRAs that have marked
> themselves correctly as fixed in 2.8.3 to be fixed in 2.8.2.
> >>
> >> Jason
> >>
> >>
> >>   On Friday, July 21, 2017 10:01 AM, Kihwal Lee
> <kihwal@yahoo-inc.com.INVALID> wrote:
> >>
> >>
> >> Thanks for driving the next 2.8 release, Junping. While I was
> committing a blocker for 2.7.4, I noticed some of the jiras are back-port=
ed
> to 2.7, but missing in branch-2.8.2.  Perhaps it is safer and easier to
> simply rebranch 2.8.2.
> >> Thanks,Kihwal
> >>
> >> On Thursday, July 20, 2017, 3:32:16 PM CDT, Junping Du <
> jdu@hortonworks.com> wrote:
> >>
> >> Hi all,
> >>   Per Vinod's previous email, we just announce Apache Hadoop 2.8.1 get
> released today which is a special security release. Now, we should work
> towards 2.8.2 release which aim for production deployment. The focus
> obviously is to fix blocker/critical issues [2], bug-fixes and *no*
> features / improvements. We currently have 13 blocker/critical issues, an=
d
> 10 of them are Patch Available.
> >>
> >> I plan to cut an RC in a month - target for releasing before end of
> Aug., to give enough time for outstanding blocker / critical issues. Will
> start moving out any tickets that are not blockers and/or won't fit the
> timeline. For progress of releasing effort, please refer our release wiki
> [2].
> >>
> >> Please share thoughts if you have any. Thanks!
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >> [1] 2.8.2 release Blockers/Criticals: https://s.apache.org/JM5x
> > Issue Navigator - ASF JIRA<https://s.apache.org/JM5x>
> > s.apache.org
> > Linked Applications. Loading=E2=80=A6 Dashboards
> >
> >
> >
> >> [2] 2.8 Release wiki: https://cwiki.apache.org/
> confluence/display/HADOOP/Hadoop+2.8+Release
> > Hadoop 2.8 Release - Hadoop - Apache Software Foundation<https://cwiki.
> apache.org/confluence/display/HADOOP/Hadoop+2.8+Release>
> > cwiki.apache.org
> > Release schedule. It has been over 1 year since last minor release
> 2.7.0, so we plan to release Hadoop 2.8.0 as soon as we can. Per
> discussions on community threads ...
> >
> >
> >
> > Hadoop 2.8 Release - Hadoop - Apache Software Foundation<https://cwiki.
> apache.org/confluence/display/HADOOP/Hadoop+2.8+Release>
> > Hadoop 2.8 Release - Hadoop - Apache Software Foundation<https://cwiki.
> apache.org/confluence/display/HADOOP/Hadoop+2.8+Release>
> > cwiki.apache.org
> > Release schedule. It has been over 1 year since last minor release
> 2.7.0, so we plan to release Hadoop 2.8.0 as soon as we can. Per
> discussions on community threads ...
> >
> >
> >
> > cwiki.apache.org
> > Release schedule. It has been over 1 year since last minor release
> 2.7.0, so we plan to release Hadoop 2.8.0 as soon as we can. Per
> discussions on community threads ...
> >
> >
> >
> >>
> >> ________________________________________
> >> From: Vinod Kumar Vavilapalli <vinodkv@apache.org>
> >> Sent: Thursday, July 20, 2017 1:05 PM
> >> To: general@hadoop.apache.org
> >> Subject: [ANNOUNCE] Apache Hadoop 2.8.1 is released
> >>
> >> Hi all,
> >>
> >> The Apache Hadoop PMC has released version 2.8.1. You can get it from
> this page: http://hadoop.apache.org/releases.html#Download
> > Apache Hadoop Releases<http://hadoop.apache.org/releases.html#Download>
> > hadoop.apache.org
> > To verify Hadoop releases using GPG: Download the release
> hadoop-X.Y.Z-src.tar.gz from a mirror site. Download the signature file
> hadoop-X.Y.Z-src.tar ...
> >
> >
> >
> > Apache Hadoop Releases<http://hadoop.apache.org/releases.html#Download>
> > Apache Hadoop Releases<http://hadoop.apache.org/releases.html#Download>
> > hadoop.apache.org
> > To verify Hadoop releases using GPG: Download the release
> hadoop-X.Y.Z-src.tar.gz from a mirror site. Download the signature file
> hadoop-X.Y.Z-src.tar ...
> >
> >
> >
> > hadoop.apache.org
> > To verify Hadoop releases using GPG: Download the release
> hadoop-X.Y.Z-src.tar.gz from a mirror site. Download the signature file
> hadoop-X.Y.Z-src.tar ...
> >
> >
> >
> >> This is a security release in the 2.8.0 release line. It consists of
> 2.8.0 plus security fixes. Users on 2.8.0 are encouraged to upgrade to
> 2.8.1.
> >>
> >> Please note that 2.8.x release line continues to be not yet ready for
> production use. Critical issues are being ironed out via testing and
> downstream adoption. Production users should wait for a subsequent releas=
e
> in the 2.8.x line.
> >>
> >> Thanks
> >> +Vinod
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> >> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
> >>
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> >>
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>
>

--001a11444128344278055824776e--