hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhihai Xu <zhihaixu2...@gmail.com>
Subject Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Date Tue, 21 Mar 2017 19:55:25 GMT
Thanks Junping for creating release 2.8.0 (RC3).

+1 (non-binding)

* Downloaded and built from source
* Verified md5 checksums and signature
* Deployed a pseudo cluster
* verified basic HDFS operations and Pi job.
* Did a sanity check for RM and NM UI.

Thanks

zhihai

On Tue, Mar 21, 2017 at 12:02 PM, Steve Loughran <stevel@hortonworks.com>
wrote:

>
> I'm an =0 I'm afraid
>
> I think those extra files need to be pulled which triggers a repackage, at
> which point there's a couple of
>
> HADOOP-14205 No FileSystem for scheme: adl
> HADOOP-14204 S3A multipart commit failing,
>
> of the two, the ADL FS one is the most significant, though if there was
> going to be a re-roll, I'd like them in.
>
> I also worry about windows test coverage, as irrespective of whether
> people are planning to run it on a windows host in production, the hadoop
> libs get used client side in job submit, and more now in Spark .
> HADOOP-14201 showed up that (a) my new VM is a mess, so complicating
> testing and (b) gzip was failing. I Worry about gzip, as that could be a
> sign of native lib problems.. someone needs to look at it a bit more.
> However, I don't think it should really be a blocker, given the ASF doesn't
> officially redistribute windows binaries.
>
> so: some issues with packaging and a couple of patches which need to go
> in. Test coverage on windows is potentially longer, and I don't want it to
> hold things up.
>
> Anyhow: =0. If everyone else is happy to ship, then let's get it out the
> door, as that's the only way that we'll get all the in-the-field bugreps
> coming back in. We will need to release some new versions in the coming
> weeks to handle those
>
> -Steve
>
>
>
>
> > On 21 Mar 2017, at 18:43, Haibo Chen <haibochen@cloudera.com> wrote:
> >
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jdu@hortonworks.com>
> wrote:
> >
> >> ?Thanks for update, John. Then we should be OK with fixing this issue in
> >> 2.8.1.
> >>
> >> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> >> up to blocker in case we could miss this in releasing 2.8.1. :)
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jzhuge@cloudera.com>
> >> Sent: Monday, March 20, 2017 10:31 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Yes, it only affects ADL. There is a workaround of adding these 2
> >> properties to core-site.xml:
> >>
> >>  <property>
> >>    <name>fs.adl.impl</name>
> >>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >>  </property>
> >>
> >>  <property>
> >>    <name>fs.AbstractFileSystem.adl.impl</name>
> >>    <value>org.apache.hadoop.fs.adl.Adl</value>
> >>  </property>
> >>
> >> I have the initial patch ready but hitting these live unit test
> failures:
> >>
> >> Failed tests:
> >>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> >> expected:<1> but was:<10>
> >>
> >> Tests in error:
> >>  TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >>
> >>
> >> Stay tuned...
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> >> jdu@hortonworks.com>> wrote:
> >>
> >> Thank you for reporting the issue, John! Does this issue only affect ADL
> >> (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> >> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> >> regression and just a new feature get broken.?
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jzhuge@cloudera.com<mailto:jzhuge@cloudera.com>>
> >> Sent: Monday, March 20, 2017 9:07 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org<mailto:common-dev@hadoop.apache.org>;
> >> hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>;
> >> yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org>;
> >> mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org>
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >> FileSystem for scheme: adl".
> >>
> >> The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> >> HADOOP-12666 should not be backported, but some changes are needed:
> >> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >>
> >> I am working on a patch.
> >>
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com
> <mailto:jd
> >> u@hortonworks.com>> wrote:
> >> Hi all,
> >>     With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>     This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~
> >> junping_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org
> <http://
> >> repository.apache.org> at: https://repository.apache.org/
> >> content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message