hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Iñigo Goiri <elgo...@gmail.com>
Subject Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)
Date Wed, 04 Apr 2018 01:51:49 GMT
+1 (non binding)

* Deployed with 4 subclusters with HDFS Router-based federation.
* Executed DistCp across subclusters through the Router
* Checked documentation and tgz

On Tue, Apr 3, 2018 at 4:25 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org>
wrote:

> We vote on the source code. The binaries are convenience artifacts.
>
> This is what I would do - (a) Just replace both the maven jars as well as
> the binaries to be consistent and correct. And then (b) Give a couple more
> days for folks who tested on the binaries to reverify - I count one such
> clear vote as of now.
>
> Thanks
> +Vinod
>
> > On Apr 3, 2018, at 3:30 PM, Wangda Tan <wheeleast@gmail.com> wrote:
> >
> > HI Arpit,
> >
> > I think it won't match if we do rebuild. It should be fine as far as
> they're signed, correct? I don't see any policy doesn't allow this.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Tue, Apr 3, 2018 at 9:33 AM, Arpit Agarwal <aagarwal@hortonworks.com
> <mailto:aagarwal@hortonworks.com>> wrote:
> > Thanks Wangda, I see the shaded jars now.
> >
> > Are the repo jars required to be the same as the binary release? They
> don’t match right now, probably they got rebuilt.
> >
> > +1 (binding), modulo that remaining question.
> >
> > * Verified signatures
> > * Verified checksums for source and binary artefacts
> > * Sanity checked jars on r.a.o.
> > * Built from source
> > * Deployed to 3 node secure cluster with NameNode HA
> > * Verified HDFS web UIs
> > * Tried out HDFS shell commands
> > * Ran sample MapReduce jobs
> >
> > Thanks!
> >
> >
> > ----------------------------------------------------------------------
> > From: Wangda Tan <wheeleast@gmail.com <mailto:wheeleast@gmail.com>>
> > Date: Monday, April 2, 2018 at 9:25 PM
> > To: Arpit Agarwal <aagarwal@hortonworks.com <mailto:
> aagarwal@hortonworks.com>>
> > Cc: Gera Shegalov <gera05@gmail.com <mailto:gera05@gmail.com>>, Sunil
G
> <sunilg@apache.org <mailto:sunilg@apache.org>>, "
> yarn-dev@hadoop.apache.org <mailto:yarn-dev@hadoop.apache.org>" <
> yarn-dev@hadoop.apache.org <mailto:yarn-dev@hadoop.apache.org>>, Hdfs-dev
> <hdfs-dev@hadoop.apache.org <mailto:hdfs-dev@hadoop.apache.org>>, Hadoop
> Common <common-dev@hadoop.apache.org <mailto:common-dev@hadoop.apache.org>>,
> "mapreduce-dev@hadoop.apache.org <mailto:mapreduce-dev@hadoop.apache.org>"
> <mapreduce-dev@hadoop.apache.org <mailto:mapreduce-dev@hadoop.apache.org>>,
> Vinod Kumar Vavilapalli <vinodkv@apache.org <mailto:vinodkv@apache.org>>
> > Subject: Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)
> >
> > As pointed by Arpit, the previously deployed shared jars are incorrect.
> Just redeployed jars and staged. @Arpit, could you please check the updated
> Maven repo? https://repository.apache.org/content/repositories/
> orgapachehadoop-1092 <https://repository.apache.org/content/repositories/
> orgapachehadoop-1092>
> >
> > Since the jars inside binary tarballs are correct (
> http://people.apache.org/~wangda/hadoop-3.1.0-RC1/ <
> http://people.apache.org/~wangda/hadoop-3.1.0-RC1/>). I think we don't
> need roll another RC, just update Maven repo should be sufficient.
> >
> > Best,
> > Wangda
> >
> >
> > On Mon, Apr 2, 2018 at 2:39 PM, Wangda Tan <mailto:wheeleast@gmail.com
> <mailto:wheeleast@gmail.com>> wrote:
> > Hi Arpit,
> >
> > Thanks for pointing out this.
> >
> > I just removed all .md5 files from artifacts. I found md5 checksums
> still exist in .mds files and I didn't remove them from .mds file because
> it is generated by create-release script and Apache guidance is "should
> not" instead of "must not". Please let me know if you think they need to be
> removed as well.
> >
> > - Wangda
> >
> >
> >
> > On Mon, Apr 2, 2018 at 1:37 PM, Arpit Agarwal <mailto:
> aagarwal@hortonworks.com <mailto:aagarwal@hortonworks.com>> wrote:
> > Thanks for putting together this RC, Wangda.
> >
> > The guidance from Apache is to omit MD5s, specifically:
> >   > SHOULD NOT supply a MD5 checksum file (because MD5 is too broken).
> >
> > https://www.apache.org/dev/release-distribution#sigs-and-sums <
> https://www.apache.org/dev/release-distribution#sigs-and-sums>
> >
> >
> >
> >
> > On Apr 2, 2018, at 7:03 AM, Wangda Tan <mailto:wheeleast@gmail.com
> <mailto:wheeleast@gmail.com>> wrote:
> >
> > Hi Gera,
> >
> > It's my bad, I thought only src/bin tarball is enough.
> >
> > I just uploaded all other things under artifact/ to
> > http://people.apache.org/~wangda/hadoop-3.1.0-RC1/ <
> http://people.apache.org/~wangda/hadoop-3.1.0-RC1/>
> >
> > Please let me know if you have any other comments.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Mon, Apr 2, 2018 at 12:50 AM, Gera Shegalov <mailto:gera05@gmail.com
> <mailto:gera05@gmail.com>> wrote:
> >
> >
> > Thanks, Wangda!
> >
> > There are many more artifacts in previous votes, e.g., see
> > http://home.apache.org/~junping_du/hadoop-2.8.3-RC0/ <
> http://home.apache.org/~junping_du/hadoop-2.8.3-RC0/> .  Among others the
> > site tarball is missing.
> >
> > On Sun, Apr 1, 2018 at 11:54 PM Sunil G <mailto:sunilg@apache.org
> <mailto:sunilg@apache.org>> wrote:
> >
> >
> > Thanks Wangda for initiating the release.
> >
> > I tested this RC built from source file.
> >
> >
> >   - Tested MR apps (sleep, wc) and verified both new YARN UI and old RM
> > UI.
> >   - Below feature sanity is done
> >      - Application priority
> >      - Application timeout
> >      - Intra Queue preemption with priority based
> >      - DS based affinity tests to verify placement constraints.
> >   - Tested basic NodeLabel scenarios.
> >      - Added couple of labels to few of nodes and behavior is coming
> >      correct.
> >      - Verified old UI  and new YARN UI for labels.
> >      - Submitted apps to labelled cluster and it works fine.
> >      - Also performed few cli commands related to nodelabel.
> >   - Test basic HA cases and seems correct.
> >   - Tested new YARN UI . All pages are getting loaded correctly.
> >
> >
> > - Sunil
> >
> > On Fri, Mar 30, 2018 at 9:45 AM Wangda Tan <mailto:wheeleast@gmail.com
> <mailto:wheeleast@gmail.com>> wrote:
> >
> >
> > Hi folks,
> >
> > Thanks to the many who helped with this release since Dec 2017 [1].
> > We've
> >
> > created RC1 for Apache Hadoop 3.1.0. The artifacts are available here:
> >
> > http://people.apache.org/~wangda/hadoop-3.1.0-RC1 <
> http://people.apache.org/~wangda/hadoop-3.1.0-RC1>
> >
> > The RC tag in git is release-3.1.0-RC1. Last git commit SHA is
> > 16b70619a24cdcf5d3b0fcf4b58ca77238ccbe6d
> >
> > The maven artifacts are available via http://repository.apache.org <
> http://repository.apache.org/> at
> > https://repository.apache.org/content/repositories/ <
> https://repository.apache.org/content/repositories/>
> > orgapachehadoop-1090/
> >
> > This vote will run 5 days, ending on Apr 3 at 11:59 pm Pacific.
> >
> > 3.1.0 contains 766 [2] fixed JIRA issues since 3.0.0. Notable additions
> > include the first class GPU/FPGA support on YARN, Native services,
> > Support
> >
> > rich placement constraints in YARN, S3-related enhancements, allow HDFS
> > block replicas to be provided by an external storage system, etc.
> >
> > For 3.1.0 RC0 vote discussion, please see [3].
> >
> > We’d like to use this as a starting release for 3.1.x [1], depending on
> > how
> >
> > it goes, get it stabilized and potentially use a 3.1.1 in several weeks
> > as
> >
> > the stable release.
> >
> > We have done testing with a pseudo cluster:
> > - Ran distributed job.
> > - GPU scheduling/isolation.
> > - Placement constraints (intra-application anti-affinity) by using
> > distributed shell.
> >
> > My +1 to start.
> >
> > Best,
> > Wangda/Vinod
> >
> > [1]
> >
> > https://lists.apache.org/thread.html/b3fb3b6da8b6357a68513a6dfd104b <
> https://lists.apache.org/thread.html/b3fb3b6da8b6357a68513a6dfd104b>
> > c9e19e559aedc5ebedb4ca08c8@%http://3Cyarn-dev.hadoop.apache.org <
> http://3cyarn-dev.hadoop.apache.org/>%3E
> >
> > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.0)
> > AND fixVersion not in (3.0.0, 3.0.0-beta1) AND status = Resolved ORDER
> > BY
> >
> > fixVersion ASC
> > [3]
> >
> > https://lists.apache.org/thread.html/b3a7dc075b7329fd660f65b48237d7 <
> https://lists.apache.org/thread.html/b3a7dc075b7329fd660f65b48237d7>
> > 2d4061f26f83547e41d0983ea6@%http://3Cyarn-dev.hadoop.apache.org <
> http://3cyarn-dev.hadoop.apache.org/>%3E
> >
> >
> >
> >
> >
> >
> >
> >
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message