hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yongjun Zhang <yzh...@cloudera.com>
Subject Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)
Date Wed, 06 Jun 2018 17:39:11 GMT
Thank you all so much for the testing and vote!

Nanda reported the following problem:

The following links in site documentation are broken
- Changelog and Release Notes
- Unix Shell API
- core-default.xml
- hdfs-default.xml
- hdfs-rbf-default.xml
- mapred-default.xml
- yarn-default.xml

Site documentation was generated using the below steps
*- mvn site:site*
*- mkdir -p /tmp/site && mvn site:stage -DstagingDirectory=/tmp/site*
*- Browse to file:///tmp/site/hadoop-project/index.html.*

Thanks,
Nanda


My build in docker was fine, and when I tested out the build outside
docker, I tested the RC binary tarbal and regular build with source
tarball, but missed out the site:site build.

I tried the steps Nanda did, and I'm even hitting an issue at the "mvn
site:site" which Nanda did not hit.

What I'm seeing is:

[INFO] ------------------------------------------------------------
------------
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-site-plugin:3.6:site
(default-cli) on project hadoop-main: failed to get report for
org.apache.maven.plugins:maven-javadoc-plugin: Failed to execute goal on
project hadoop-hdfs: Could not resolve dependencies for project
org.apache.hadoop:hadoop-hdfs:jar:3.0.3: *Failure to find
org.apache.hadoop:hadoop-hdfs-client:jar:3.0.3
in https://s3-us-west-2.amazonaws.com/dynamodb-local/release
<https://s3-us-west-2.amazonaws.com/dynamodb-local/release>* was cached in
the local repository, resolution will not be reattempted until the update
interval of dynamodb-local-oregon has elapsed or updates are forced ->
[Help 1]
[ERROR]


I wonder if anyone else hit this problem. I was able to build 3.0.2 with
the same env. The build is looking for 3.0.3 artifacts in
*https://s3-us-west-2.amazonaws.com/dynamodb-local/release
<https://s3-us-west-2.amazonaws.com/dynamodb-local/release>, *which I'm
sure is the right thing, because 3.0.3 is not yet released. Or probably I
missed something in my env for this, which I have to investigate.

If anyone can follow Nanda's steps do see different outcome, that would be
interesting. Very much appreciated.

Thanks again!

--Yongjun

On Wed, Jun 6, 2018 at 12:59 AM, Kitti Nánási <knanasi@cloudera.com> wrote:

> Thanks Yongjun for working on this!
>
> +1 (non-binding)
>
> - checked out git tag release-3.0.3-RC0
> - built from source on Mac OS X 10.13.4, java version 1.8.0_171
> - deployed on a 3 node cluster
> - ran terasort, teragen, teravalidate with success
> - executed basic hdfs commands
> - created snapshots and produced snapshot diffs
>
> Thanks,
> Kitti Nanasi
>
>
> On Wed, Jun 6, 2018 at 8:06 AM, Takanobu Asanuma <tasanuma@yahoo-corp.jp>
> wrote:
>
>> Sorry, I forgot to write my vote in the last message.
>>
>> +1 (non-binding)
>>
>> Regards,
>> Takanobu
>>
>> > -----Original Message-----
>> > From: Takanobu Asanuma [mailto:tasanuma@yahoo-corp.jp]
>> > Sent: Wednesday, June 06, 2018 10:22 AM
>> > To: Gabor Bota <gabor.bota@cloudera.com>
>> > Cc: Yongjun Zhang <yzhang@cloudera.com>; nvadivelu@hortonworks.com;
>> > sbanerjee@hortonworks.com; Hadoop Common <common-dev@hadoop.apache.org>
>> ;
>> > Hdfs-dev <hdfs-dev@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org;
>> > yarn-dev@hadoop.apache.org
>> > Subject: RE: [VOTE] Release Apache Hadoop 3.0.3 (RC0)
>> >
>> > Thanks for driving this release, Yongjun!
>> >
>> > - Verified checksums
>> > - Succeeded native package build on CentOS 7
>> > - Started a cluster with 1 master and 5 slaves
>> > - Verified Web UI (NN, RM, JobHistory, Timeline)
>> > - Verified Teragen/Terasort jobs
>> > - Verified some operations for erasure coding
>> >
>> > Regards,
>> > Takanobu
>> >
>> > > -----Original Message-----
>> > > From: Gabor Bota [mailto:gabor.bota@cloudera.com]
>> > > Sent: Tuesday, June 05, 2018 9:18 PM
>> > > To: nvadivelu@hortonworks.com
>> > > Cc: Yongjun Zhang <yzhang@cloudera.com>; sbanerjee@hortonworks.com;
>> > > Hadoop Common <common-dev@hadoop.apache.org>; Hdfs-dev
>> > > <hdfs-dev@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org;
>> > > yarn-dev@hadoop.apache.org
>> > > Subject: Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)
>> > >
>> > > Thanks for the work Yongjun!
>> > >
>> > > +1 (non-binding)
>> > >
>> > >    - checked out git tag release-3.0.3-RC0. Thanks for adding this
>> > Yongjun,
>> > >    it worked.
>> > >    - S3A integration (mvn verify) test run were successful on
>> eu-west-1
>> > >    besides of one test issue reported in HADOOP-14927.
>> > >    - built from source on Mac OS X 10.13.4, java version
>> 8.0.171-oracle
>> > >    - deployed on a 3 node cluster (HDFS HA, Non-HA YARN)
>> > >    - verified pi job (yarn), teragen, terasort and teravalidate
>> > >
>> > > Regards,
>> > > Gabor Bota
>> > >
>> > > On Mon, Jun 4, 2018 at 7:38 PM Nandakumar Vadivelu <
>> > > nvadivelu@hortonworks.com> wrote:
>> > >
>> > > > Thanks Yongjun for all the hard work on this release.
>> > > >
>> > > > - Verified signatures and checksums
>> > > > - Tested both the binary and also built from source on a pseudo
>> > > > distributed cluster
>> > > > - Verified filesystem shell commands
>> > > > - Verified admin commands
>> > > > - Tested snapshot feature
>> > > > - Sanity check of NN and DN UI
>> > > > - Verified site documentation
>> > > >
>> > > > The following links in site documentation are broken
>> > > > - Changelog and Release Notes
>> > > > - Unix Shell API
>> > > > - core-default.xml
>> > > > - hdfs-default.xml
>> > > > - hdfs-rbf-default.xml
>> > > > - mapred-default.xml
>> > > > - yarn-default.xml
>> > > >
>> > > > Site documentation was generated using the below steps
>> > > > - mvn site:site
>> > > > - mkdir -p /tmp/site && mvn site:stage -DstagingDirectory=/tmp/site
>> > > > - Browse to file:///tmp/site/hadoop-project/index.html.
>> > > >
>> > > > Thanks,
>> > > > Nanda
>> > > >
>> > > > On 6/3/18, 8:55 AM, "Yongjun Zhang" <yzhang@cloudera.com>
wrote:
>> > > >
>> > > >     Hi Gabor,
>> > > >
>> > > >     I got the git tag in, it's release-3.0.3-RC0. Would you please
>> > > > give it a
>> > > >     try?
>> > > >
>> > > >     It should correspond to
>> > > >
>> > > >     commit 37fd7d752db73d984dc31e0cdfd590d252f5e075
>> > > >     Author: Yongjun Zhang <yzhang@cloudera.com>
>> > > >     Date:   Wed May 30 00:07:33 2018 -0700
>> > > >
>> > > >         Update version to 3.0.3 to prepare for 3.0.3 release
>> > > >
>> > > >
>> > > >     Thanks,
>> > > >
>> > > >     --Yongjun
>> > > >
>> > > >     On Fri, Jun 1, 2018 at 4:17 AM, Gabor Bota
>> > > > <gabor.bota@cloudera.com>
>> > > > wrote:
>> > > >
>> > > >     > Hi Yongjun,
>> > > >     >
>> > > >     > Thank you for working on this release. Is there a git tag
in
>> > > > the upstream
>> > > >     > repo which can be checked out? I'd like to build the release
>> > > > from source.
>> > > >     >
>> > > >     > Regards,
>> > > >     > Gabor
>> > > >     >
>> > > >     > On Fri, Jun 1, 2018 at 7:57 AM Shashikant Banerjee <
>> > > >     > sbanerjee@hortonworks.com> wrote:
>> > > >     >
>> > > >     >> Looks like the link with the filter seems to be private.
I
>> > > > can't see the
>> > > >     >> blocker list.
>> > > >     >> https://issues.apache.org/jira/issues/?filter=12343997
>> > > >     >>
>> > > >     >> Meanwhile , I will be working on testing the release.
>> > > >     >>
>> > > >     >> Thanks
>> > > >     >> Shashi
>> > > >     >> On 6/1/18, 11:18 AM, "Yongjun Zhang" <yjzhangal@apache.org>
>> > wrote:
>> > > >     >>
>> > > >     >>     Greetings all,
>> > > >     >>
>> > > >     >>     I've created the first release candidate (RC0) for
Apache
>> > > Hadoop
>> > > >     >>     3.0.3. This is our next maintenance release to follow
up
>> > 3.0.2.
>> > > > It
>> > > >     >> includes
>> > > >     >>     about 249
>> > > >     >>     important fixes and improvements, among which there
are 8
>> > > > blockers.
>> > > >     >> See
>> > > >     >>     https://issues.apache.org/jira/issues/?filter=12343997
>> > > >     >>
>> > > >     >>     The RC artifacts are available at:
>> > > >     >>     https://dist.apache.org/repos/dist/dev/hadoop/3.0.3-RC0/
>> > > >     >>
>> > > >     >>     The maven artifacts are available via
>> > > >     >>     https://repository.apache.org/content/repositories/
>> > > >     >> orgapachehadoop-1126
>> > > >     >>
>> > > >     >>     Please try the release and vote; the vote will run
for
>> the
>> > > > usual 5
>> > > >     >> working
>> > > >     >>     days, ending on 06/07/2018 PST time. Would really
>> appreciate
>> > > > your
>> > > >     >>     participation here.
>> > > >     >>
>> > > >     >>     I bumped into quite some issues along the way, many
>> thanks
>> > > to
>> > > > quite a
>> > > >     >> few
>> > > >     >>     people who helped, especially Sammi Chen, Andrew
Wang,
>> > Junping
>> > > > Du,
>> > > >     >> Eddy Xu.
>> > > >     >>
>> > > >     >>     Thanks,
>> > > >     >>
>> > > >     >>     --Yongjun
>> > > >     >>
>> > > >     >>
>> > > >     >>
>> > > >
>> > > >
>> > > >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message