hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yongjun Zhang <yzh...@cloudera.com.INVALID>
Subject Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)
Date Tue, 10 Jul 2018 15:22:47 GMT
Welcome Jonathan.

http://hadoop.apache.org/releases.html stated:
"Hadoop is released as source code tarballs with corresponding binary
tarballs for convenience. "

and Andrew Wang said "The binary artifacts (including JARs) are technically
just convenience artifacts" and it seems not an uncommon practice to do
follow-up builds to release maven artifacts.

IIRC, Andrew once shared with me that we started in 3.x to use a single
build to to do both release binaries creation and maven artifacts
deployment, prior releases are using multiple builds:

Referring to https://wiki.apache.org/hadoop/HowToRelease

   - 3.x: step 4 in   "Creating the release candidate (X.Y.Z-RC<N>)"
   section does both release binaries creation and maven artifacts deployment.
   - prior to 3.x: step 4 does release binary creation, and step 10 does
   maven artifacts deployment, *each step does its build so two builds here*.
   As a matter of fact, I did not run step 10 for 3.0.3.

That said, I agree that ideally it's better to do a single build to
generate release binaries and deploy maven artifacts from the same build.

Hope it helps. Welcome other folks to chime in.

Best,

--Yongjun






On Mon, Jul 9, 2018 at 2:08 PM, Jonathan Eagles <jeagles@gmail.com> wrote:

> Thank you, Yongjun Zhang for resolving this issue for me. I have verified
> the 3.0.3 build is now working for me for tez to specify as a hadoop
> dependency.
>
> As for release procedure, can someone comment on what to do now that the
> artifacts published to maven are different than the voted on artifacts. I
> believe the source code is what is voted on and the maven artifacts are
> just for convenience, but would like an "official" answer.
>
> Reference:
> https://issues.apache.org/jira/browse/TEZ-3955
>
> Regards,
> jeagles
>
> On Mon, Jul 9, 2018 at 12:26 PM, Yongjun Zhang <yzhang@cloudera.com>
> wrote:
>
>> HI Jonathan,
>>
>> I have updated the artifacts, so now
>>
>> https://repository.apache.org/#nexus-search;gav~org.apache.hadoop~~3
>> .0.2~~
>> https://repository.apache.org/#nexus-search;gav~org.apache.hadoop~~3.0.3
>> ~~
>>
>> are more consistent, except that 3.0.3 has an extra entry for rbf. Would
>> you please try again?
>>
>> The propagation to
>> https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-project
>> will take some time. I did nothing different than last time, so keep
>> finger crossed that it will propagate there.
>>
>> Thanks Sammi Chen and Andrew Wang for info and advice, and sorry for the
>> inconvenience again.
>>
>> Best,
>>
>> --Yongjun
>>
>> On Mon, Jul 2, 2018 at 9:30 AM, Jonathan Eagles <jeagles@gmail.com>
>> wrote:
>>
>>> Release 3.0.3 is still broken due to the missing artifacts. Any update
>>> on when these artifacts will be published?
>>>
>>> On Wed, Jun 27, 2018 at 8:25 PM, Chen, Sammi <sammi.chen@intel.com>
>>> wrote:
>>>
>>>> Hi Yongjun,
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> The artifacts will be pushed to https://mvnrepository.com/arti
>>>> fact/org.apache.hadoop/hadoop-project after step 6 of Publishing steps.
>>>>
>>>>
>>>> For 2.9.1, I remember I absolutely did the step before. I redo the step
>>>> 6 today and now 2.9.1 is pushed to the mvn repo.
>>>>
>>>> You can double check it. I suspect sometimes Nexus may fail to notify
>>>> user when this is unexpected failures.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Bests,
>>>>
>>>> Sammi
>>>>
>>>> *From:* Yongjun Zhang [mailto:yzhang@cloudera.com]
>>>> *Sent:* Sunday, June 17, 2018 12:17 PM
>>>> *To:* Jonathan Eagles <jeagles@gmail.com>; Chen, Sammi <
>>>> sammi.chen@intel.com>
>>>> *Cc:* Eric Payne <erichadoop-1@yahoo.com>; Hadoop Common <
>>>> common-dev@hadoop.apache.org>; Hdfs-dev <hdfs-dev@hadoop.apache.org>;
>>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)
>>>>
>>>>
>>>>
>>>> + Junping, Sammi
>>>>
>>>>
>>>>
>>>> Hi Jonathan,
>>>>
>>>>
>>>>
>>>> Many thanks for reporting the issues and sorry for the inconvenience.
>>>>
>>>>
>>>>
>>>> 1. Shouldn't the build be looking for artifacts in
>>>>
>>>>
>>>>
>>>> https://repository.apache.org/content/repositories/releases
>>>>
>>>> rather than
>>>>
>>>>
>>>>
>>>> https://repository.apache.org/content/repositories/snapshots
>>>>
>>>> ?
>>>>
>>>>
>>>>
>>>> 2.
>>>>
>>>> Not seeing the artifact published here as well.
>>>>
>>>> https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-project
>>>>
>>>>
>>>>
>>>> Indeed, I did not see 2.9.1 there too. So included Sammi Chen.
>>>>
>>>>
>>>>
>>>> Hi Junping, would you please share which step in
>>>>
>>>> https://wiki.apache.org/hadoop/HowToRelease
>>>>
>>>> should have done this?
>>>>
>>>>
>>>>
>>>> Thanks a lot.
>>>>
>>>>
>>>>
>>>> --Yongjun
>>>>
>>>>
>>>>
>>>> On Fri, Jun 15, 2018 at 10:52 PM, Jonathan Eagles <jeagles@gmail.com>
>>>> wrote:
>>>>
>>>> Upgraded Tez dependency to hadoop 3.0.3 and found this issue. Anyone
>>>> else seeing this issue?
>>>>
>>>>
>>>>
>>>> [ERROR] Failed to execute goal on project hadoop-shim: Could not
>>>> resolve dependencies for project org.apache.tez:hadoop-shim:jar:0.10.0-SNAPSHOT:
>>>> Failed to collect dependencies at org.apache.hadoop:hadoop-yarn-api:jar:3.0.3:
>>>> Failed to read artifact descriptor for org.apache.hadoop:hadoop-yarn-api:jar:3.0.3:
>>>> Could not find artifact org.apache.hadoop:hadoop-project:pom:3.0.3 in
>>>> apache.snapshots.https (https://repository.apache.org
>>>> /content/repositories/snapshots) -> [Help 1]
>>>>
>>>> [ERROR]
>>>>
>>>> [ERROR] To see the full stack trace of the errors, re-run Maven with
>>>> the -e switch.
>>>>
>>>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>>>>
>>>> [ERROR]
>>>>
>>>> [ERROR] For more information about the errors and possible solutions,
>>>> please read the following articles:
>>>>
>>>> [ERROR] [Help 1] http://cwiki.apache.org/conflu
>>>> ence/display/MAVEN/DependencyResolutionException
>>>>
>>>> [ERROR]
>>>>
>>>> [ERROR] After correcting the problems, you can resume the build with
>>>> the command
>>>>
>>>> [ERROR]   mvn <goals> -rf :hadoop-shim
>>>>
>>>>
>>>>
>>>> Not seeing the artifact published here as well.
>>>>
>>>> https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-project
>>>>
>>>>
>>>>
>>>> On Tue, Jun 12, 2018 at 6:44 PM, Yongjun Zhang <yzhang@cloudera.com>
>>>> wrote:
>>>>
>>>> Thanks Eric!
>>>>
>>>> --Yongjun
>>>>
>>>>
>>>> On Mon, Jun 11, 2018 at 8:05 AM, Eric Payne <erichadoop-1@yahoo.com>
>>>> wrote:
>>>>
>>>> > Sorry, Yongjun. My +1 is also binding
>>>> > +1 (binding)
>>>> > -Eric Payne
>>>> >
>>>> > On Friday, June 1, 2018, 12:25:36 PM CDT, Eric Payne <
>>>> > eric.payne1000@yahoo.com> wrote:
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > Thanks a lot, Yongjun, for your hard work on this release.
>>>> >
>>>> > +1
>>>> > - Built from source
>>>> > - Installed on 6 node pseudo cluster
>>>> >
>>>> >
>>>> > Tested the following in the Capacity Scheduler:
>>>> > - Verified that running apps in labelled queues restricts tasks to the
>>>> > labelled nodes.
>>>> > - Verified that various queue config properties for CS are refreshable
>>>> > - Verified streaming jobs work as expected
>>>> > - Verified that user weights work as expected
>>>> > - Verified that FairOrderingPolicy in a CS queue will evenly assign
>>>> > resources
>>>> > - Verified running yarn shell application runs as expected
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > On Friday, June 1, 2018, 12:48:26 AM CDT, Yongjun Zhang <
>>>> > yjzhangal@apache.org> wrote:
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > Greetings all,
>>>> >
>>>> > I've created the first release candidate (RC0) for Apache Hadoop
>>>> > 3.0.3. This is our next maintenance release to follow up 3.0.2. It
>>>> includes
>>>> > about 249
>>>> > important fixes and improvements, among which there are 8 blockers.
>>>> See
>>>> > https://issues.apache.org/jira/issues/?filter=12343997
>>>> >
>>>> > The RC artifacts are available at:
>>>> > https://dist.apache.org/repos/dist/dev/hadoop/3.0.3-RC0/
>>>> >
>>>> > The maven artifacts are available via
>>>> > https://repository.apache.org/content/repositories/orgapache
>>>> hadoop-1126
>>>> >
>>>> > Please try the release and vote; the vote will run for the usual 5
>>>> working
>>>> > days, ending on 06/07/2018 PST time. Would really appreciate your
>>>> > participation here.
>>>> >
>>>> > I bumped into quite some issues along the way, many thanks to quite
a
>>>> few
>>>> > people who helped, especially Sammi Chen, Andrew Wang, Junping Du,
>>>> Eddy Xu.
>>>> >
>>>> > Thanks,
>>>> >
>>>> > --Yongjun
>>>> >
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message