hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Allen Wittenauer ...@effectivemachines.com>
Subject Re: [VOTE] Release Apache Hadoop 3.0.0 RC0
Date Tue, 21 Nov 2017 07:33:43 GMT

	The original release script and instructions broke the build up into three or so steps. When
I rewrote it, I kept that same model. It’s probably time to re-think that.  In particular,
it should probably be one big step that even does the maven deploy.  There’s really no harm
in doing that given that there is still a manual step to release the deployed jars into the
production area.

	We just need need to:

a) add an option to do deploy instead of just install.  if c-r is in asf mode, always activate
deploy
b) pull the maven settings.xml file (and only the maven settings file… we don’t want the
repo!) into the docker build environment
c) consolidate the mvn steps

	This has the added benefit of greatly speeding up the build by removing several passes.

	Probably not a small change, but I’d have to look at the code.  I’m on a plane tomorrow
morning though.

Also:

>> 
>> Major
>> - The previously supported way of being able to use different tar-balls
>> for different sub-modules is completely broken - common and HDFS tar.gz are
>> completely empty.
>> 
> 
> Is this something people use? I figured that the sub-tarballs were a relic
> from the project split, and nowadays Hadoop is one project with one release
> tarball. I actually thought about getting rid of these extra tarballs since
> they add extra overhead to a full build.

	I’m guessing no one noticed the tar errors when running mvn -Pdist.  Not sure when they
started happening.

> >   - When did we stop putting CHANGES files into the source artifacts?
> 
> CHANGES files were removed by https://issues.apache.org/jira/browse/HADOOP-11792

	To be a bit more specific about it, the maven assembly for source only includes things (more
or less) that are part of the git repo.  When CHANGES.txt was removed from the source tree,
it also went away from the tar ball.  This isn’t too much of an issue in practice though
given the notes are put up on the web, part of the binary tar ball, and can be generated by
following the directions in BUILDING.txt.  I don’t remember if Hadoop uploads them into
the dist area, but if not probably should.

> - $HADOOP_YARN_HOME/sbin/yarn-daemon.sh start historyserver doesn't even work. Not just
deprecated in favor of timelineserver as was advertised.

	This works for me in trunk and the bash code doesn’t appear to have changed in a very long
time.  Probably something local to your install.  (I do notice that the deprecation message
says “starting” which is awkward when the stop command is given though.)  Also: is the
deprecation message even true at this point?

>> - Cannot enable new UI in YARN because it is under a non-default
>> compilation flag. It should be on by default.
>> 
> 
> The yarn-ui profile has always been off by default, AFAIK. It's documented
> to turn it on in BUILDING.txt for release builds, and we do it in
> create-release.
> 
> IMO not a blocker. I think it's also more of a dev question (do we want to
> do this on every YARN build?) than a release one.

	-1 on making yarn-ui always build.

	For what is effectively an optional component (the old UI is still there), it’s heavy dependency
requirements make it a special burden outside of the Docker container.  If it can be changed
such that it either always downloads the necessary bits (regardless of the OS/chipset!) and/or
doesn’t kill the maven build if those bits can’t be found  (i.e., truly optional), then
I’d be less opposed.  (and, actually, quite pleased because then the docker image build
would be significantly faster.)



---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org


Mime
View raw message