hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Allen Wittenauer ...@apache.org>
Subject ASF OS X Build Infrastructure
Date Thu, 19 May 2016 16:41:11 GMT
	
	Some of you may not know that the ASF actually does have an OS X machine (a Mac mini, so
it’s not a speed demon) in the build infrastructure.  While messing around with getting
all? of the trunk jobs reconfigured to do Java 8 and separate maven repos, I noticed that
this box tends to sit idle most of the day. Why take advantage of it?  Therefore, I also setup
two jobs for us to use to help alleviate the “I don’t have access to anything but Linux”
excuse when writing code that may not work in a portable manner.

Jobs #1:

	https://builds.apache.org/view/H-L/view/Hadoop/job/Precommit-HADOOP-OSX

	This basically runs Apache Yetus precommit with quite a few of the unnecessary tests disabled.
 For example, there’s no point in running checkstyle.  Note that this job takes the *full*
JIRA issue id as input.  So ‘HADOOP-9902’ not ‘9902’.  This allows for one Jenkins
job to be used for all the Hadoop sub-projects (HADOOP, HDFS, MR, YARN).  “But my code is
on github and I don’t want to upload a patch!”  I haven’t tested it, but it should also
take a URL, so just add a .diff to the end of your github compare URL and put that in the
issue box.  It hypothetically should work.

Job #2:

	I’m still hammering on this one because the email notifications aren’t working to my
satisfaction plus we have some extremely Linux-specific code in YARN… but 

	https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-trunk-osx-java8/

	… is a “build the world” job similar to what is currently running under the individual
sub projects.  (This actually makes it one of the few “build everything” jobs we have
running. Most of the other jobs only build that particular sub project.).  It does not run
the full unit test suite and it also does not build all of the native code.  This gives us
a place to start on our journey of making Hadoop actually, truly run everywhere.  (Interesting
side note: It’s been *extremely* consistent in what fails vs. the Linux build hosts.)

	At some point, likely after YETUS-390 is complete, I’ll switch this job over to be run
by Apache Yetus in qbt mode so that it’s actually easier to track failures across all dirs.
 A huge advantage over raw maven commands.

	Happy testing everyone.

	NOTE: if you don’t have access to launch jobs on builds.apache.org, you’ll need to send
a request to private@.  The Apache Hadoop PMC has the keys to give access to folks.



---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org


Mime
View raw message