hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject Re: Hadoop CI with alternate architectures.
Date Thu, 19 May 2016 09:00:52 GMT

> On 18 May 2016, at 16:29, Allen Wittenauer <aw@apache.org> wrote:
> 	That’s really a question for infrastructure-dev@apache.org . They manage the ASF build
infrastructure which Apache Hadoop and lots of other projects utilize.  (Bigtop uses something
custom, which I think is funded by Cloudera.)

There's no dedicated Hadoop boxes, just a pool of machines for all projects. I don't know
about bigtop.

There's a mailing list, builds@apache.org, for people who want to get involved

> 	Once it is registered with builds.apache.org, it’s just a matter of us enabling a
Jenkins job for it. I suspect we’ll likely do it like we have done other architectures in
the past: set up a nightly job that people will need to cognizant of.  Adding it to precommit
would likely be too much and/or require some significant Yetus work to support reports from
multiple hosts.

Nightly would be a good initial step; the hard part is getting it stable enough for its build-failed
messages to not get ignored (it should be on send-no-emails mode until stable). Then comes
the cultural shift: getting people to care that the build is failing. That'll be from native,
OS and JVM issues.

Maybe Yetus could be set to warn that on changes which go near the .C or makefile code, that
someone should explicitly kick off a precommit run on the PPC. That could be enough to remind
those people playing with native code that that's something that really needs a check

> 	FWIW:
> 		* We have a Mac mini which, ironically, I thought I’d see if I could get a nightly
job running on while I type this.
> 		* We have a dedicated Windows box which appears to be completely screwed up. 

the default state of all windows boxes, sadly. The VMs are odd too; the FS permissions are
enough to stop MiniYarnClusters coming up.

> 		* In the past, I think it was IBM that hinted that they would be willing to give ASF
a PowerPC box but that offer seems to have disappeared.

View raw message