hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11937) Guarantee a full build of all native code during pre-commit.
Date Mon, 11 May 2015 17:21:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14538206#comment-14538206
] 

Chris Nauroth commented on HADOOP-11937:
----------------------------------------

Thank you for looking, Allen and Sean.

I don't see much opportunity for safely optimizing pre-commit run times by skipping the native
build.  The Java layers are tightly coupled with their corresponding native layers, so it's
possible to break either native code itself or its integration with Java even if a patch only
touches Java files.  hadoop-common and libhadoop.so are tighly coupled on function names and
variable names.  libhdfs runs the Java classes in an embedded JVM, so it's possible to break
a libhdfs test by changing only Java code.  {{LinuxContainerExecutor}} and the native binary
must use an agreed interface in terms of executable name and order of arguments.

The logic for skipping irrelevant modules or even skipping compilation entirely for script
and doc changes is of course still valid.

Considering all of that, I'd summarize the requirements as follows.  (Feedback welcome.)
# hadoop-common, if built, always builds with -Pnative, -Drequire.bzip2, -Drequire.snappy,
-Drequire.openssl and -Drequire.test.libhadoop.
# hadoop-hdfs, if built, always builds with -Pnative, -Drequire.fuse, -Drequire.libwebhdfs
and -Drequire.test.libhadoop.
# hadoop-yarn-nodemanager, if built, always builds with -Pnative.
# hadoop-mapreduce-client-nativetask, if built, always builds with -Pnative and -Drequire.snappy.
# hadoop-pipes, if built, always builds with -Pnative.

> Guarantee a full build of all native code during pre-commit.
> ------------------------------------------------------------
>
>                 Key: HADOOP-11937
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11937
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: scripts
>            Reporter: Chris Nauroth
>             Fix For: 2.8.0
>
>
> Some of the native components of the build are considered optional and either will not
build at all without passing special flags to Maven or will allow a build to proceed if dependencies
are missing from the build machine.  If these components do not get built, then pre-commit
isn't really providing full coverage of the build.  This issue proposes to update test-patch.sh
so that it does a full build of all native components.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message