Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Mon, 11 Jun 2012 19:58:46 +0000 (UTC)
From: "Hudson (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <804394820.3957.1339444726916.JavaMail.jiratomcat@issues-vm>
In-Reply-To: 
 <1709016574.36550.1336426129196.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HADOOP-8368) Use CMake rather than autotools to
 build native code
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/HADOOP-8368?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D132=
93023#comment-13293023 ]=20

Hudson commented on HADOOP-8368:
--------------------------------

Integrated in Hadoop-Common-trunk-Commit #2341 (See [https://builds.apache.=
org/job/Hadoop-Common-trunk-Commit/2341/])
    HADOOP-8368. Amendment to add entry in CHANGES.txt (Revision 1348960)
HADOOP-8368. Use CMake rather than autotools to build native code (ccccabe =
via tucu) (Revision 1348957)

     Result =3D SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=3DApache-SVN&view=3Drev&rev=
=3D1348960
Files :=20
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt

tucu : http://svn.apache.org/viewcvs.cgi/?root=3DApache-SVN&view=3Drev&rev=
=3D1348957
Files :=20
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/CMakeLists.t=
xt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/config.h.cma=
ke
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
.autom4te.cfg
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
Makefile.am
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
acinclude.m4
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
configure.ac
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
lib/Makefile.am
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
src/org/apache/hadoop/io/compress/lz4/Lz4Compressor.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
src/org/apache/hadoop/io/compress/lz4/Lz4Decompressor.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
src/org/apache/hadoop/io/compress/snappy/SnappyDecompressor.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
src/org/apache/hadoop/io/compress/snappy/org_apache_hadoop_io_compress_snap=
py.h
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
src/org/apache/hadoop/io/compress/zlib/Makefile.am
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
src/org/apache/hadoop/io/compress/zlib/ZlibCompressor.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
src/org/apache/hadoop/io/compress/zlib/ZlibDecompressor.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
src/org/apache/hadoop/io/compress/zlib/org_apache_hadoop_io_compress_zlib.h
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
src/org/apache/hadoop/io/nativeio/NativeIO.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
src/org/apache/hadoop/util/NativeCrc32.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/=
src/org_apache_hadoop.h
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/CMakeLists.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/config.h.cmake
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs=
/Makefile.am
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs=
/acinclude.m4
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs=
/configure.ac
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs=
/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs=
/src/CMakeLists.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs=
/src/Makefile.am
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs=
/src/fuse_dfs.h
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native/Make=
file.am
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native/conf=
igure.ac
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native/m4/a=
pfunctions.m4
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native/m4/a=
pjava.m4
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native/m4/a=
psupport.m4
* /hadoop/common/trunk/hadoop-hdfs-project/pom.xml
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-ser=
ver/hadoop-yarn-server-nodemanager/pom.xml
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-ser=
ver/hadoop-yarn-server-nodemanager/src/CMakeLists.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-ser=
ver/hadoop-yarn-server-nodemanager/src/config.h.cmake
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-ser=
ver/hadoop-yarn-server-nodemanager/src/main/native/container-executor/.auto=
m4te.cfg
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-ser=
ver/hadoop-yarn-server-nodemanager/src/main/native/container-executor/.deps=
/container-executor.Po
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-ser=
ver/hadoop-yarn-server-nodemanager/src/main/native/container-executor/Makef=
ile.am
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-ser=
ver/hadoop-yarn-server-nodemanager/src/main/native/container-executor/confi=
gure.ac
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-ser=
ver/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/=
main.c

               =20
> Use CMake rather than autotools to build native code
> ----------------------------------------------------
>
>                 Key: HADOOP-8368
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8368
>             Project: Hadoop Common
>          Issue Type: Improvement
>    Affects Versions: 2.0.0-alpha
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Minor
>             Fix For: 2.0.1-alpha
>
>         Attachments: HADOOP-8368-b2.001.patch, HADOOP-8368-b2.001.rm.patc=
h, HADOOP-8368-b2.001.trimmed.patch, HADOOP-8368-b2.002.rm.patch, HADOOP-83=
68-b2.002.trimmed.patch, HADOOP-8368-b2.003.rm.patch, HADOOP-8368-b2.003.tr=
immed.patch, HADOOP-8368.001.patch, HADOOP-8368.005.patch, HADOOP-8368.006.=
patch, HADOOP-8368.007.patch, HADOOP-8368.008.patch, HADOOP-8368.009.patch,=
 HADOOP-8368.010.patch, HADOOP-8368.012.half.patch, HADOOP-8368.012.patch, =
HADOOP-8368.012.rm.patch, HADOOP-8368.014.trimmed.patch, HADOOP-8368.015.tr=
immed.patch, HADOOP-8368.016.trimmed.patch, HADOOP-8368.018.trimmed.patch, =
HADOOP-8368.020.rm.patch, HADOOP-8368.020.trimmed.patch, HADOOP-8368.021.tr=
immed.patch, HADOOP-8368.023.trimmed.patch, HADOOP-8368.024.trimmed.patch, =
HADOOP-8368.025.trimmed.patch, HADOOP-8368.026.rm.patch, HADOOP-8368.026.tr=
immed.patch, HADOOP-8368.028.rm.patch, HADOOP-8368.028.trimmed.patch, HADOO=
P-8368.029.patch, HADOOP-8368.030.patch, HADOOP-8368.030.patch, HADOOP-8368=
.030.rm.patch, HADOOP-8368.030.trimmed.patch
>
>
> It would be good to use cmake rather than autotools to build the native (=
C/C++) code in Hadoop.
> Rationale:
> 1. automake depends on shell scripts, which often have problems running o=
n different operating systems.  It would be extremely difficult, and perhap=
s impossible, to use autotools under Windows.  Even if it were possible, it=
 might require horrible workarounds like installing cygwin.  Even on Linux =
variants like Ubuntu 12.04, there are major build issues because /bin/sh is=
 the Dash shell, rather than the Bash shell as it is in other Linux version=
s.  It is currently impossible to build the native code under Ubuntu 12.04 =
because of this problem.
> CMake has robust cross-platform support, including Windows.  It does not =
use shell scripts.
> 2. automake error messages are very confusing.  For example, "autoreconf:=
 cannot empty /tmp/ar0.4849: Is a directory" or "Can't locate object method=
 "path" via package "Autom4te..." are common error messages.  In order to e=
ven start debugging automake problems you need to learn shell, m4, sed, and=
 the a bunch of other things.  With CMake, all you have to learn is the syn=
tax of CMakeLists.txt, which is simple.
> CMake can do all the stuff autotools can, such as making sure that requir=
ed libraries are installed.  There is a Maven plugin for CMake as well.
> 3. Different versions of autotools can have very different behaviors.  Fo=
r example, the version installed under openSUSE defaults to putting librari=
es in /usr/local/lib64, whereas the version shipped with Ubuntu 11.04 defau=
lts to installing the same libraries under /usr/local/lib.  (This is why th=
e FUSE build is currently broken when using OpenSUSE.)  This is another sou=
rce of build failures and complexity.  If things go wrong, you will often g=
et an error message which is incomprehensible to normal humans (see point #=
2).
> CMake allows you to specify the minimum_required_version of CMake that a =
particular CMakeLists.txt will accept.  In addition, CMake maintains strict=
 backwards compatibility between different versions.  This prevents build b=
ugs due to version skew.
> 4. autoconf, automake, and libtool are large and rather slow.  This adds =
to build time.
> For all these reasons, I think we should switch to CMake for compiling na=
tive (C/C++) code in Hadoop.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrato=
rs: https://issues.apache.org/jira/secure/ContactAdministrators!default.jsp=
a
For more information on JIRA, see: http://www.atlassian.com/software/jira