hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8368) Use CMake rather than autotools to build native code
Date Thu, 07 Jun 2012 13:54:24 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291003#comment-13291003
] 

Hudson commented on HADOOP-8368:
--------------------------------

Integrated in Hadoop-Mapreduce-trunk #1103 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1103/])
    svn merge -c -1346491 for re-committing HADOOP-8368. (tucu) (Revision 1347092)

     Result = FAILURE
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347092
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/CMakeLists.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/config.h.cmake
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/.autom4te.cfg
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/Makefile.am
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/acinclude.m4
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/configure.ac
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/lib/Makefile.am
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/compress/lz4/Lz4Compressor.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/compress/lz4/Lz4Decompressor.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/compress/snappy/SnappyDecompressor.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/compress/snappy/org_apache_hadoop_io_compress_snappy.h
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/compress/zlib/Makefile.am
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/compress/zlib/ZlibCompressor.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/compress/zlib/ZlibDecompressor.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/compress/zlib/org_apache_hadoop_io_compress_zlib.h
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/util/NativeCrc32.c
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org_apache_hadoop.h
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/CMakeLists.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/config.h.cmake
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs/Makefile.am
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs/acinclude.m4
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs/configure.ac
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs/src/CMakeLists.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs/src/Makefile.am
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/fuse-dfs/src/fuse_dfs.h
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native/Makefile.am
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native/configure.ac
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native/m4/apfunctions.m4
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native/m4/apjava.m4
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native/m4/apsupport.m4
* /hadoop/common/trunk/hadoop-hdfs-project/pom.xml
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/CMakeLists.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/config.h.cmake
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/.autom4te.cfg
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/.deps/container-executor.Po
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/Makefile.am
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/configure.ac
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/main.c

                
> Use CMake rather than autotools to build native code
> ----------------------------------------------------
>
>                 Key: HADOOP-8368
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8368
>             Project: Hadoop Common
>          Issue Type: Improvement
>    Affects Versions: 2.0.0-alpha
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Minor
>             Fix For: 2.0.1-alpha
>
>         Attachments: HADOOP-8368-b2.001.patch, HADOOP-8368-b2.001.rm.patch, HADOOP-8368-b2.001.trimmed.patch,
HADOOP-8368-b2.002.rm.patch, HADOOP-8368-b2.002.trimmed.patch, HADOOP-8368.001.patch, HADOOP-8368.005.patch,
HADOOP-8368.006.patch, HADOOP-8368.007.patch, HADOOP-8368.008.patch, HADOOP-8368.009.patch,
HADOOP-8368.010.patch, HADOOP-8368.012.half.patch, HADOOP-8368.012.patch, HADOOP-8368.012.rm.patch,
HADOOP-8368.014.trimmed.patch, HADOOP-8368.015.trimmed.patch, HADOOP-8368.016.trimmed.patch,
HADOOP-8368.018.trimmed.patch, HADOOP-8368.020.rm.patch, HADOOP-8368.020.trimmed.patch, HADOOP-8368.021.trimmed.patch,
HADOOP-8368.023.trimmed.patch, HADOOP-8368.024.trimmed.patch, HADOOP-8368.025.trimmed.patch,
HADOOP-8368.026.rm.patch, HADOOP-8368.026.trimmed.patch, HADOOP-8368.028.rm.patch, HADOOP-8368.028.trimmed.patch
>
>
> It would be good to use cmake rather than autotools to build the native (C/C++) code
in Hadoop.
> Rationale:
> 1. automake depends on shell scripts, which often have problems running on different
operating systems.  It would be extremely difficult, and perhaps impossible, to use autotools
under Windows.  Even if it were possible, it might require horrible workarounds like installing
cygwin.  Even on Linux variants like Ubuntu 12.04, there are major build issues because /bin/sh
is the Dash shell, rather than the Bash shell as it is in other Linux versions.  It is currently
impossible to build the native code under Ubuntu 12.04 because of this problem.
> CMake has robust cross-platform support, including Windows.  It does not use shell scripts.
> 2. automake error messages are very confusing.  For example, "autoreconf: cannot empty
/tmp/ar0.4849: Is a directory" or "Can't locate object method "path" via package "Autom4te..."
are common error messages.  In order to even start debugging automake problems you need to
learn shell, m4, sed, and the a bunch of other things.  With CMake, all you have to learn
is the syntax of CMakeLists.txt, which is simple.
> CMake can do all the stuff autotools can, such as making sure that required libraries
are installed.  There is a Maven plugin for CMake as well.
> 3. Different versions of autotools can have very different behaviors.  For example, the
version installed under openSUSE defaults to putting libraries in /usr/local/lib64, whereas
the version shipped with Ubuntu 11.04 defaults to installing the same libraries under /usr/local/lib.
 (This is why the FUSE build is currently broken when using OpenSUSE.)  This is another source
of build failures and complexity.  If things go wrong, you will often get an error message
which is incomprehensible to normal humans (see point #2).
> CMake allows you to specify the minimum_required_version of CMake that a particular CMakeLists.txt
will accept.  In addition, CMake maintains strict backwards compatibility between different
versions.  This prevents build bugs due to version skew.
> 4. autoconf, automake, and libtool are large and rather slow.  This adds to build time.
> For all these reasons, I think we should switch to CMake for compiling native (C/C++)
code in Hadoop.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message