hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-11505) Various native parts use bswap incorrectly and unportably
Date Mon, 11 Jan 2016 22:21:40 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15092797#comment-15092797
] 

Allen Wittenauer edited comment on HADOOP-11505 at 1/11/16 10:21 PM:
---------------------------------------------------------------------

(Moving response to a new comment so it doesn't get lost)



was (Author: aw):
I'm not sure I agree with [~alanburlison]'s assessment, but I might be missing something.
 All of the cmake files appear to be building their own config.h files.  Plus *zero* of our
native builds are failing when we don't build hadoop-common prior to building anything else.

{code}
hadoop-common-project/hadoop-common/src/CMakeLists.txt:configure_file(${CMAKE_SOURCE_DIR}/config.h.cmake
${CMAKE_BINARY_DIR}/config.h)
hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt:configure_file(${CMAKE_SOURCE_DIR}/config.h.cmake
${CMAKE_BINARY_DIR}/config.h)
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/CMakeLists.txt:configure_file(${CMAKE_SOURCE_DIR}/config.h.cmake
${CMAKE_BINARY_DIR}/config.h)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/CMakeLists.txt:configure_file(${CMAKE_SOURCE_DIR}/config.h.cmake
${CMAKE_BINARY_DIR}/config.h)
{code}


Additionally, for lz4, the pom.xml file does the magic to make it work:

{code}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml:
                   <copy file="${basedir}/../../../hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/compress/lz4/lz4.h"
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml:
                   <copy file="${basedir}/../../../hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/compress/lz4/lz4.c"
{code}


So no, really, this is adding a new compile-time dependency that isn't there.


> Various native parts use bswap incorrectly and unportably
> ---------------------------------------------------------
>
>                 Key: HADOOP-11505
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11505
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Alan Burlison
>             Fix For: 3.0.0
>
>         Attachments: HADOOP-11505.001.patch, HADOOP-11505.003.patch, HADOOP-11505.004.patch,
HADOOP-11505.005.patch, HADOOP-11505.006.patch, HADOOP-11505.007.patch
>
>
> hadoop-mapreduce-client-nativetask fails to use x86 optimizations in some cases.  Also,
on some alternate, non-x86, non-ARM architectures the generated code is incorrect.  Thanks
to Steve Loughran and Edward Nevill for finding this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message