hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11505) Various native parts use bswap incorrectly and unportably
Date Mon, 21 Dec 2015 18:39:46 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15066842#comment-15066842

Colin Patrick McCabe commented on HADOOP-11505:

Thanks for working on this, everyone.

[~aw], if you want to run the native unit tests for hadoop-hdfs, you must build the native
part of hadoop-common first, so that you can have access to {{libhadoop.so}}.  That's why
the old (pre-Yetus) build script had a hacky piece of code that specifically compiled hadoop-common
if anything native had changed.  I don't think the native components inside nativetask and
yarn have the same issue, since I don't think they actually make calls into libhadoop.  So
theoretically we might be able to decouple those from the compilation of hadoop-common.  As
[~cnauroth] commented, one way of doing this would be splitting out stuff like the {{config.h}}
generation code into a separate module or file that could be invoked by all {{CMakeLists.txt}}.

As [~alanburlison] pointed out, the patch here doesn't add any additional dependencies or
change the existing ones.  I think we should address the build dependency issues in a separate

{{MCollectorOutputHandler.cc}}: I'm confused by the {{Endium}} enum here.  First of all, shouldn't
it be {{Endianness}}?  "Endium" isn't a word.  Secondly, it looks hard-coded to {{LARGE_ENDIUM}},
but there is an "if" statement.  Is the intention to add additional flexibility in the future?
 If we're not using this functionality right now, I would rather avoid "speculative code"
that we may or may not end up using.

> Various native parts use bswap incorrectly and unportably
> ---------------------------------------------------------
>                 Key: HADOOP-11505
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11505
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Alan Burlison
>             Fix For: 3.0.0
>         Attachments: HADOOP-11505.001.patch, HADOOP-11505.003.patch, HADOOP-11505.004.patch,
HADOOP-11505.005.patch, HADOOP-11505.006.patch, HADOOP-11505.007.patch
> hadoop-mapreduce-client-nativetask fails to use x86 optimizations in some cases.  Also,
on some alternate, non-x86, non-ARM architectures the generated code is incorrect.  Thanks
to Steve Loughran and Edward Nevill for finding this.

This message was sent by Atlassian JIRA

View raw message