hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Burlison (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11505) Various native parts use bswap incorrectly and unportably
Date Thu, 10 Dec 2015 13:53:11 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15050982#comment-15050982
] 

Alan Burlison commented on HADOOP-11505:
----------------------------------------

I've dug into this some more and there's already a bigger problem in the form of the {{config.h}}
file, which is generated by the CMake script in {{hadoop-common-project/hadoop-common}} (the
same place where {{hadoop_endian.h}} is generated). {{config.h}} is included in 31 other places
across 4 subcomponents, with 7 files in {{hadoop-mapreduce-project/hadoop-mapreduce-client}}
including {{config.h}}, so builds of {{hadoop-mapreduce-project/hadoop-mapreduce-client}}
already require {{hadoop-common-project/hadoop-common}} to be built first. That's not surprising
as {{hadoop-common-project/hadoop-common}} is supposed to contain the common Hadoop code.
Also, {{hadoop-mapreduce-project/hadoop-mapreduce-client}} has a dependency on the lz4 code
that's in {{hadoop-common-project/hadoop-common}} so even if the header file issues were addressed
you'd still have to build {{hadoop-common-project/hadoop-common}} before building {{hadoop-mapreduce-project/hadoop-mapreduce-client}}.

I was going to suggest that straightening out the config.h and hadoop_endian.h generation
should be done as one piece of work, but as {{hadoop-mapreduce-project/hadoop-mapreduce-client}}
already requires code from {{hadoop-common-project/hadoop-common}} there's really no point.
I think this patch should therefore stand as-is.

> Various native parts use bswap incorrectly and unportably
> ---------------------------------------------------------
>
>                 Key: HADOOP-11505
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11505
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Alan Burlison
>             Fix For: 3.0.0
>
>         Attachments: HADOOP-11505.001.patch, HADOOP-11505.003.patch, HADOOP-11505.004.patch,
HADOOP-11505.005.patch, HADOOP-11505.006.patch, HADOOP-11505.007.patch
>
>
> hadoop-mapreduce-client-nativetask fails to use x86 optimizations in some cases.  Also,
on some alternate, non-x86, non-ARM architectures the generated code is incorrect.  Thanks
to Steve Loughran and Edward Nevill for finding this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message