hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11064) UnsatisifedLinkError with hadoop 2.4 JARs on hadoop-2.6 due NativeCRC32 method changes
Date Tue, 09 Sep 2014 19:57:29 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14127468#comment-14127468
] 

Colin Patrick McCabe commented on HADOOP-11064:
-----------------------------------------------

bq. BTW, I believe the patch you posted would break Windows. NativeCodeLoader would attempt
to load hadoop-<version>.dll, but the Windows native build hasn't been updated to produce
a hadoop-<version>.dll instead of a hadoop.dll.

Perhaps I'm missing something obvious here, but shouldn't the following code be enough to
produce {{hadoop-<version>.dll}} under Windows?

{code}
SET(CMAKE_BUILD_WITH_INSTALL_RPATH TRUE)
SET(HADOOP_LIB_NAME hadoop-${HADOOP_VERSION})
add_dual_library(${HADOOP_LIB_NAME}
    main/native/src/exception.c
    ....
{code}

Would be nice if someone could give the patch a try on Windows and give me some feedback.

bq. I'm +1 for short-term reinstatement of the old function signatures and ongoing work on
the long-term policy. Colin, I agree that it's not ideal to lock ourselves into maintaining
every function signature that ever existed over time. At least speaking for myself, I have
no intention of pointing at this as a precedent to block implementation of a proper versioning
scheme. Hopefully that helps ease some of the very legitimate concerns that you raised.

There are a few months until 2.6 will be released-- do we really need to hack this?  I think
the versioning solution is quite clean and easy to implement.  Trying to mix build products
from different releases, even as a short-term solution, just puts us down the wrong path,
I think-- the same as people trying to mix jars from different releases.

> UnsatisifedLinkError with hadoop 2.4 JARs on hadoop-2.6 due NativeCRC32 method changes
> --------------------------------------------------------------------------------------
>
>                 Key: HADOOP-11064
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11064
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: native
>    Affects Versions: 2.6.0
>         Environment: Hadoop 2.6 cluster, trying to run code containing hadoop 2.4 JARs
>            Reporter: Steve Loughran
>            Assignee: Colin Patrick McCabe
>            Priority: Blocker
>         Attachments: HADOOP-11064.001.patch
>
>
> The private native method names and signatures in {{NativeCrc32}} were changed in HDFS-6561
... as a result hadoop-common-2.4 JARs get unsatisifed link errors when they try to perform
checksums. 
> This essentially stops Hadoop 2.4 applications running on Hadoop 2.6 unless rebuilt and
repackaged with the hadoop- 2.6 JARs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message