hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11127) Improve versioning and compatibility support in native library for downstream hadoop-common users.
Date Thu, 08 Oct 2015 19:45:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949249#comment-14949249

Steve Loughran commented on HADOOP-11127:

one of the problems with naming policies is that it'll have to map 2.7.1 to 2.7.0, etc, and
it will require every hadoop installation to have the complete set of native libs for the
target systems.

In-JAR bundling is what snappy does. 

Where it gets fun in hadoop is that there is the need to handle the systems we declare supported,
(rhel, debian, osx, windows), which complicates the build process somewhat. That native lib
is going to have to be built with a pool of VMs, or with help from others (I can see solaris
as an example). We'd also need a failback mechanism where, if the lib isn't in the JAR, some
option (config file setting?) lists the location. Or at that point, fall back to libhadoop.

> Improve versioning and compatibility support in native library for downstream hadoop-common
> --------------------------------------------------------------------------------------------------
>                 Key: HADOOP-11127
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11127
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: native
>            Reporter: Chris Nauroth
>            Assignee: Alan Burlison
>         Attachments: HADOOP-11064.003.patch, proposal.txt
> There is no compatibility policy enforced on the JNI function signatures implemented
in the native library.  This library typically is deployed to all nodes in a cluster, built
from a specific source code version.  However, downstream applications that want to run in
that cluster might choose to bundle a hadoop-common jar at a different version.  Since there
is no compatibility policy, this can cause link errors at runtime when the native function
signatures expected by hadoop-common.jar do not exist in libhadoop.so/hadoop.dll.

This message was sent by Atlassian JIRA

View raw message