hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7405) libhadoop is all or nothing
Date Mon, 20 Jun 2011 16:14:52 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13052033#comment-13052033

Alejandro Abdelnur commented on HADOOP-7405:

Since Hadoop Kerberos Mac OS X support was never fully there, it is not possible to compile
libhadoop due to some compiler errors.

Having libhadoop working partially on different platforms does not seem a good idea as it
will mislead users to believe that a particular platform is supported when is 'partially'
supported. I'd rather go for the principle of least surprise here (all or nothing), this will
make Hadoop easier to user.

A while ago I've opened a HADOOP-7083 to enable running Hadoop with Kerberos ON without relying
on some libhadoop functionality and the argument there was that doing that was a security

I'm not arguing for HADOOP-7083 to be considered again, what I'm trying to state is that if
we go the partial libhadoop functionality we'll have to come up with a checklist that will
have to identify if certain functionality provided by native libs is optional, can hadoop
work without it, is a security risk, etc. This will complicate the not only the lives of developers
but of users.

Because of this my take is that if we require native code to run Hadoop, we should provide
the full set of native code for each platform we are building for. Only for functionality
that there is Java fallback (at a cost of performance) native-code should be optional.

> libhadoop is all or nothing
> ---------------------------
>                 Key: HADOOP-7405
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7405
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: native
>    Affects Versions:, 0.23.0
>         Environment: Everything not Linux
>            Reporter: Allen Wittenauer
>            Priority: Blocker
>              Labels: regression
> As a result of a ton of new code in libhadoop being added in 0.20.203/0.22, a lot of
features that used to work no longer do reliably.  The most common problem is native compression,
but other issues such as Mac OS X's group support broke as well.  The native code checks need
to be refactored such that libhadoop.so should report what it supports rather than having
the Java-side assume that if it loads, it is all supported.  This would allow us to stub routines
until they've been vetted, removing the chances of such regressions appearing in the future.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message