incubator-hcatalog-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Travis Crawford <traviscrawf...@gmail.com>
Subject Re: Support for non-secure Hadoop?
Date Wed, 17 Oct 2012 23:16:07 GMT
Guessing you're primarily interested in hcatalog-core.jar; a quick
look at the security imports shows the following classes.

$ find ~/src/hcatalog/src/java -name '*.java' -exec grep -l 'import
org.apache.hadoop.security' {} \;
/Users/travis/src/hcatalog/src/java/org/apache/hcatalog/common/HCatUtil.java
/Users/travis/src/hcatalog/src/java/org/apache/hcatalog/common/HiveClientCache.java
/Users/travis/src/hcatalog/src/java/org/apache/hcatalog/mapreduce/FileOutputCommitterContainer.java
/Users/travis/src/hcatalog/src/java/org/apache/hcatalog/mapreduce/HCatOutputFormat.java
/Users/travis/src/hcatalog/src/java/org/apache/hcatalog/mapreduce/Security.java
/Users/travis/src/hcatalog/src/java/org/apache/hcatalog/security/HdfsAuthorizationProvider.java
$

My guess is there's not much general interest, but if the changes are
not burdensome to maintain you could look at adding a shims layer for
your Hadoop version, and associated refactor. Are y'all ever getting
back on the Apache releases of Hadoop? Seems like this is going to be
increasingly problematic.

--travis


On Tue, Oct 16, 2012 at 5:44 PM, Nitay Joffe <nitayj@gmail.com> wrote:
> Hey guys,
>
> Is there any plans to support non-secure Hadoop (e.g. Facebook's branch of Hadoop based
on 0.20). Specifically I am talking about hadoop that contain little to no classes at all
under org.apache.hadoop.security, for example no Credentials class at all.
>
> We need this in order to use HCatalog at Facebook. Months ago we hacked it by ripping
out any mention of security from the code by hand and using a version based off of that, but
that makes it hard to update to latest trunk. I would think this would be a necessity in order
to support Hive since it doesn't have any security stuff also?
>
> Thanks,
> - Nitay

Mime
View raw message