incubator-hcatalog-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nitay Joffe <nitay.jo...@gmail.com>
Subject Re: Support for non-secure Hadoop?
Date Sat, 20 Oct 2012 05:31:46 GMT
Yeah I'm whipping up an internal diff that rips out security stuff. I'll look into the shims
layer and if it's not too much burden to maintain post a diff with it.

I spoke with our Hadoop experts - unfortunately no on the Hadoop upgrade. Way too much inertia.

- Nitay

On Oct 17, 2012, at 4:16 PM, Travis Crawford <traviscrawford@gmail.com> wrote:

> Guessing you're primarily interested in hcatalog-core.jar; a quick
> look at the security imports shows the following classes.
> 
> $ find ~/src/hcatalog/src/java -name '*.java' -exec grep -l 'import
> org.apache.hadoop.security' {} \;
> /Users/travis/src/hcatalog/src/java/org/apache/hcatalog/common/HCatUtil.java
> /Users/travis/src/hcatalog/src/java/org/apache/hcatalog/common/HiveClientCache.java
> /Users/travis/src/hcatalog/src/java/org/apache/hcatalog/mapreduce/FileOutputCommitterContainer.java
> /Users/travis/src/hcatalog/src/java/org/apache/hcatalog/mapreduce/HCatOutputFormat.java
> /Users/travis/src/hcatalog/src/java/org/apache/hcatalog/mapreduce/Security.java
> /Users/travis/src/hcatalog/src/java/org/apache/hcatalog/security/HdfsAuthorizationProvider.java
> $
> 
> My guess is there's not much general interest, but if the changes are
> not burdensome to maintain you could look at adding a shims layer for
> your Hadoop version, and associated refactor. Are y'all ever getting
> back on the Apache releases of Hadoop? Seems like this is going to be
> increasingly problematic.
> 
> --travis
> 
> 
> On Tue, Oct 16, 2012 at 5:44 PM, Nitay Joffe <nitayj@gmail.com> wrote:
>> Hey guys,
>> 
>> Is there any plans to support non-secure Hadoop (e.g. Facebook's branch of Hadoop
based on 0.20). Specifically I am talking about hadoop that contain little to no classes at
all under org.apache.hadoop.security, for example no Credentials class at all.
>> 
>> We need this in order to use HCatalog at Facebook. Months ago we hacked it by ripping
out any mention of security from the code by hand and using a version based off of that, but
that makes it hard to update to latest trunk. I would think this would be a necessity in order
to support Hive since it doesn't have any security stuff also?
>> 
>> Thanks,
>> - Nitay


Mime
View raw message