hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@yahoo.com>
Subject Re: DISCUSS: Have hbase require at least hadoop 1.0.0 in hbase 0.96.0?
Date Wed, 07 Mar 2012 22:39:48 GMT
I have no strong opinion either way: separate profile or merge could be made to work. I'm happy
to maintain security related sources as a module as long as the necessary accommodations are
made by other devs; e.g. don't break our sources by changing the coprocessor API or RPC without
also fixing up the security module or at least making it straightforward for us to do those
fixups. 

Best regards,

    - Andy


On Mar 7, 2012, at 11:42 AM, Mikhail Bautin <bautin.mailing.lists@gmail.com> wrote:

> The current support for multiple versions of HDFS is in my opinion actually
> one of the strengths of HBase, and the project will lose that advantage if
> we cut support for earlier versions of Hadoop. I think HBase should only
> require the simplest possible universally available subset of HDFS API, and
> security should be an optional feature, discovered through reflection or
> enabled in some other ways.
> 
> We have a custom version of Hadoop at Facebook that is not planning to
> implement security any time soon. This version of Hadoop runs underneath
> what we believe to be some of the largest existing production HBase
> deployments. We are currently running the 0.89-fb version of HBase in
> production, but are considering moving to a more recent version of HBase at
> some point, and it would be great to be able to do that independently of
> changing the underlying Hadoop distribution for migration complexity
> reasons. Currently we are able to run public HBase trunk on our version of
> Hadoop, but once in a while we have to satisfy new dependences on Hadoop
> features that are added to HBase. If the changes proposed in this thread
> happen, we would have to pull in a lot more security-related dependencies
> into our version of Hadoop and, most likely, implement a lot of no-op
> stubs. However, that may not be a trivial project, and it certainly would
> not add any clarity or value to our Hadoop codebase or HBase / HDFS
> interaction.
> 
> I imagine there are other custom flavors of Hadoop out there where HBase
> support would be desirable. For example, does MapR implement the same
> security API as Hadoop 1.0.0 does? Restricting HBase to a smaller subset of
> Hadoop versions complicates life for existing users, and makes HBase a less
> likely choice for new users, who could go with something like Hypertable
> where they have an extra abstraction layer between the database and the
> underlying distributed file system implementation.
> 
> Thanks,
> --Mikhail
> 
> On Wed, Mar 7, 2012 at 10:20 AM, Devaraj Das <ddas@hortonworks.com> wrote:
> 
>> Given that the token/ugi APIs are being used in other ecosystem components
>> too (like Hive, HCatalog & Oozie), and in general, that security model will
>> probably hold for other projects too, I think that its not an unfair
>> expectation from Hadoop that it should maintain compatibility on UGI/Token*
>> interfaces (*smile*).
>> 
>> On Mar 6, 2012, at 11:57 AM, Arun C Murthy wrote:
>> 
>>> Andy - could you please start a discussion?
>>> 
>>> We could, at the very least, mark UGI as LimitedPrivate for HBase and
>> work with you guys to maintain compatibility for the future. Makes sense?
>>> 
>>> thanks,
>>> Arun
>>> 
>>> On Mar 6, 2012, at 10:21 AM, Andrew Purtell wrote:
>>> 
>>>> After that, I believe we can merge the security sources in. However we
>> may have an issue going forward because UGI is an unstable/private API.
>> Needs sorting out with core at some point.
>>>> 
>>>> Best regards,
>>>> 
>>>>  - Andy
>>>> 
>>>> 
>>>> On Mar 6, 2012, at 9:55 AM, Stack <stack@duboce.net> wrote:
>>>> 
>>>>> On Tue, Mar 6, 2012 at 9:10 AM, Andrew Purtell <apurtell@apache.org>
>> wrote:
>>>>>> ...however we can't easily build a single artifact because the secure
>> RPC engine, as it interacts with the Hadoop auth framework, must use
>> UserGroupInformation.
>>>>>> 
>>>>> 
>>>>> OK.  So security story needs a bit of work.  Sounds like we have
>>>>> enough votes though to require hadoop 1.0.0 at least in 0.96.
>>>>> 
>>>>> St.Ack
>>> 
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> 
>> 
>> 

Mime
View raw message