hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@yahoo.com>
Subject Re: DISCUSS: Have hbase require at least hadoop 1.0.0 in hbase 0.96.0?
Date Wed, 07 Mar 2012 22:39:48 GMT
I have no strong opinion either way: separate profile or merge could be made to work. I'm happy
to maintain security related sources as a module as long as the necessary accommodations are
made by other devs; e.g. don't break our sources by changing the coprocessor API or RPC without
also fixing up the security module or at least making it straightforward for us to do those

Best regards,

    - Andy

On Mar 7, 2012, at 11:42 AM, Mikhail Bautin <bautin.mailing.lists@gmail.com> wrote:

> The current support for multiple versions of HDFS is in my opinion actually
> one of the strengths of HBase, and the project will lose that advantage if
> we cut support for earlier versions of Hadoop. I think HBase should only
> require the simplest possible universally available subset of HDFS API, and
> security should be an optional feature, discovered through reflection or
> enabled in some other ways.
> We have a custom version of Hadoop at Facebook that is not planning to
> implement security any time soon. This version of Hadoop runs underneath
> what we believe to be some of the largest existing production HBase
> deployments. We are currently running the 0.89-fb version of HBase in
> production, but are considering moving to a more recent version of HBase at
> some point, and it would be great to be able to do that independently of
> changing the underlying Hadoop distribution for migration complexity
> reasons. Currently we are able to run public HBase trunk on our version of
> Hadoop, but once in a while we have to satisfy new dependences on Hadoop
> features that are added to HBase. If the changes proposed in this thread
> happen, we would have to pull in a lot more security-related dependencies
> into our version of Hadoop and, most likely, implement a lot of no-op
> stubs. However, that may not be a trivial project, and it certainly would
> not add any clarity or value to our Hadoop codebase or HBase / HDFS
> interaction.
> I imagine there are other custom flavors of Hadoop out there where HBase
> support would be desirable. For example, does MapR implement the same
> security API as Hadoop 1.0.0 does? Restricting HBase to a smaller subset of
> Hadoop versions complicates life for existing users, and makes HBase a less
> likely choice for new users, who could go with something like Hypertable
> where they have an extra abstraction layer between the database and the
> underlying distributed file system implementation.
> Thanks,
> --Mikhail
> On Wed, Mar 7, 2012 at 10:20 AM, Devaraj Das <ddas@hortonworks.com> wrote:
>> Given that the token/ugi APIs are being used in other ecosystem components
>> too (like Hive, HCatalog & Oozie), and in general, that security model will
>> probably hold for other projects too, I think that its not an unfair
>> expectation from Hadoop that it should maintain compatibility on UGI/Token*
>> interfaces (*smile*).
>> On Mar 6, 2012, at 11:57 AM, Arun C Murthy wrote:
>>> Andy - could you please start a discussion?
>>> We could, at the very least, mark UGI as LimitedPrivate for HBase and
>> work with you guys to maintain compatibility for the future. Makes sense?
>>> thanks,
>>> Arun
>>> On Mar 6, 2012, at 10:21 AM, Andrew Purtell wrote:
>>>> After that, I believe we can merge the security sources in. However we
>> may have an issue going forward because UGI is an unstable/private API.
>> Needs sorting out with core at some point.
>>>> Best regards,
>>>>  - Andy
>>>> On Mar 6, 2012, at 9:55 AM, Stack <stack@duboce.net> wrote:
>>>>> On Tue, Mar 6, 2012 at 9:10 AM, Andrew Purtell <apurtell@apache.org>
>> wrote:
>>>>>> ...however we can't easily build a single artifact because the secure
>> RPC engine, as it interacts with the Hadoop auth framework, must use
>> UserGroupInformation.
>>>>> OK.  So security story needs a bit of work.  Sounds like we have
>>>>> enough votes though to require hadoop 1.0.0 at least in 0.96.
>>>>> St.Ack
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/

View raw message