hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Bautin <bautin.mailing.li...@gmail.com>
Subject Re: DISCUSS: Have hbase require at least hadoop 1.0.0 in hbase 0.96.0?
Date Wed, 07 Mar 2012 19:42:38 GMT
The current support for multiple versions of HDFS is in my opinion actually
one of the strengths of HBase, and the project will lose that advantage if
we cut support for earlier versions of Hadoop. I think HBase should only
require the simplest possible universally available subset of HDFS API, and
security should be an optional feature, discovered through reflection or
enabled in some other ways.

We have a custom version of Hadoop at Facebook that is not planning to
implement security any time soon. This version of Hadoop runs underneath
what we believe to be some of the largest existing production HBase
deployments. We are currently running the 0.89-fb version of HBase in
production, but are considering moving to a more recent version of HBase at
some point, and it would be great to be able to do that independently of
changing the underlying Hadoop distribution for migration complexity
reasons. Currently we are able to run public HBase trunk on our version of
Hadoop, but once in a while we have to satisfy new dependences on Hadoop
features that are added to HBase. If the changes proposed in this thread
happen, we would have to pull in a lot more security-related dependencies
into our version of Hadoop and, most likely, implement a lot of no-op
stubs. However, that may not be a trivial project, and it certainly would
not add any clarity or value to our Hadoop codebase or HBase / HDFS

I imagine there are other custom flavors of Hadoop out there where HBase
support would be desirable. For example, does MapR implement the same
security API as Hadoop 1.0.0 does? Restricting HBase to a smaller subset of
Hadoop versions complicates life for existing users, and makes HBase a less
likely choice for new users, who could go with something like Hypertable
where they have an extra abstraction layer between the database and the
underlying distributed file system implementation.


On Wed, Mar 7, 2012 at 10:20 AM, Devaraj Das <ddas@hortonworks.com> wrote:

> Given that the token/ugi APIs are being used in other ecosystem components
> too (like Hive, HCatalog & Oozie), and in general, that security model will
> probably hold for other projects too, I think that its not an unfair
> expectation from Hadoop that it should maintain compatibility on UGI/Token*
> interfaces (*smile*).
> On Mar 6, 2012, at 11:57 AM, Arun C Murthy wrote:
> > Andy - could you please start a discussion?
> >
> > We could, at the very least, mark UGI as LimitedPrivate for HBase and
> work with you guys to maintain compatibility for the future. Makes sense?
> >
> > thanks,
> > Arun
> >
> > On Mar 6, 2012, at 10:21 AM, Andrew Purtell wrote:
> >
> >> After that, I believe we can merge the security sources in. However we
> may have an issue going forward because UGI is an unstable/private API.
> Needs sorting out with core at some point.
> >>
> >> Best regards,
> >>
> >>   - Andy
> >>
> >>
> >> On Mar 6, 2012, at 9:55 AM, Stack <stack@duboce.net> wrote:
> >>
> >>> On Tue, Mar 6, 2012 at 9:10 AM, Andrew Purtell <apurtell@apache.org>
> wrote:
> >>>> ...however we can't easily build a single artifact because the secure
> RPC engine, as it interacts with the Hadoop auth framework, must use
> UserGroupInformation.
> >>>>
> >>>
> >>> OK.  So security story needs a bit of work.  Sounds like we have
> >>> enough votes though to require hadoop 1.0.0 at least in 0.96.
> >>>
> >>> St.Ack
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message