hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: DISCUSS: Have hbase require at least hadoop 1.0.0 in hbase 0.96.0?
Date Wed, 07 Mar 2012 20:17:31 GMT
+1 on what Mikhail said below.

On Wed, Mar 7, 2012 at 11:42 AM, Mikhail Bautin <
bautin.mailing.lists@gmail.com> wrote:

> The current support for multiple versions of HDFS is in my opinion actually
> one of the strengths of HBase, and the project will lose that advantage if
> we cut support for earlier versions of Hadoop. I think HBase should only
> require the simplest possible universally available subset of HDFS API, and
> security should be an optional feature, discovered through reflection or
> enabled in some other ways.
>
> We have a custom version of Hadoop at Facebook that is not planning to
> implement security any time soon. This version of Hadoop runs underneath
> what we believe to be some of the largest existing production HBase
> deployments. We are currently running the 0.89-fb version of HBase in
> production, but are considering moving to a more recent version of HBase at
> some point, and it would be great to be able to do that independently of
> changing the underlying Hadoop distribution for migration complexity
> reasons. Currently we are able to run public HBase trunk on our version of
> Hadoop, but once in a while we have to satisfy new dependences on Hadoop
> features that are added to HBase. If the changes proposed in this thread
> happen, we would have to pull in a lot more security-related dependencies
> into our version of Hadoop and, most likely, implement a lot of no-op
> stubs. However, that may not be a trivial project, and it certainly would
> not add any clarity or value to our Hadoop codebase or HBase / HDFS
> interaction.
>
> I imagine there are other custom flavors of Hadoop out there where HBase
> support would be desirable. For example, does MapR implement the same
> security API as Hadoop 1.0.0 does? Restricting HBase to a smaller subset of
> Hadoop versions complicates life for existing users, and makes HBase a less
> likely choice for new users, who could go with something like Hypertable
> where they have an extra abstraction layer between the database and the
> underlying distributed file system implementation.
>
> Thanks,
> --Mikhail
>
> On Wed, Mar 7, 2012 at 10:20 AM, Devaraj Das <ddas@hortonworks.com> wrote:
>
> > Given that the token/ugi APIs are being used in other ecosystem
> components
> > too (like Hive, HCatalog & Oozie), and in general, that security model
> will
> > probably hold for other projects too, I think that its not an unfair
> > expectation from Hadoop that it should maintain compatibility on
> UGI/Token*
> > interfaces (*smile*).
> >
> > On Mar 6, 2012, at 11:57 AM, Arun C Murthy wrote:
> >
> > > Andy - could you please start a discussion?
> > >
> > > We could, at the very least, mark UGI as LimitedPrivate for HBase and
> > work with you guys to maintain compatibility for the future. Makes sense?
> > >
> > > thanks,
> > > Arun
> > >
> > > On Mar 6, 2012, at 10:21 AM, Andrew Purtell wrote:
> > >
> > >> After that, I believe we can merge the security sources in. However we
> > may have an issue going forward because UGI is an unstable/private API.
> > Needs sorting out with core at some point.
> > >>
> > >> Best regards,
> > >>
> > >>   - Andy
> > >>
> > >>
> > >> On Mar 6, 2012, at 9:55 AM, Stack <stack@duboce.net> wrote:
> > >>
> > >>> On Tue, Mar 6, 2012 at 9:10 AM, Andrew Purtell <apurtell@apache.org>
> > wrote:
> > >>>> ...however we can't easily build a single artifact because the
> secure
> > RPC engine, as it interacts with the Hadoop auth framework, must use
> > UserGroupInformation.
> > >>>>
> > >>>
> > >>> OK.  So security story needs a bit of work.  Sounds like we have
> > >>> enough votes though to require hadoop 1.0.0 at least in 0.96.
> > >>>
> > >>> St.Ack
> > >
> > > --
> > > Arun C. Murthy
> > > Hortonworks Inc.
> > > http://hortonworks.com/
> > >
> > >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message