hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: DISCUSS: Have hbase require at least hadoop 1.0.0 in hbase 0.96.0?
Date Thu, 08 Mar 2012 06:09:25 GMT
On Wed, Mar 7, 2012 at 11:42 AM, Mikhail Bautin
<bautin.mailing.lists@gmail.com> wrote:
> The current support for multiple versions of HDFS is in my opinion actually
> one of the strengths of HBase, and the project will lose that advantage if
> we cut support for earlier versions of Hadoop.

It just gets a little tough to keep up when the span to support is
broad: branch-0.20-append up through 0.23.x.  I'm not sure if its
tenable keeping it up after we get beyond a certain breadth.

The issue that prompted this discussion was in part "HBASE-5419)
FileAlreadyExistsException has moved from mapred to fs package", a
helpful patch by Dhruba to get us off a deprecated class.  Its
application will break our building against hadoop's older than 1.0.0
(I believe).

I suppose we can keep up (hacky) reflection but at a certain stage its
maintenance becomes "difficult".

> I think HBase should only
> require the simplest possible universally available subset of HDFS API

This notion.  I like.  How would we ensure we keep to a narrow subset
(excepting security for the moment)?  If we want to use an exotic hdfs
api, we go there via reflection?

>, and
> security should be an optional feature, discovered through reflection or
> enabled in some other ways.

If we can't assume 1.0.0, and you've made a point that we can't and
shouldn't (because we'd be leaving behind our biggest deploy -- which
would just be silly), then security is done via the modularization
route that has been discussed previous and that has had some work
applied (you fellas good w/ that?).

> If the changes proposed in this thread
> happen, we would have to pull in a lot more security-related dependencies
> into our version of Hadoop and, most likely, implement a lot of no-op
> stubs.

Lets not have you have to do this.

How do you suggest we ensure we minimize you or anyone else having to
address '...new dependencies on Hadoop features that are added to

> I imagine there are other custom flavors of Hadoop out there where HBase
> support would be desirable. For example, does MapR implement the same
> security API as Hadoop 1.0.0 does?

I don't know.  I hoped the lads over there would speak up if this were
a suggestion that would mess them up.

> Restricting HBase to a smaller subset of
> Hadoop versions complicates life for existing users...

Agreed but I was thinking that some versions of Hadoop so old and odd
-- e.g. branch-0.20-append -- that we could leave them behind by the
time we get to 0.96, which I SWAG to be autumn of this year.

Good on you Mikhail,

View raw message