hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <ey...@hortonworks.com>
Subject Re: HBase .92 maven artifacts compiled against different releases of Hadoop
Date Fri, 11 Nov 2011 18:45:23 GMT
In general, if HBase uses only the public API of Hadoop.  HBase should be able to work with
multiple version of Hadoop.  However, the reality is not so trivial.  Some Hadoop API are
not fully compatible between major version of Hadoop.  This was observed between HBase 0.90.x
and Hadoop 0.20.203.  i.e. First introduction of Hadoop Metrics V2 Framework and removed Hadoop
Metrics V1 framework caused Hadoop 0.20.200-203 to be incompatible with HBase.  Some effort
was put into restore and forward porting features to ensure HBase 0.90.x and Hadoop 0.20.205.0
can work together.  I recommend that one HBase release should be certified for one major release
of Hadoop to reduce risk.  Perhaps when public Hadoop API are rock solid, then it will become
feasible to have a version of HBase that work across multiple version of Hadoop.

In proposed HBase structure layout change (HBASE-4337), the packaging process excludes inclusion
of Hadoop jar file, and pick up from constructed class path.  In the effort of ensuring Hadoop
related technology can work together in integrated fashion (File system layout change in HADOOP-6255).
 This is the starting point to ensure that Hadoop can be swap out with a different major version
for test.  Once the proposed structure is adopted, HBase community can setup integration test
for HBase with multiple Hadoop major release.

regards,
Eric

On Nov 11, 2011, at 9:27 AM, Stack wrote:

> On Fri, Nov 11, 2011 at 8:58 AM, Roman Shaposhnik <rvs@apache.org> wrote:
>> On Fri, Nov 11, 2011 at 8:56 AM, Alejandro Abdelnur <tucu@cloudera.com> wrote:
>>> Hi Stack,
>>> 
>>> Are all those builds exactly the same bits?
>> 
>> No, they are not. That's exactly the problem I was alluding to.
>> 
> 
> They are not?  We were trying to use reflection to figure what our
> underpinnings are and then proceed accordingly.  I can imagine we've
> missed spots but thats overall intent; i.e. we'd like the same bits
> work on 0.20.205.x, 0.22.x, and 0.23.x hadoops.
> 
> Releasing a tarball per hadoop (multiply the number over of versions
> by two since we'll want to have secure and insecure hbase I believe)
> makes me queasy.
> 
> St.Ack


Mime
View raw message