hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roman Shaposhnik <...@apache.org>
Subject Re: HBase .92 maven artifacts compiled against different releases of Hadoop
Date Wed, 16 Nov 2011 03:25:22 GMT
Sorry for reviving the dead thread, but I have, like, a real problem
to solve ;-)

On Fri, Nov 11, 2011 at 2:09 PM, Alejandro Abdelnur <tucu@cloudera.com> wrote:
> Eric,
>
> Do you mean that the HBASE published POM won't have a Hadoop artifact
> as a dependency?

Ok, so lets see what the problem here really is (and also verify that
my understanding is correct):
  1. we have a binary Maven artifact for HBase that is pretty well
  insulated from the underlying Hadoop by a layer of shims and thus
  can work with *any* version of Hadoop selected at run time. Basically,
  my reading of Todd's reply is that there's no *reason* to have multiple
  versions of hbase-0.92.jar

  2. we have a test binary artifact that is NOT insulated. Case in point:
  org.apache.hadoop.hbase.mapreduce.NMapInputFormat is failing because
  org.apache.hadoop.mapreduce.JobContext is either class or interface
  depending on which version of Hadoop you compile it against.

Now given the above it sounds like a proper way to fix this is to provide
a level of shims for tests and make them run against any version of Hadoop.

Agreed?

That leaves us with a single problem -- when both artifacts become hadoop
agnostic we still have to put *some* version of Hadoop into our POM file
as a dependency AND we have to either:
  1. make that dependency optional
  2. make all of the downstream exclude it

Personally, I'd go with #1 since it makes things much more explicit. But what
do others think?

Thanks,
Roman.

Mime
View raw message