accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <busbey+li...@cloudera.com>
Subject Re: documentation on dealing with legacy Hadoop versions
Date Fri, 03 Jan 2014 16:45:01 GMT
On Fri, Jan 3, 2014 at 10:26 AM, Josh Elser <josh.elser@gmail.com> wrote:

>
> Point of reference: HBase-0.96.0 will pull *all* dependencies into
> $HBASE_HOME/lib. Now, while I don't think I want to re-package all of the
> Hadoop jars and its dependencies, I don't think it's unreasonable to
> repackage ones that may be duplicated by the hadoop distribution that we
> specifically need (thinking specifically of the commons-*).
>
>
Definitely we should not include the Hadoop jars, since it led HBase to
have to include deployment steps like "remove the packaged Hadoop jars"[1].

I think repackaging any transitive dependencies of Hadoop is a good idea,
since it makes us more robust to classpath issues. My one concern would be
that we make sure we pick versions that are not going to cause
compatibility issues based on wether our or Hadoop's version ends up on the
classpath first[2]. In the simplest case, that would mean having our
version match Hadoop's.

-Sean

[1]: http://hbase.apache.org/book.html#replace.hadoop
[2]: this already happened with Guava in ACCUMULO-2127

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message