hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Bautin <bautin.mailing.li...@gmail.com>
Subject Re: loading Hadoop native libraries in HBase unit tests
Date Sun, 12 Feb 2012 20:36:24 GMT
One difficulty with simply importing libhadoop.so into HBase codebase is
that the dynamic library is probably a bit different for different versions
of Hadoop. Is there a way to pull the .so file from Maven for the
configured Hadoop version? Ideally this should be done in a
platform-independent way, too, but making it work on Linux would be the
first step.


On Fri, Feb 10, 2012 at 8:52 PM, Stack <stack@duboce.net> wrote:

> On Fri, Feb 10, 2012 at 7:22 PM, Mikhail Bautin
> <bautin.mailing.lists@gmail.com> wrote:
> > Hello,
> >
> > We currently do not enable native Hadoop libraries in unit tests (at
> least
> > when running on Hadoop QA), but we do use them in production.  Should we
> > try to close this discrepancy between tests and production? Some possible
> > approaches would be:
> >
> >   - Enable native libraries by default (e.g. import libhadoop.so into a
> >   special location in the HBase repository and add
> >   -Djava.library.path=/that/path to unit test JVM options).
> >   - In addition to the above, modify some unit tests (e.g. those using
> >   Gzip compression) to run both with and without native libraries enabled
> >
> > Please let me know if this sounds reasonable.
> >
> We used to be careful w/ native libs to ensure they were present and
> hbase would use them if available but we let that drop a couple of
> versions ago and haven't picked it up since.  Thanks for reviving them
> Mikhail.  I can help making sure works on apache build box.
> St.Ack

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message