hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@cloudera.com>
Subject Re: [DISCUSS] Dependency compatibility
Date Wed, 11 Mar 2015 23:17:51 GMT
On Wed, Mar 11, 2015 at 5:25 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:

> >
> > > Furthermore, "hadoop jar" is how you're supposed to launch YARN apps.
> If
> > we
> > > say that doing things via the hbase command is acceptable, we're
> opening
> > > ourselves up to an expansion of what the hbase command has to do. (i.e.
> > > perhaps it should detect if the passed class is a YARN driver and then
> > use
> > > the hadoop jar command? or should it always pass through to the hadoop
> > jar
> > > command?)
> > >
> > Traditionally, and in our documentation, HBase owned MR classes
> (CopyTable,
> > Import, etc) are run
> > with the hbase script, not the hadoop script. It is a regression in that
> > sense still. Yes, there is a
> > workaround, but why we bother where we can fix this easily.
> Can we side-step some of the issues here by fixing the hbase script for
> launching jobs?
Only if it's only for launching jobs, I think. Otherwise we break non-YARN
things by bringing an incompatible version of jackson into the classpath
(and a version different from what our pom says will be there).

Maybe we could add "hbase yarn-driver" or some such that does the modified
classpath? That would break operational compatibility, but our compat guide
says we can do that on minor version.

That would also let us hide all the mapreduce classpath setting and
consolidate our examples on a single way of launching against hbase. We
could just list the current way (classpath + hadoop jar) as an note of what
to do if you need to work with some other hadoop-integrated library as well.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message