hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jerry He <jerry...@gmail.com>
Subject Re: [DISCUSSION] Upgrading core dependencies
Date Thu, 09 Feb 2017 06:10:55 GMT
I was more on the thinking to avoid/reduce pulling in hbase dependencies
into hbase-spark, and that maybe even hbase-spark can depend on shaded
client and server -- it will be easier and more feasible if the shaded
client becomes the default as you mentioned.

Your idea that hbase-spark itself becomes a shaded artifact sounds better
if I understand you correctly. The spark/scala dependencies are 'provided'


On Wed, Feb 8, 2017 at 6:14 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:

> On Wed, Feb 8, 2017 at 10:24 AM Jerry He <jerryjch@gmail.com> wrote:
> > Yeah.  Talking about the dependency, the hbase-spark module already has
> > dependency on hbase-server (coming from the spark bulk load producing
> > hfiles).
> > This is not very good. We have to be careful not entangling it more.
> > Also, there is already problem running the hbase-spark due to dependency
> > conflict, and one has to be careful about the order of the classpath to
> > make it work.
> We own the hbase-spark module, do we not? In that case, we control our own
> destiny. An explicit goal of this effort would be to make use of that
> module agnostic to classpath load order. As per my earlier reply,
> hbase-spark could itself be an artifact shaded over hbase-server and all of
> its dependencies. That way the user doesn't need to think about it at all.
> It further seems to me that the maven-shade-plugin could gain a new analyze
> goal, similar to the that of the dependency-plugin, which would audit the
> classes packaged in a jar vs the contract defined in configuration. This
> could further be used to fail the build of there's any warnings reported.
> I've found myself wanting this very functionality as I consume ES and
> Phoenix, shaded in downstream projects.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message