hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Enis Söztutar <enis....@gmail.com>
Subject Re: [DISCUSS] Dependency compatibility
Date Fri, 13 Mar 2015 18:56:13 GMT
>
>
> I think we can solve this generally for Hadoop 2.6.0+. There's no reason
> our HDFS usage should be exposed in the HBase client code, and I think the
> application classpath feature for YARN in that version can isolate us on
> the MR side. I am willing to do this work in time for 1.1. Realistically I
> don't know the timeline for that version yet. If it turns out the work is
> more involved or my time is more constrained then I think, I'm willing to
> accept promise weakening as a practical matter.
>

HBase-1.x series SHOULD work with Hadoop versions as old as 2.2. That is
what we promise for 1.x series. So solving the problem in Hadoop-2.6+ will
not
solve it for 1.1.

It is great if we can help Hadoop with classloading issues, do shading
there, and
do shading in HBase and reduce the dependencies etc. However, since we
cannot
do these in HBase-1.x series (we cannot shade deps in the same manner), I
do
not see a way how to get around this by doing anything other than what I
propose.

We have discussed many of the compat dimensions before we adopted them in
PMC.
For some of those (like binary compat), we decided that we cannot support
them
between minor versions in 1.x , so we decided 'false" on that dimension.
For these we
explicitly decided that when we can realistically have more guarantees, we
can start
supporting this dimension (client binary compat in minor versions) and have
2.x or
later support those.

I see the dependency compat dimension in the same vein. It is clear (at
least to me)
that we cannot support pragmatically any dep compat in 1.x series. If
you/we can
make all the necessary changes in Hadoop and HBase, we can reintroduce this
back. Until then though, I would rather not block any progress and drop the
support.
As you said the timeline is not clear, so why are we cornering ourselves
(especially
for 1.x series) for this?



>
> I'd be much more comfortable weakening our dependency promises for
> coprocessor than doing it in general. Folks running coprocessors should
> already be more risk tolerant and familiar with our internals.
>
> For upstreams that don't have the leverage on us of Hadoop, we solve this
> problem simply by not updating dependencies that we can't trust to not
> break our downstreams.
>
>
>
> > I would be disappointed to see a VOTE thread. That means we failed to
> reach
> > consensus and needed to fall back to process to resolve differences.
> >
> >
>
> That's fair. What about the wider audience issue on user@? There's no
> reason our DISCUSS threads couldn't go there as well.
>
>
>
> > Why don't we do the doc update and call it a day?
> >
>
> I've been burned by dependency changes in projects I rely on many times in
> the past, usually over changes in code sections that folks didn't think
> were likely to be used. So I'm very willing to do work now to save
> downstream users of HBase that same headache.
>
> --
> Sean
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message