hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Kanter <rkan...@cloudera.com>
Subject Re: Time to address the Guava version problem
Date Tue, 23 Sep 2014 20:55:08 GMT
At the same time, not being able to use Curator will require a lot of extra
code, a lot of which we probably already have from the ZKRMStateStore, but
it's not available to use in hadoop-auth.  We'd need to create our own ZK
libraries that Hadoop components can use, but (a) that's going to take a
while, and (b) it seems silly to reinvent the wheel when Curator already
does all this.

I agree that upgrading Guava will be a compatibility problem though...

On Tue, Sep 23, 2014 at 9:30 AM, Sandy Ryza <sandy.ryza@cloudera.com> wrote:

> If we've broken compatibility in branch-2, that's a bug that we need to
> fix. HADOOP-10868 has not yet made it into a release; I don't see it as a
> justification for solidifying the breakage.
>
> -1 to upgrading Guava in branch-2.
>
> On Tue, Sep 23, 2014 at 3:06 AM, Steve Loughran <stevel@hortonworks.com>
> wrote:
>
> > +1 to upgrading guava. Irrespective of downstream apps, the hadoop source
> > tree is now internally inconsistent
> >
> > On 22 September 2014 17:56, Sangjin Lee <sjlee@apache.org> wrote:
> >
> > > I agree that a more robust solution is to have better classloading
> > > isolation.
> > >
> > > Still, IMHO guava (and possibly protobuf as well) sticks out like a
> sore
> > > thumb. There are just too many issues in trying to support both guava
> 11
> > > and guava 16. Independent of what we may do with the classloading
> > > isolation, we should still consider upgrading guava.
> > >
> > > My 2 cents.
> > >
> > > On Sun, Sep 21, 2014 at 3:11 PM, Karthik Kambatla <kasha@cloudera.com>
> > > wrote:
> > >
> > > > Upgrading Guava version is tricky. While it helps in many cases, it
> can
> > > > break existing applications/deployments. I understand we do not have
> a
> > > > policy for updating dependencies, but still we should be careful with
> > > > Guava.
> > > >
> > > > I would be more inclined towards a more permanent solution to this
> > > problem
> > > > - how about prioritizing classpath isolation so applications aren't
> > > > affected by Hadoop dependency updates at all? I understand that will
> > also
> > > > break user applications, but it might be the driving feature for
> Hadoop
> > > > 3.0?
> > > >
> > > > On Fri, Sep 19, 2014 at 5:13 PM, Sangjin Lee <sjlee@apache.org>
> wrote:
> > > >
> > > > > I would also agree on upgrading guava. Yes I am aware of the
> > potential
> > > > > impact on customers who might rely on hadoop bringing in guava 11.
> > > > However,
> > > > > IMHO the balance tipped over to the other side a while ago; i.e.
I
> > > think
> > > > > there are far more people using guava 16 in their code and
> scrambling
> > > to
> > > > > make things work than the other way around.
> > > > >
> > > > > On Thu, Sep 18, 2014 at 2:40 PM, Steve Loughran <
> > > stevel@hortonworks.com>
> > > > > wrote:
> > > > >
> > > > > > I know we've been ignoring the Guava version problem, but
> > > HADOOP-10868
> > > > > > added a transitive dependency on Guava 16 by way of Curator
2.6.
> > > > > >
> > > > > > Maven currently forces the build to use Guava 11.0.2, but this
is
> > > > hiding
> > > > > at
> > > > > > compile timeall code paths from curator which may use classes
&
> > > methods
> > > > > > that aren't there.
> > > > > >
> > > > > > I need curator for my own work (2.4.1 & Guava 14.0 was what
I'd
> > been
> > > > > > using), so don't think we can go back.
> > > > > >
> > > > > > HADOOP-11102 covers the problem -but doesn't propose a specific
> > > > solution.
> > > > > > But to me the one that seems most likely to work is: update
Guava
> > > > > >
> > > > > > -steve
> > > > > >
> > > > > > --
> > > > > > CONFIDENTIALITY NOTICE
> > > > > > NOTICE: This message is intended for the use of the individual
or
> > > > entity
> > > > > to
> > > > > > which it is addressed and may contain information that is
> > > confidential,
> > > > > > privileged and exempt from disclosure under applicable law.
If
> the
> > > > reader
> > > > > > of this message is not the intended recipient, you are hereby
> > > notified
> > > > > that
> > > > > > any printing, copying, dissemination, distribution, disclosure
or
> > > > > > forwarding of this communication is strictly prohibited. If
you
> > have
> > > > > > received this communication in error, please contact the sender
> > > > > immediately
> > > > > > and delete it from your system. Thank You.
> > > > > >
> > > > >
> > > >
> > >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message