hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alejandro Abdelnur <tuc...@gmail.com>
Subject Re: Guava
Date Mon, 10 Nov 2014 17:26:32 GMT
IMO we should:

1* have a clean and thin client API JAR (which does not drag any 3rd party
dependencies, or a well defined small set -i.e. slf4j & log4j-)
2* have a client implementation that uses a classloader to isolate client
impl 3rd party deps from app dependencies.

#2 can be done using a stock URLClassLoader (i would just subclass it to
forbid packages in the API JAR and exposed 3rd parties to be loaded from
the app JAR)

#1 is the tricky thing as our current API modules don't have a clean
API/impl separation.

thx
PS: If folks are interested in pursing this, I can put together a prototype
of how  #2 would work (I don't think it will be more than 200 lines of code)


On Mon, Nov 10, 2014 at 5:18 AM, Steve Loughran <stevel@hortonworks.com>
wrote:

> Yes, Guava is a constant pain; there's lots of open JIRAs related to it, as
> its the one we can't seamlessly upgrade. Not unless we do our own fork and
> reinsert the missing classes.
>
> The most common uses in the code are
>
> @VisibleForTesting (easily replicated)
> and the Precondition.check() operations
>
> The latter is also easily swapped out, and we could even add the check they
> forgot:
> Preconditions.checkArgNotNull(argname, arg)
>
>
> These are easy; its the more complex data structures that matter more.
>
> I think for Hadoop 2.7 & java 7 we need to look at this problem and do
> something. Even if we continue to ship Guava 11 so that the HBase team
> don't send any (more) death threats, we can/should rework Hadoop to build
> and run against Guava 16+ too. That's needed to fix some of the recent java
> 7/8+ changes.
>
> -Everything in v11 dropped from v16 MUST  to be implemented with our own
> versions.
> -anything tagged as deprecated in 11+ SHOULD be replaced by newer stuff,
> wherever possible.
>
> I think for 2.7+ we should add some new profiles to the POM, for Java 8 and
> 9 alongside the new baseline java 7. For those later versions we could
> perhaps mandate Guava 16.
>
>
>
> On 10 November 2014 00:42, Arun C Murthy <acm@hortonworks.com> wrote:
>
> > … has been a constant pain w.r.t compatibility etc.
> >
> > Should we consider adopting a policy to not use guava in
> Common/HDFS/YARN?
> >
> > MR doesn't matter too much since it's application-side issue, it does
> hurt
> > end-users though since they still might want a newer guava-version, but
> at
> > least they can modify MR.
> >
> > Thoughts?
> >
> > thanks,
> > Arun
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message