apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Munagala Ramanath <...@datatorrent.com>
Subject Re: Shading Guava In Apex
Date Wed, 25 Nov 2015 20:08:51 GMT
Chetan,

Could you elaborate on what you mean by "class leak" and why the issue
is complicated ?

Thanks.

Ram

On Wed, Nov 25, 2015 at 12:02 PM, Chetan Narsude (cnarsude) <
cnarsude@cisco.com> wrote:

> Apex is prone to class leak so even application developers have to use the
> guava version that underlying Hadoop uses. Platform developers have the
> least say other than fixing the class leak and free application developers
> to use whatever they want. The solution is said to be super complicated
> and there are no takers motivated enough to implement it.
>
> ‹
> Chetan
>
>
>
> On 11/25/15, 5:01 AM, "Ted Dunning" <ted.dunning@gmail.com> wrote:
>
> >Flink definitely hit the problem.
> >
> >The issue is that people use different versions of guava in their
> >applications and they aren't compatible.
> >
> >You are likely to want to use a more advanced version than the Hadoop
> >systems use, also.
> >
> >
> >
> >On Wed, Nov 25, 2015 at 3:13 PM, Timothy Farkas <tim@datatorrent.com>
> >wrote:
> >
> >> I haven't seen an issue :)
> >>
> >> On Tue, Nov 24, 2015 at 10:57 PM, Thomas Weise <thomas@datatorrent.com>
> >> wrote:
> >>
> >> > That certainly may be the case. But have we seen problems sticking to
> >>the
> >> > Hadoop 2.2 compatible subset? AFAIK Apex runs on all Hadoop versions
> >> > starting from 2.2 across all major distros.
> >> >
> >> > On Tue, Nov 24, 2015 at 10:36 PM, Timothy Farkas <tim@datatorrent.com
> >
> >> > wrote:
> >> >
> >> > > I was under the impression that the guava version varies for
> >>different
> >> > > hadoop distributions and versions, also guava is not strictly
> >>backwards
> >> > > compatible. Is that not the case?
> >> > >
> >> > > On Tue, Nov 24, 2015 at 10:30 PM, Thomas Weise
> >><thomas@datatorrent.com
> >> >
> >> > > wrote:
> >> > >
> >> > > > We do it for ASM. But why do we need to use a different guava
> >> version?
> >> > > >
> >> > > >
> >> > > > On Tue, Nov 24, 2015 at 10:27 PM, Timothy Farkas <
> >> tim@datatorrent.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Flink shades the guava dependency so that Flink can use
a
> >>version
> >> of
> >> > > > guava
> >> > > > > that is different from the one provided by the hadoop
> >>distribution.
> >> > Is
> >> > > > this
> >> > > > > something that we should be doing as well?
> >> > > > >
> >> > > > >
> >> > >
> >> https://github.com/apache/flink/blob/master/flink-shaded-hadoop/pom.xml
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message