flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefano Baghino <stefano.bagh...@radicalbit.io>
Subject Re: Behavior of lib directory shipping on YARN
Date Tue, 29 Mar 2016 13:09:31 GMT
Yup, I shall open an issue for both this one and my other thread (re:
Kerberos).
Thanks for the pointer on this issue.

On Tue, Mar 29, 2016 at 12:44 PM, Maximilian Michels <mxm@apache.org> wrote:

> Hi Stefano,
>
> Thanks for pointing out this bug. Your analysis is correct. The per-job
> cluster does not ship the /lib directory by default. Would you like to open
> an issue/PR? We should let the ship_path default to the /lib directory.
>
> The mechanism with the environment variables is the same. They used to be
> defined in a different location (FlinkYarnClient) and have been moved to a
> separate class (YarnConfigKeys).
>
> Cheers,
> Max
>
>
>
> On Wed, Mar 23, 2016 at 10:06 AM, Stefano Baghino <
> stefano.baghino@radicalbit.io> wrote:
>
> > Thanks for pointing out Max's work (awesome PR, btw). It actually seem to
> > have introduced an environment variable regarding ship directories, it
> > would be good to have his feedback on this.
> >
> > On Tue, Mar 22, 2016 at 10:24 PM, Ufuk Celebi <uce@apache.org> wrote:
> >
> > > On Tue, Mar 22, 2016 at 8:42 PM, Stefano Baghino
> > > <stefano.baghino@radicalbit.io> wrote:
> > > > My feeling is that running a job on YARN should
> > > > end up in having more or less the same effect, regardless of the way
> > the
> > > > job is run.
> > >
> > > +1
> > >
> > > I think that the current behaviour is buggy. The resource management
> > > is currently undergoing a massive refactoring
> > > (https://github.com/apache/flink/pull/1741). Maybe it's already fixed
> > > there (if the issue is independent of the scripts).
> > >
> > > Would be great to have a fix for this. If #1751 does not fix it, feel
> > > free to open an issue and PR. :-)
> > >
> > > – Ufuk
> > >
> >
> >
> >
> > --
> > BR,
> > Stefano Baghino
> >
> > Software Engineer @ Radicalbit
> >
>



-- 
BR,
Stefano Baghino

Software Engineer @ Radicalbit

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message