flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: Behavior of lib directory shipping on YARN
Date Tue, 29 Mar 2016 10:44:59 GMT
Hi Stefano,

Thanks for pointing out this bug. Your analysis is correct. The per-job
cluster does not ship the /lib directory by default. Would you like to open
an issue/PR? We should let the ship_path default to the /lib directory.

The mechanism with the environment variables is the same. They used to be
defined in a different location (FlinkYarnClient) and have been moved to a
separate class (YarnConfigKeys).

Cheers,
Max



On Wed, Mar 23, 2016 at 10:06 AM, Stefano Baghino <
stefano.baghino@radicalbit.io> wrote:

> Thanks for pointing out Max's work (awesome PR, btw). It actually seem to
> have introduced an environment variable regarding ship directories, it
> would be good to have his feedback on this.
>
> On Tue, Mar 22, 2016 at 10:24 PM, Ufuk Celebi <uce@apache.org> wrote:
>
> > On Tue, Mar 22, 2016 at 8:42 PM, Stefano Baghino
> > <stefano.baghino@radicalbit.io> wrote:
> > > My feeling is that running a job on YARN should
> > > end up in having more or less the same effect, regardless of the way
> the
> > > job is run.
> >
> > +1
> >
> > I think that the current behaviour is buggy. The resource management
> > is currently undergoing a massive refactoring
> > (https://github.com/apache/flink/pull/1741). Maybe it's already fixed
> > there (if the issue is independent of the scripts).
> >
> > Would be great to have a fix for this. If #1751 does not fix it, feel
> > free to open an issue and PR. :-)
> >
> > – Ufuk
> >
>
>
>
> --
> BR,
> Stefano Baghino
>
> Software Engineer @ Radicalbit
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message