flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: Proposal: YARN session per-job Kerberos authentication
Date Wed, 23 Mar 2016 18:06:07 GMT
Hi Stefano,

Sounds great. Please go ahead! Note that Flink already provides the
proposed feature for per-job Yarn clusters. However, it is a valuable
addition to realize this feature for the Yarn session.

The only blocker that I can think of is probably this PR which changes
a lot of the Yarn classes: https://github.com/apache/flink/pull/1741
There are also changes planned for the client side to decouple the
Yarn support from the job submission process and make it easier to
integrate other frameworks (like Mesos). I don't think that will block
your contribution since a lot of the logic is probably going to be
contained in separate classes which can be integrated even when code
changes. Let's just stay in sync.

If you like, you could start off by opening an issue and submitting a
short design document.

Cheers,
Max

On Wed, Mar 23, 2016 at 3:55 PM, Stefano Baghino
<stefano.baghino@radicalbit.io> wrote:
> Hello everybody,
>
> some of us at Radicalbit spent the last few weeks experimenting to improve
> the understanding of the compatibility of Flink with secure cluster
> environments and with Kerberos in particular.
>
> We’ve found a possible area of improvement and would like to work on it as
> part of our effort to contribute to Flink in the open: after a few tests
> and a short exchange on the user mailing list
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kerberos-on-YARN-delegation-or-proxying-tp5315p5318.html>
> we’ve come to realize that currently a long-running session on YARN acts on
> behalf of the user that originally ran the session, not the one who’s
> submitting the job.
>
> We think it would be a nice improvement to be able to have a single Flink
> session that keeps running with several users submitting their jobs with
> their own credentials.
>
> We’d like to develop, test and document this improvement.
>
> Do you think this is feasible? Are there any blockers we should be aware of
> before undertaking this task? Would this be something of interest for the
> community? Are there any other ongoing efforts that aim toward this?
>
> We’d love to have the feedback of the community on this, thank you in
> advance to anyone who’s willing to share their insight and opinion with us.
>
> --
> BR,
> Stefano Baghino
>
> Software Engineer @ Radicalbit

Mime
View raw message