flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eron Wright <eronwri...@gmail.com>
Subject [DISCUSS] Service Authorization (redux)
Date Mon, 24 Jul 2017 01:11:13 GMT
Hello, now might be a good time to revisit an important enhancement to
Flink security, so-called service authorization.   This means the hardening
of a Flink cluster against unauthorized use with some sort of
authentication and authorization scheme.   Today, Flink relies entirely on
network isolation to protect itself from unauthorized job submission and
control, and to protect the secrets contained within a Flink cluster.
This is a problem in multi-user environments like YARN/Mesos/K8.

Last fall, an effort was made to implement service authorization but the PR
was ultimately rejected.   The idea was to add a simple secret key to all
network communication between the client, JM, and TM.   Akka itself has
such a feature which formed the basis of the solution.  There are usability
challenges with this solution, including a dependency on SSL.

Since then, the situation has evolved somewhat, and the use of SSL mutual
authentication is more viable.   Mutual auth is supported in Akka 2.4.12+
(or could be backported to Flakka).  My proposal is:

1. Upgrade Akka or backport the functionality to Flakka (see commit
5d03902c5ec3212cd28f26c9b3ef7c3b628b9451).
2. Implement SSL on any endpoint that doesn't yet support it (e.g.
queryable state).
3. Enable mutual auth in Akka and implement it on non-Akka endpoints.
4. Implement a simple authorization layer that accepts any authenticated
connection.
5. (stretch) generate and store a certificate automatically in YARN mode.
6. (stretch) Develop an alternate authentication method for the Web UI.

Are folks interested in this capability?  Thoughts on the use of SSL mutual
auth versus something else?  Thanks!

-Eron

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message