Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
MIME-Version: 1.0
In-Reply-To: <CAKiyyaGRwvXEy6wS7hpqD0S-dw+c+EjA_T7Kn0pEFi7VgfXG5g@mail.gmail.com>
References: <CAGKOEz25smFBoWhvZ-YtNjRcZXmfD4Yv15-wtFJcJA9wUW97yg@mail.gmail.com>
 <B7DB26F7-6A6B-440B-9C82-D70A0AE6FF11@apache.org> <CAC27z=O1=kD+9bo8==Rn7Qya1jg-ueEtkWv7LU8dP+G-6O9kpw@mail.gmail.com>
 <CAGKOEz00R_Zq46br-D=CPPby2zoFk6icEUJe1g3iNuOuxuSAFA@mail.gmail.com>
 <CAC27z=MEUvVxOmS1f+gY_c57Bg3vdig_-GnuZeVSo3bDB-raTQ@mail.gmail.com> <CAKiyyaGRwvXEy6wS7hpqD0S-dw+c+EjA_T7Kn0pEFi7VgfXG5g@mail.gmail.com>
From: Eron Wright <eronwright@gmail.com>
Date: Thu, 3 Aug 2017 11:11:55 -0700
Message-ID: <CAGKOEz38BK7ZSKxPNFyisSEBJyh=V4ecLnpk_C0PnyvQpMP2aQ@mail.gmail.com>
Subject: Re: [DISCUSS] Service Authorization (redux)
To: dev@flink.apache.org
Content-Type: multipart/alternative; boundary="001a113d5246db53790555dd504d"
archived-at: Thu, 03 Aug 2017 18:12:12 -0000

--001a113d5246db53790555dd504d
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Till, with (c) are you suggesting that we'd use Akka 2.3 for Scala 2.10 and
Akka 2.4+ for Scala 2.11+?   Sounds reasonable but I don't know how
feasible it is.   I will say I'm optimistic because a) Akka 2.4 is said to
be binary compatible, and b) the Flakka fork appears to be subsumed by 2.4.

Let us then take (c) as the tentative plan.

I agree the community should discuss dropping Scala 2.10 but I don't want
to drive that conversation.

Thanks


On Thu, Aug 3, 2017 at 6:24 AM, Ufuk Celebi <uce@apache.org> wrote:

> I haven't followed this discussion in detail nor am I familiar with
> the service authorization topic or Flakka, but a) sounds like a lot of
> maintenance work to me.
>
> If possible I would go with c) and maybe start a discussion about
> dropping Scala 2.10 support to check whether that is a viable option
> or not.
>
> =E2=80=93 Ufuk
>
>
> On Thu, Aug 3, 2017 at 1:59 PM, Till Rohrmann <trohrmann@apache.org>
> wrote:
> > Alternatively there would also be an option
> >
> > c) only support mutual auth for Akka 2.4+ if the backport is unrealisti=
c
> to
> > do
> >
> > But this of course would break security for Scala 2.10. On the other ha=
nd
> > people are already using Flink without this feature.
> >
> > Cheers,
> > Till
> >
> > On Wed, Aug 2, 2017 at 7:21 PM, Eron Wright <eronwright@gmail.com>
> wrote:
> >
> >> Thanks Till and Aljoscha for the feedback.
> >>
> >> Seems there are two ways to proceed here, if we accept mutual SSL as t=
he
> >> basis.
> >>
> >> a) Backport mutual-auth support from Akka 2.4 to Flakka.
> >> b) Drop support for Scala 2.10 (FLINK-?), move to Akka 2.4 (FLINK-3662=
).
> >>
> >> Let's assume (a) for now.
> >>
> >>
> >>
> >> On Tue, Aug 1, 2017 at 3:05 PM, Till Rohrmann <trohrmann@apache.org>
> >> wrote:
> >>
> >> > Dropping Java 7 alone is not enough to move to Akka 2.4+. For that w=
e
> >> need
> >> > at least Scala 2.11.
> >> >
> >> > Cheers,
> >> > Till
> >> >
> >> > On Tue, Aug 1, 2017 at 4:22 PM, Aljoscha Krettek <aljoscha@apache.or=
g
> >
> >> > wrote:
> >> >
> >> > > Hi Eron,
> >> > >
> >> > > I think after Dropping support for Java 7 we will move to Akka
> 2.4+, so
> >> > we
> >> > > should be good there. I think quite some users should find a (more=
)
> >> > secure
> >> > > Flink interesting.
> >> > >
> >> > > Best,
> >> > > Aljoscha
> >> > > > On 24. Jul 2017, at 03:11, Eron Wright <eronwright@gmail.com>
> wrote:
> >> > > >
> >> > > > Hello, now might be a good time to revisit an important
> enhancement
> >> to
> >> > > > Flink security, so-called service authorization.   This means th=
e
> >> > > hardening
> >> > > > of a Flink cluster against unauthorized use with some sort of
> >> > > > authentication and authorization scheme.   Today, Flink relies
> >> entirely
> >> > > on
> >> > > > network isolation to protect itself from unauthorized job
> submission
> >> > and
> >> > > > control, and to protect the secrets contained within a Flink
> cluster.
> >> > > > This is a problem in multi-user environments like YARN/Mesos/K8.
> >> > > >
> >> > > > Last fall, an effort was made to implement service authorization
> but
> >> > the
> >> > > PR
> >> > > > was ultimately rejected.   The idea was to add a simple secret
> key to
> >> > all
> >> > > > network communication between the client, JM, and TM.   Akka
> itself
> >> has
> >> > > > such a feature which formed the basis of the solution.  There ar=
e
> >> > > usability
> >> > > > challenges with this solution, including a dependency on SSL.
> >> > > >
> >> > > > Since then, the situation has evolved somewhat, and the use of S=
SL
> >> > mutual
> >> > > > authentication is more viable.   Mutual auth is supported in Akk=
a
> >> > 2.4.12+
> >> > > > (or could be backported to Flakka).  My proposal is:
> >> > > >
> >> > > > 1. Upgrade Akka or backport the functionality to Flakka (see
> commit
> >> > > > 5d03902c5ec3212cd28f26c9b3ef7c3b628b9451).
> >> > > > 2. Implement SSL on any endpoint that doesn't yet support it (e.=
g.
> >> > > > queryable state).
> >> > > > 3. Enable mutual auth in Akka and implement it on non-Akka
> endpoints.
> >> > > > 4. Implement a simple authorization layer that accepts any
> >> > authenticated
> >> > > > connection.
> >> > > > 5. (stretch) generate and store a certificate automatically in
> YARN
> >> > mode.
> >> > > > 6. (stretch) Develop an alternate authentication method for the
> Web
> >> UI.
> >> > > >
> >> > > > Are folks interested in this capability?  Thoughts on the use of
> SSL
> >> > > mutual
> >> > > > auth versus something else?  Thanks!
> >> > > >
> >> > > > -Eron
> >> > >
> >> > >
> >> >
> >>
>

--001a113d5246db53790555dd504d--