flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: Security in Flink
Date Mon, 18 Jan 2016 10:59:03 GMT
Hi Welly,

There is no fixed timeline yet but we plan to make progress in terms
of authentication and encryption after the 1.0.0 release.

Cheers,
Max

On Wed, Jan 13, 2016 at 8:34 AM, Welly Tambunan <if05041@gmail.com> wrote:
> Hi Stephan,
>
> Thanks a lot for the explanation.
>
> Is there any timeline on when this will be released ? I guess this one will
> be the important for our case if we want Flink to be deployed in production.
>
> Cheers
>
> On Tue, Jan 12, 2016 at 6:19 PM, Stephan Ewen <sewen@apache.org> wrote:
>>
>> Hi Sourav!
>>
>> If you want to use Flink in a cluster where neither Hadoop/YARN (not soon
>> Mesos) is available, then I assume you have installed Flink in a standalone
>> mode on the cluster already.
>>
>> There is no support in Flink currently to manage user authentication. Few
>> thoughts on how that may evolve
>>
>> 1) It should be not too hard to add authentication to the web dashboard.
>> That way, if the cluster is otherwise blocked off (the master's RPC ports
>> are firewalled), one would have restricted job starts.
>>
>> 2) We plan to add authenticated / encrypted connections soon. With that,
>> the client that submits the program would need to have access to the
>> keystore or key and the corresponding password to connect.
>>
>> Greetings,
>> Stephan
>>
>>
>>
>> On Mon, Jan 11, 2016 at 3:46 PM, Sourav Mazumder
>> <sourav.mazumder00@gmail.com> wrote:
>>>
>>> Thanks Steven for your details response. Things are more clear to me now.
>>>
>>> A follow up Qs -
>>> Looks like most of the security support depends on Hadoop ? What happens
>>> if anyone wants to use Flink with Hadoop (in a cluster where Hadoop is not
>>> there) ?
>>>
>>> Regards,
>>> Sourav
>>>
>>> On Sun, Jan 10, 2016 at 12:41 PM, Stephan Ewen <sewen@apache.org> wrote:
>>>>
>>>> Hi Sourav!
>>>>
>>>> There is user-authentication support in Flink via the Hadoop / Kerberos
>>>> infrastructure. If you run Flink on YARN, it should seamlessly work that
>>>> Flink acquires the Kerberos tokens of the user that submits programs, and
>>>> authenticate itself at YARN, HDFS, and HBase with that.
>>>>
>>>> If you run Flink standalone, Flink can still authenticate at HDFS/HBase
>>>> via Kerberos, with a bit of manual help by the user (running kinit on the
>>>> workers).
>>>>
>>>> With Kafka 0.9 and Flink's upcoming connector
>>>> (https://github.com/apache/flink/pull/1489), streaming programs can
>>>> authenticate themselves as stream brokers via SSL (and read via encrypted
>>>> connections).
>>>>
>>>>
>>>> What we have on the roadmap for the coming months it the following:
>>>>   - Encrypt in-flight data streams that are exchanged between worker
>>>> nodes (TaskManagers).
>>>>   - Encrypt the coordination messages between client/master/workers.
>>>> Note that these refer to encryption between Flink's own components only,
>>>> which would use transient keys generated just for a specific job or session
>>>> (hence would not need any user involvement).
>>>>
>>>>
>>>> Let us know if that answers your questions, and if that meets your
>>>> requirements.
>>>>
>>>> Greetings,
>>>> Stephan
>>>>
>>>>
>>>> On Fri, Jan 8, 2016 at 3:23 PM, Sourav Mazumder
>>>> <sourav.mazumder00@gmail.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Can anyone point me to ant documentation on support for Security in
>>>>> Flink ?
>>>>>
>>>>> The type of information I'm looking for are -
>>>>>
>>>>> 1. How do I do user level authentication to ensure that a job is
>>>>> submitted/deleted/modified by the right user ? Is it possible though
the web
>>>>> client ?
>>>>> 2. Authentication across multiple slave nodes (where the task managers
>>>>> are running) and driver program so that they can communicate with each
other
>>>>> 3. Support for SSL/encryption for data exchanged happening across the
>>>>> slave nodes
>>>>> 4. Support for pluggable authentication with existing solution like
>>>>> LDAP
>>>>>
>>>>> If not there today is there a roadmap for these security features ?
>>>>>
>>>>> Regards,
>>>>> Sourav
>>>>
>>>>
>>>
>>
>
>
>
> --
> Welly Tambunan
> Triplelands
>
> http://weltam.wordpress.com
> http://www.triplelands.com

Mime
View raw message