flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Welly Tambunan <if05...@gmail.com>
Subject Re: Security in Flink
Date Wed, 13 Jan 2016 07:34:39 GMT
Hi Stephan,

Thanks a lot for the explanation.

Is there any timeline on when this will be released ? I guess this one will
be the important for our case if we want Flink to be deployed in
production.

Cheers

On Tue, Jan 12, 2016 at 6:19 PM, Stephan Ewen <sewen@apache.org> wrote:

> Hi Sourav!
>
> If you want to use Flink in a cluster where neither Hadoop/YARN (not soon
> Mesos) is available, then I assume you have installed Flink in a standalone
> mode on the cluster already.
>
> There is no support in Flink currently to manage user authentication. Few
> thoughts on how that may evolve
>
> 1) It should be not too hard to add authentication to the web dashboard.
> That way, if the cluster is otherwise blocked off (the master's RPC ports
> are firewalled), one would have restricted job starts.
>
> 2) We plan to add authenticated / encrypted connections soon. With that,
> the client that submits the program would need to have access to the
> keystore or key and the corresponding password to connect.
>
> Greetings,
> Stephan
>
>
>
> On Mon, Jan 11, 2016 at 3:46 PM, Sourav Mazumder <
> sourav.mazumder00@gmail.com> wrote:
>
>> Thanks Steven for your details response. Things are more clear to me now.
>>
>> A follow up Qs -
>> Looks like most of the security support depends on Hadoop ? What happens
>> if anyone wants to use Flink with Hadoop (in a cluster where Hadoop is not
>> there) ?
>>
>> Regards,
>> Sourav
>>
>> On Sun, Jan 10, 2016 at 12:41 PM, Stephan Ewen <sewen@apache.org> wrote:
>>
>>> Hi Sourav!
>>>
>>> There is user-authentication support in Flink via the Hadoop / Kerberos
>>> infrastructure. If you run Flink on YARN, it should seamlessly work that
>>> Flink acquires the Kerberos tokens of the user that submits programs, and
>>> authenticate itself at YARN, HDFS, and HBase with that.
>>>
>>> If you run Flink standalone, Flink can still authenticate at HDFS/HBase
>>> via Kerberos, with a bit of manual help by the user (running kinit on the
>>> workers).
>>>
>>> With Kafka 0.9 and Flink's upcoming connector (
>>> https://github.com/apache/flink/pull/1489), streaming programs can
>>> authenticate themselves as stream brokers via SSL (and read via encrypted
>>> connections).
>>>
>>>
>>> What we have on the roadmap for the coming months it the following:
>>>   - Encrypt in-flight data streams that are exchanged between worker
>>> nodes (TaskManagers).
>>>   - Encrypt the coordination messages between client/master/workers.
>>> Note that these refer to encryption between Flink's own components only,
>>> which would use transient keys generated just for a specific job or session
>>> (hence would not need any user involvement).
>>>
>>>
>>> Let us know if that answers your questions, and if that meets your
>>> requirements.
>>>
>>> Greetings,
>>> Stephan
>>>
>>>
>>> On Fri, Jan 8, 2016 at 3:23 PM, Sourav Mazumder <
>>> sourav.mazumder00@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Can anyone point me to ant documentation on support for Security in
>>>> Flink ?
>>>>
>>>> The type of information I'm looking for are -
>>>>
>>>> 1. How do I do user level authentication to ensure that a job is
>>>> submitted/deleted/modified by the right user ? Is it possible though the
>>>> web client ?
>>>> 2. Authentication across multiple slave nodes (where the task managers
>>>> are running) and driver program so that they can communicate with each other
>>>> 3. Support for SSL/encryption for data exchanged happening across the
>>>> slave nodes
>>>> 4. Support for pluggable authentication with existing solution like LDAP
>>>>
>>>> If not there today is there a roadmap for these security features ?
>>>>
>>>> Regards,
>>>> Sourav
>>>>
>>>
>>>
>>
>


-- 
Welly Tambunan
Triplelands

http://weltam.wordpress.com
http://www.triplelands.com <http://www.triplelands.com/blog/>

Mime
View raw message