flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8981) Add end-to-end test for running on YARN with Kerberos
Date Fri, 20 Jul 2018 09:41:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550582#comment-16550582
] 

ASF GitHub Bot commented on FLINK-8981:
---------------------------------------

Github user aljoscha commented on a diff in the pull request:

    https://github.com/apache/flink/pull/6377#discussion_r203990327
  
    --- Diff: flink-end-to-end-tests/test-scripts/docker-hadoop-secure-cluster/README.md ---
    @@ -0,0 +1,118 @@
    +# Apache Hadoop Docker image with Kerberos enabled
    +
    +This image is modified version of Knappek/docker-hadoop-secure
    + * Knappek/docker-hadoop-secure <https://github.com/Knappek/docker-hadoop-secure>
    +
    +With bits and pieces added from Lewuathe/docker-hadoop-cluster to extend it to start
a proper kerberized Hadoop cluster:
    + * Lewuathe/docker-hadoop-cluster <https://github.com/Lewuathe/docker-hadoop-cluster>
    +
    +And a lot of added stuff for making this an actual, properly configured, kerberized cluster
with proper user/permissions structure.
    +
    +Versions
    +--------
    +
    +* JDK8
    +* Hadoop 2.8.3
    +
    +Default Environment Variables
    +-----------------------------
    +
    +| Name | Value | Description |
    +| ---- | ----  | ---- |
    +| `KRB_REALM` | `EXAMPLE.COM` | The Kerberos Realm, more information [here](https://web.mit.edu/kerberos/krb5-1.12/doc/admin/conf_files/krb5_conf.html#)
|
    +| `DOMAIN_REALM` | `example.com` | The Kerberos Domain Realm, more information [here](https://web.mit.edu/kerberos/krb5-1.12/doc/admin/conf_files/krb5_conf.html#)
|
    +| `KERBEROS_ADMIN` | `admin/admin` | The KDC admin user |
    +| `KERBEROS_ADMIN_PASSWORD` | `admin` | The KDC admin password |
    +
    +You can simply define these variables in the `docker-compose.yml`.
    +
    +Run image
    +---------
    +
    +Clone the [Github project](https://github.com/aljoscha/docker-hadoop-secure-cluster)
and run
    --- End diff --
    
    fixing


> Add end-to-end test for running on YARN with Kerberos
> -----------------------------------------------------
>
>                 Key: FLINK-8981
>                 URL: https://issues.apache.org/jira/browse/FLINK-8981
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Security, Tests
>    Affects Versions: 1.5.0
>            Reporter: Till Rohrmann
>            Assignee: Aljoscha Krettek
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 1.6.0
>
>
> We should add an end-to-end test which verifies Flink's integration with Kerberos security.
In order to do this, we should start a Kerberos secured Hadoop, ZooKeeper and Kafka cluster.
Then we should start a Flink cluster with HA enabled and run a job which reads from and writes
to Kafka. We could use a simple pipe job for that purpose which has some state for checkpointing
to HDFS.
> See [security docs| https://ci.apache.org/projects/flink/flink-docs-master/ops/security-kerberos.html]
for how more information about Flink's Kerberos integration.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message