flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8981) Add end-to-end test for running on YARN with Kerberos
Date Mon, 23 Jul 2018 09:09:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552522#comment-16552522
] 

ASF GitHub Bot commented on FLINK-8981:
---------------------------------------

Github user aljoscha commented on a diff in the pull request:

    https://github.com/apache/flink/pull/6377#discussion_r204330950
  
    --- Diff: flink-end-to-end-tests/test-scripts/docker-hadoop-secure-cluster/docker-compose.yml
---
    @@ -0,0 +1,87 @@
    +################################################################################
    +# Licensed to the Apache Software Foundation (ASF) under one
    +# or more contributor license agreements.  See the NOTICE file
    +# distributed with this work for additional information
    +# regarding copyright ownership.  The ASF licenses this file
    +# to you under the Apache License, Version 2.0 (the
    +# "License"); you may not use this file except in compliance
    +# with the License.  You may obtain a copy of the License at
    +#
    +#     http://www.apache.org/licenses/LICENSE-2.0
    +#
    +# Unless required by applicable law or agreed to in writing, software
    +# distributed under the License is distributed on an "AS IS" BASIS,
    +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    +# See the License for the specific language governing permissions and
    +# limitations under the License.
    +################################################################################
    +version: '3.5'
    +
    +networks:
    +  docker-hadoop-cluster-network:
    +    driver: bridge
    +    name: docker-hadoop-cluster-network
    +
    +services:
    +  kdc:
    +    container_name: "kdc"
    +    hostname: kdc.kerberos.com
    +    image: sequenceiq/kerberos
    +    networks:
    +      - docker-hadoop-cluster-network
    +    environment:
    +      REALM: EXAMPLE.COM
    +      DOMAIN_REALM: kdc.kerberos.com
    +
    +  master:
    +    image: ${DOCKER_HADOOP_IMAGE_NAME:-flink/docker-hadoop-secure-cluster:latest}
    +    command: master
    +    depends_on:
    +      - kdc
    +    ports:
    +      - "50070:50070"
    --- End diff --
    
    This was because the setup was meant to be accessible for more generic use and access
from outside. I'm removing it.


> Add end-to-end test for running on YARN with Kerberos
> -----------------------------------------------------
>
>                 Key: FLINK-8981
>                 URL: https://issues.apache.org/jira/browse/FLINK-8981
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Security, Tests
>    Affects Versions: 1.5.0
>            Reporter: Till Rohrmann
>            Assignee: Aljoscha Krettek
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 1.6.0
>
>
> We should add an end-to-end test which verifies Flink's integration with Kerberos security.
In order to do this, we should start a Kerberos secured Hadoop, ZooKeeper and Kafka cluster.
Then we should start a Flink cluster with HA enabled and run a job which reads from and writes
to Kafka. We could use a simple pipe job for that purpose which has some state for checkpointing
to HDFS.
> See [security docs| https://ci.apache.org/projects/flink/flink-docs-master/ops/security-kerberos.html]
for how more information about Flink's Kerberos integration.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message