flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2288) Setup ZooKeeper for distributed coordination
Date Tue, 07 Jul 2015 14:42:04 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14616791#comment-14616791

ASF GitHub Bot commented on FLINK-2288:

Github user StephanEwen commented on a diff in the pull request:

    --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/JobManager.scala
    @@ -993,9 +1000,34 @@ object JobManager {
           configuration.setString(ConfigConstants.FLINK_BASE_DIR_PATH_KEY, configDir + "/..")
    -    val hostname = configuration.getString(ConfigConstants.JOB_MANAGER_IPC_ADDRESS_KEY,
    -    val port = configuration.getInteger(ConfigConstants.JOB_MANAGER_IPC_PORT_KEY,
    -      ConfigConstants.DEFAULT_JOB_MANAGER_IPC_PORT)
    +    // HA mode
    +    val (hostname, port) = if (ZooKeeperUtil.isJobManagerHighAvailabilityEnabled(configuration))
    +      // TODO @removeme @tillrohrmann This is the place where the host and random port
for JM is
    +      // chosen.  For the FlinkMiniCluster you have to choose it on your own.
    +      LOG.info("HA mode.")
    +      if (config.getHost == null) {
    --- End diff --
    Across Java/Scala, this is a tricky discussion.
    It holds if you are within the Scala Ecosystem. There, you define a method with parenthesis,
or without. The definition of the method clearly indicates the purity.
    When cross-referencing Java methods, that does not work any more. The implementer cannot
state the purity of the method. Letting the caller call the method in a way that he hopes
the method behaves is not very clean.
    In this case it works, but we have a lot of our Java methods start with `get` and have
side effects. They get called without parenthesis frequently, because IDEs suggest that methods
starting with `get` should have no parenthesis (which assumes all Java code ever was written
with Scala semantics assumed).
    To be consistent in Scala means calling empty-parenthesis methods with parenthesis. I
find it strange to not apply the same principle to Java methods (which are all defined as
empty-parenthesis methods).

> Setup ZooKeeper for distributed coordination
> --------------------------------------------
>                 Key: FLINK-2288
>                 URL: https://issues.apache.org/jira/browse/FLINK-2288
>             Project: Flink
>          Issue Type: Sub-task
>          Components: JobManager, TaskManager
>            Reporter: Ufuk Celebi
>            Assignee: Ufuk Celebi
>             Fix For: 0.10
> Having standby JM instances for job manager high availabilty requires distributed coordination
between JM, TM, and clients. For this, we will use ZooKeeper (ZK).
> Pros:
> - Proven solution (other projects use it for this as well)
> - Apache TLP with large community, docs, and library with required "recipies" like leader
election (see below)
> Related Wiki: https://cwiki.apache.org/confluence/display/FLINK/JobManager+High+Availability

This message was sent by Atlassian JIRA

View raw message