flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-6489) Rework 'start-local.bat' to 'start-local-cluster.bat'
Date Mon, 19 Feb 2018 18:16:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-6489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369390#comment-16369390

ASF GitHub Bot commented on FLINK-6489:

GitHub user StephanEwen opened a pull request:


    [FLINK-6489] [FLINK-8696] [shell scripts] Remove JobManager local mode from the Shell

    ## What is the purpose of the change
    The JobManager local mode is problematic:
      - The JobManager does not start all components properly, making some operations fail/timeout
      - It introduces code complexity in maintaining the JobManager entry point that also
spawns an embedded TaskManager
    The JobManager local mode is also unnecessary. The `start-cluster.sh` script sets up a
proper local cluster without additional configuration.
    This pull request removes the local mode from all UNIX scripts and changes the Windows
`start-local.bat` to a  `start-cluster.bat` that also starts two separate processes for JobManager
and TaskManager.
    ## Brief change log
      - Changes the `start-local.bat` to a  `start-cluster.bat` which spawns two processes
(using `start` command). The two processes show up as separate windows and can be stopped
      - Removes the `start-local-sh` and `stop-local.sh` scripts.
      - Removes the passing of the *local* parameter between the shell scripts and always
passes *cluster* as the execution mode parameter to the JobManager.
    The next step would be to remove the execution mode from the JobManagers command line
argument parsing and setup logic.
    ## Verifying this change
      - Building Flink, starting a cluster via the shell scripts (Linux, MacOS, Windows) and
checking the WebUI that the cluster is properly up. No actual job execution needed.
    ## Does this pull request potentially affect one of the following parts:
      - Dependencies (does it add or upgrade a dependency): (yes / **no)**
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes
/ **no)**
      - The serializers: (yes / **no** / don't know)
      - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing,
Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
      - The S3 file system connector: (yes / **no** / don't know)
    ## Documentation
      - Does this pull request introduce a new feature? (yes / **no)**
      - If yes, how is the feature documented? (not applicable / **docs** / JavaDocs / not
    Docs are updated to not refer to `start-local.xy` any more.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/StephanEwen/incubator-flink remove_local_mode

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5528
commit 99d564928f8299d2ad868ed5c23a9c88d8899a47
Author: Stephan Ewen <sewen@...>
Date:   2018-02-16T20:14:41Z

    [FLINK-6489] [shell scripts] Remove local mode from Windows start-local.bat
    Instead, this uses 'start' to start JobManager and TaskManager background processes.

commit f142deebd951a498f2247b2ce243c15b19a81e84
Author: Stephan Ewen <sewen@...>
Date:   2018-02-16T20:15:49Z

    [FLINK-8696] [shell scripts] Remove JobManager local mode from shell scripts

commit 11e87c4e2e879afb753354660ce3e33fb38a60d8
Author: Stephan Ewen <sewen@...>
Date:   2018-02-19T18:13:12Z

    [FLINK-6489] [FLINK-8696] [docs] Update docs to use 'start-cluster.sh' instead of 'start-local.sh'
    (and likewise for 'start-cluster.bat' vs. 'start-local.bat')


> Rework 'start-local.bat' to 'start-local-cluster.bat'
> -----------------------------------------------------
>                 Key: FLINK-6489
>                 URL: https://issues.apache.org/jira/browse/FLINK-6489
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Startup Shell Scripts
>            Reporter: Stephan Ewen
>            Assignee: Stephan Ewen
>            Priority: Major
> To get rid of the JobManager local mode, we need to make sure that the {{start-local.bat}}
script for windows also starts a separate JobManager and TaskManager process, rather than
only the JobManager (and relying on that one to spawn a TaskManager)

This message was sent by Atlassian JIRA

View raw message