flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stavros Kontopoulos (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (FLINK-5536) Config option: HA
Date Tue, 02 May 2017 20:07:04 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993627#comment-15993627
] 

Stavros Kontopoulos edited comment on FLINK-5536 at 5/2/17 8:06 PM:
--------------------------------------------------------------------

Ok can do that. I verified that these work in dc/os:
extra-args: -Dhigh-availability=zookeeper -Dhigh-availability.zookeeper.quorum=master.mesos:2181
-Dhigh-availability.zookeeper.storageDir=hdfs://hdfs/flink/recovery -Drecovery.zookeeper.path.mesos-workers=/flink

Hdfs: config-url: http://api.hdfs.marathon.l4lb.thisdcos.directory/v1/endpoints

An hdfs folder is requirement.

Of course need to verify that a job restarts ok etc...

Passing them to the service is trivial. As noon as I check this I will create a PR.


was (Author: skonto):
Ok can do that. I verified that these work in dc/os:
extra-args: -Dhigh-availability=zookeeper -Dhigh-availability.zookeeper.quorum=master.mesos:2181
-Dhigh-availability.zookeeper.storageDir=hdfs://hdfs/flink/recovery -Drecovery.zookeeper.path.mesos-workers=/flink

Hdfs: config-url: http://api.hdfs.marathon.l4lb.thisdcos.directory/v1/endpoints
Of course need to verify that a job restarts ok etc...

Passing them to the service is trivial. As noon as I check this I will create a PR.

> Config option: HA
> -----------------
>
>                 Key: FLINK-5536
>                 URL: https://issues.apache.org/jira/browse/FLINK-5536
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Mesos
>            Reporter: Eron Wright 
>            Assignee: Stavros Kontopoulos
>
> Configure Flink HA thru package options plus good defaults.   The main components are
ZK configuration and state backend configuration.
> - The ZK information can be defaulted to `master.mesos` as with other packages
> - Evaluate whether ZK can be fully configured by default, even if a state backend isn't
configured.
> - Use DCOS HDFS as the filesystem for the state backend.  Evaluate whether to assume
that DCOS HDFS is installed by default, or whether to make it explicit.
> - To use DCOS HDFS, the init script should download the core-site.xml and hdfs-site.xml
from the HDFS 'connection' endpoint.   Supply a default value for the endpoint address; see
[https://docs.mesosphere.com/service-docs/hdfs/connecting-clients/].



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message