flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mar_zieh <m.marzieh.ghas...@gmail.com>
Subject Running Flink on Marathon
Date Sun, 28 Apr 2019 07:56:39 GMT
I have three physical nodes with docker installed on them. I have one docker
container with Mesos, Marathon, Hadoop and Flink. I configured Master node
and Slave nodes for Mesos,Zookeeper and Marathon. I do these works step by
step. First, In Master node, I enter to docker container with this command:

docker run -v /home/user/.ssh:/root/.ssh --privileged -p 5050:5050 -p
5051:5051 -p 5052:5052 -p 2181:2181 -p 8082:8081 -p 6123:6123 -p 8080:8080
-p 50090:50090 -p 50070:50070 -p 9000:9000 -p 2888:2888 -p 3888:3888 -p
4041:4040 -p 7077:7077 -p 52222:22 -e WEAVE_CIDR= -e
MESOS_RESOURCES=ports*:[11000-11999] -ti hadoop_marathon_mesos_flink_2

Then run Mesos and Zookeeper :

/home/zookeeper-3.4.14/bin/zkServer.sh restart
/home/mesos-1.7.2/build/bin/mesos-master.sh --ip=
--hostname= --roles=marathon,flink --quorum=1
--work_dir=/var/run/mesos --log_dir=/var/log/mesos 

After that run Marathon in the same container:

/home/marathon-1.7.189-48bfd6000/bin/marathon --master --zk
zk:// --hostname --webui_url
--logging_level debug

And finally, I run hadoop:


Marathon,Mesos and Hadoop are run without any problems. The most important
part of my work is running Flink in Marathon. I configured Flink in docker
container like this:

 env.java.home: /opt/java
  high-availability: zookeeper
  high-availability.storageDir: hdfs:///flink/ha/
  recovery.zookeeper.path.mesos-workers: /mesos-workers

In Marathon UI, I create Application and put this JSON file on it, but it is

  "id": "flink",
  "cmd": "/home/flink-1.7.0/bin/mesos-appmaster.sh 
  "cpus": 1.0,
  "mem": 1024

Flink application is failed in Mesos UI.
This is the error:

I0428 06:01:39.586699 6155 exec.cpp:162] Version: 1.7.2

I0428 06:01:39.596458 6154 exec.cpp:236] Executor registered on agent

I0428 06:01:39.598870 6157 executor.cpp:188] Received SUBSCRIBED event

I0428 06:01:39.599761 6157 executor.cpp:192] Subscribed executor on

I0428 06:01:39.599963 6157 executor.cpp:188] Received LAUNCH event

I0428 06:01:39.601236 6157 executor.cpp:697] Starting task

I0428 06:01:39.613719 6157 executor.cpp:712] Forked command at 6163

I0428 06:01:39.787395 6157 executor.cpp:1013] Command exited with status 1
(pid: 6163)

I0428 06:01:40.791885 6162 process.cpp:927] Stopped the socket accept loop

The strange thing is that in STDout, I see this text; even though I set
JAVA_HOME in /etc/environment and flink-conf.yam.

Please specify JAVA_HOME. Either in Flink config ./conf/flink-conf.yaml or
as system-wide JAVA_HOME.

Would you please tell me what I should do for that problem?

Many Thanks.

Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

View raw message