Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 8A5BC200C24 for ; Thu, 23 Feb 2017 21:34:14 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 88CDB160B64; Thu, 23 Feb 2017 20:34:14 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id F05D1160B3E for ; Thu, 23 Feb 2017 21:34:10 +0100 (CET) Received: (qmail 47504 invoked by uid 500); 23 Feb 2017 20:34:09 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 47495 invoked by uid 99); 23 Feb 2017 20:34:09 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Feb 2017 20:34:09 +0000 Received: from mail-it0-f44.google.com (mail-it0-f44.google.com [209.85.214.44]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 6F0C91A00A2 for ; Thu, 23 Feb 2017 20:34:09 +0000 (UTC) Received: by mail-it0-f44.google.com with SMTP id d9so6584210itc.0 for ; Thu, 23 Feb 2017 12:34:09 -0800 (PST) X-Gm-Message-State: AMke39khcL5nm8t90X7HNUTFk/qap90EazQgiQIUEUdYr9wGESU9qn22EZ0l8qkM5QDYdJfu3NaEL24tZQF6dw== X-Received: by 10.107.202.135 with SMTP id a129mr30981373iog.129.1487882048493; Thu, 23 Feb 2017 12:34:08 -0800 (PST) MIME-Version: 1.0 Received: by 10.64.135.1 with HTTP; Thu, 23 Feb 2017 12:33:47 -0800 (PST) In-Reply-To: References: <5950fc888cf44a2ab1367da1aa11611e@SH1MAIL02.corp.vipshop.com> From: Robert Metzger Date: Thu, 23 Feb 2017 21:33:47 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Can't run flink on yarn on version 1.2.0 To: "user@flink.apache.org" Content-Type: multipart/alternative; boundary=94eb2c0bd0f8f7e6940549388815 archived-at: Thu, 23 Feb 2017 20:34:14 -0000 --94eb2c0bd0f8f7e6940549388815 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi, were both JDKs from the same vendor? (say OpenJDK?) both installed "vanilla" from the package manager? Java is usually pretty good with backwards compatibility. I think this issue is caused by some other effects we are overseeing here. On Thu, Feb 23, 2017 at 10:43 AM, Bruno Aranda wrote: > Hi, > > Good you found a solution, but are you sure it is the JDK version? > > We are running Flink 1.2.0 on Yarn on an AWS EMR Cluster with no issues, > using JDK 8 (1.8.0_121). > > Cheers, > > Bruno > > On Thu, 23 Feb 2017 at 09:26 Howard,Li(vip.com) > wrote: > >> Hi All: >> >> We finally find out the problem. >> >> The Flink on Yarn only works on JDK7, but not JDK8. If you use >> JDK8, you may meet the problem discussed before. >> >> For more information: OS: CentOS 6.6. JDK7 version: 1.7.0u75 >> JDK8 version: 1.8.0u111. >> >> >> >> This problem may have some relationship with akka. >> >> >> >> *=E5=8F=91=E4=BB=B6=E4=BA=BA:* Till Rohrmann [mailto:trohrmann@apache.or= g] >> *=E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4:* 2017=E5=B9=B42=E6=9C=8817=E6=97= =A5 18:33 >> >> *=E6=94=B6=E4=BB=B6=E4=BA=BA:* user@flink.apache.org >> *=E4=B8=BB=E9=A2=98:* Re: Can't run flink on yarn on version 1.2.0 >> >> >> >> Hi Howard, >> >> >> >> could you check whether the JobManager's actor system was bound to " >> vip-rc-vsubu.vclound.com:55926"? You should see that in the job manager >> logs. Furthermore, have you checked that you Yarn cluster nodes are >> actually reachable from the node where you start the Flink application? = If >> so, the logs of the cli client as well as the JobManager logs (both on >> debug level) would be tremendously helpful. >> >> >> >> Cheers, >> >> Till >> >> >> >> On Fri, Feb 17, 2017 at 10:41 AM, Howard,Li(vip.com) < >> howard.li@vipshop.com> wrote: >> >> Sorry for the confusion I made. I do copy the wrong log, but we do meet >> this problem on 1.2.0. >> >> for version 1.1.4 however, we meet this in one cluster but not in >> another. We are still trying to figure out what happened. >> >> >> >> The following is the log for 1.2.0 version: >> >> >> >> 2017-02-17 15:51:37,775 INFO org.apache.flink.yarn.cli. >> FlinkYarnSessionCli - No path for the flink jar passed. >> Using the location of class org.apache.flink.yarn.YarnClusterDescriptor >> to locate the jar >> >> 2017-02-17 15:51:37,775 INFO org.apache.flink.yarn.cli. >> FlinkYarnSessionCli - No path for the flink jar passed. >> Using the location of class org.apache.flink.yarn.YarnClusterDescriptor >> to locate the jar >> >> 2017-02-17 15:51:37,803 INFO org.apache.flink.yarn.YarnClusterDescripto= r >> - Using values: >> >> 2017-02-17 15:51:37,804 INFO org.apache.flink.yarn. >> YarnClusterDescriptor - TaskManager count =3D 2 >> >> 2017-02-17 15:51:37,804 INFO org.apache.flink.yarn. >> YarnClusterDescriptor - JobManager memory =3D 1024 >> >> 2017-02-17 15:51:37,804 INFO org.apache.flink.yarn. >> YarnClusterDescriptor - TaskManager memory =3D 1024 >> >> 2017-02-17 15:51:37,827 INFO org.apache.hadoop.yarn.client. >> RMProxy - Connecting to ResourceManager at / >> 0.0.0.0:8032 >> >> 2017-02-17 15:51:38,672 WARN org.apache.flink.yarn. >> YarnClusterDescriptor - The configuration directory >> ('/home/software/flink-1.2.0/conf') contains both LOG4J and Logback >> configuration files. Please delete or rename one of them. >> >> 2017-02-17 15:51:38,685 INFO org.apache.flink.yarn.Utils >> - Copying from >> file:/home/software/flink-1.2.0/examples/batch/WordCount.jar to hdfs:// >> 10.199.202.161:9000/user/root/.flink/application_ >> 1487247313588_0016/WordCount.jar >> >> 2017-02-17 15:51:38,992 INFO org.apache.flink.yarn.Utils >> - Copying from >> file:/home/software/flink-1.2.0/conf/log4j.properties to hdfs:// >> 10.199.202.161:9000/user/root/.flink/application_ >> 1487247313588_0016/log4j.properties >> >> 2017-02-17 15:51:39,058 INFO org.apache.flink.yarn.Utils >> - Copying from >> file:/home/software/flink-1.2.0/conf/logback.xml to hdfs:// >> 10.199.202.161:9000/user/root/.flink/application_ >> 1487247313588_0016/logback.xml >> >> 2017-02-17 15:51:39,085 INFO org.apache.flink.yarn.Utils >> - Copying from >> file:/home/software/flink-1.2.0/lib to hdfs://10.199.202.161:9000/ >> user/root/.flink/application_1487247313588_0016/lib >> >> 2017-02-17 15:51:39,695 INFO org.apache.flink.yarn.Utils >> - Copying from >> file:/home/software/flink-1.2.0/lib/flink-dist_2.11-1.2.0.jar to hdfs:// >> 10.199.202.161:9000/user/root/.flink/application_ >> 1487247313588_0016/flink-dist_2.11-1.2.0.jar >> >> 2017-02-17 15:51:40,493 INFO org.apache.flink.yarn.Utils >> - Copying from >> /home/software/flink-1.2.0/conf/flink-conf.yaml to hdfs:// >> 10.199.202.161:9000/user/root/.flink/application_ >> 1487247313588_0016/flink-conf.yaml >> >> 2017-02-17 15:51:40,547 INFO org.apache.flink.yarn. >> YarnClusterDescriptor - Submitting application master >> application_1487247313588_0016 >> >> 2017-02-17 15:51:40,585 INFO org.apache.hadoop.yarn.client. >> api.impl.YarnClientImpl - Submitted application >> application_1487247313588_0016 >> >> 2017-02-17 15:51:40,585 INFO org.apache.flink.yarn. >> YarnClusterDescriptor - Waiting for the cluster to be >> allocated >> >> 2017-02-17 15:51:40,587 INFO org.apache.flink.yarn. >> YarnClusterDescriptor - Deploying cluster, current >> state ACCEPTED >> >> 2017-02-17 15:51:45,879 INFO org.apache.flink.yarn. >> YarnClusterDescriptor - YARN application has been >> deployed successfully. >> >> Cluster started: Yarn cluster with application id >> application_1487247313588_0016 >> >> Using address vip-rc-vsubu.vclound.com:55926 to connect to JobManager. >> >> JobManager web interface address http://vip-rc-ucsww.vclound. >> com:8088/proxy/application_1487247313588_0016/ >> >> Using the parallelism provided by the remote cluster (8). To use another >> parallelism, set it at the ./bin/flink client. >> >> Starting execution of program >> >> 2017-02-17 15:51:46,704 INFO org.apache.flink.yarn. >> YarnClusterClient - Starting program in >> interactive mode >> >> Executing WordCount example with default input data set. >> >> Use --input to specify file input. >> >> Printing result to stdout. Use --output to specify output path. >> >> 2017-02-17 15:51:47,029 INFO org.apache.flink.yarn. >> YarnClusterClient - Waiting until all TaskManagers >> have connected >> >> Waiting until all TaskManagers have connected >> >> 2017-02-17 15:51:47,029 INFO org.apache.flink.yarn. >> YarnClusterClient - Starting client actor system. >> >> >> >> ------------------------------------------------------------ >> >> The program finished with the following exception: >> >> >> >> org.apache.flink.client.program.ProgramInvocationException: The main >> method caused an error. >> >> at org.apache.flink.client.program.PackagedProgram. >> callMainMethod(PackagedProgram.java:545) >> >> at org.apache.flink.client.program.PackagedProgram. >> invokeInteractiveModeForExecution(PackagedProgram.java:419) >> >> at org.apache.flink.client.program.ClusterClient.run( >> ClusterClient.java:339) >> >> at org.apache.flink.client.CliFrontend.executeProgram( >> CliFrontend.java:831) >> >> at org.apache.flink.client.CliFrontend.run(CliFrontend.java:256= ) >> >> at org.apache.flink.client.CliFrontend.parseParameters( >> CliFrontend.java:1073) >> >> at org.apache.flink.client.CliFrontend$2.call( >> CliFrontend.java:1120) >> >> at org.apache.flink.client.CliFrontend$2.call( >> CliFrontend.java:1117) >> >> at org.apache.flink.runtime.security. >> HadoopSecurityContext$1.run(HadoopSecurityContext.java:43) >> >> at java.security.AccessController.doPrivileged(Native Method) >> >> at javax.security.auth.Subject.doAs(Subject.java:422) >> >> at org.apache.hadoop.security.UserGroupInformation.doAs( >> UserGroupInformation.java:1657) >> >> at org.apache.flink.runtime.security.HadoopSecurityContext. >> runSecured(HadoopSecurityContext.java:40) >> >> at org.apache.flink.client.CliFrontend.main(CliFrontend. >> java:1116) >> >> Caused by: java.lang.RuntimeException: Unable to get ClusterClient statu= s >> from Application Client >> >> at org.apache.flink.yarn.YarnClusterClient.getClusterStatus( >> YarnClusterClient.java:248) >> >> at org.apache.flink.yarn.YarnClusterClient. >> waitForClusterToBeReady(YarnClusterClient.java:520) >> >> at org.apache.flink.client.program.ClusterClient.run( >> ClusterClient.java:412) >> >> at org.apache.flink.yarn.YarnClusterClient.submitJob( >> YarnClusterClient.java:210) >> >> at org.apache.flink.client.program.ClusterClient.run( >> ClusterClient.java:400) >> >> at org.apache.flink.client.program.ClusterClient.run( >> ClusterClient.java:387) >> >> at org.apache.flink.client.program.ContextEnvironment. >> execute(ContextEnvironment.java:62) >> >> at org.apache.flink.api.java.ExecutionEnvironment.execute( >> ExecutionEnvironment.java:926) >> >> at org.apache.flink.api.java.DataSet.collect(DataSet.java:410) >> >> at org.apache.flink.api.java.DataSet.print(DataSet.java:1605) >> >> at org.apache.flink.examples.java.wordcount.WordCount.main( >> WordCount.java:92) >> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> >> at sun.reflect.NativeMethodAccessorImpl.invoke( >> NativeMethodAccessorImpl.java:62) >> >> at sun.reflect.DelegatingMethodAccessorImpl.invoke( >> DelegatingMethodAccessorImpl.java:43) >> >> at java.lang.reflect.Method.invoke(Method.java:498) >> >> at org.apache.flink.client.program.PackagedProgram. >> callMainMethod(PackagedProgram.java:528) >> >> ... 13 more >> >> Caused by: org.apache.flink.runtime.leaderretrieval.LeaderRetrievalExcep= tion: >> Could not retrieve the leader gateway >> >> at org.apache.flink.runtime.util.LeaderRetrievalUtils. >> retrieveLeaderGateway(LeaderRetrievalUtils.java:141) >> >> at org.apache.flink.client.program.ClusterClient. >> getJobManagerGateway(ClusterClient.java:691) >> >> at org.apache.flink.yarn.YarnClusterClient.getClusterStatus( >> YarnClusterClient.java:242) >> >> ... 28 more >> >> Caused by: java.util.concurrent.TimeoutException: Futures timed out >> after [10000 milliseconds] >> >> at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise. >> scala:219) >> >> at scala.concurrent.impl.Promise$DefaultPromise.result(Promise. >> scala:223) >> >> at scala.concurrent.Await$$anonfun$result$1.apply( >> package.scala:190) >> >> at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn( >> BlockContext.scala:53) >> >> at scala.concurrent.Await$.result(package.scala:190) >> >> at scala.concurrent.Await.result(package.scala) >> >> at org.apache.flink.runtime.util.LeaderRetrievalUtils. >> retrieveLeaderGateway(LeaderRetrievalUtils.java:139) >> >> ... 30 more >> >> 2017-02-17 15:52:21,145 INFO org.apache.flink.yarn. >> YarnClusterClient - Sending shutdown request to >> the Application Master >> >> 2017-02-17 15:52:21,145 INFO org.apache.flink.yarn. >> YarnClusterClient - Start application client. >> >> 2017-02-17 15:52:21,151 WARN org.apache.flink.yarn. >> YarnClusterClient - YARN reported application >> state FAILED >> >> 2017-02-17 15:52:21,152 WARN org.apache.flink.yarn. >> YarnClusterClient - Diagnostics: Application >> application_1487247313588_0016 failed 1 times due to AM Container for >> appattempt_1487247313588_0016_000001 exited with exitCode: -103 >> >> For more detailed output, check application tracking page: >> http://vip-rc-ucsww.vclound.com:8088/cluster/app/ >> application_1487247313588_0016Then, click on links to logs of each >> attempt. >> >> Diagnostics: Container [pid=3D18590,containerID=3D >> container_1487247313588_0016_01_000001] is running beyond virtual memory >> limits. Current usage: 266.1 MB of 1 GB physical memory used; 2.2 GB of = 2.1 >> GB virtual memory used. Killing container. >> >> Dump of the process-tree for container_1487247313588_0016_01_000001 : >> >> |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) >> SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE >> >> |- 18598 18590 18590 18590 (java) 894 48 2294116352 >> <(229)%20411-6352> 67782 /home/software/jdk1.8.0_111/bin/java -Xmx424M >> -Dlog.file=3D/home/software/hadoop-2.7.3/logs/userlogs/ >> application_1487247313588_0016/container_1487247313588_0016_01_000001/jo= bmanager.log >> -Dlogback.configurationFile=3Dfile:logback.xml -Dlog4j.configuration=3Df= ile:log4j.properties >> org.apache.flink.yarn.YarnApplicationMasterRunner >> >> |- 18590 18588 18590 18590 (bash) 0 0 108605440 334 /bin/bash -= c >> /home/software/jdk1.8.0_111/bin/java -Xmx424M -Dlog.file=3D/home/softwa= re/ >> hadoop-2.7.3/logs/userlogs/application_1487247313588_ >> 0016/container_1487247313588_0016_01_000001/jobmanager.log >> -Dlogback.configurationFile=3Dfile:logback.xml -Dlog4j.configuration=3Df= ile:log4j.properties >> org.apache.flink.yarn.YarnApplicationMasterRunner >> 1>/home/software/hadoop-2.7.3/logs/userlogs/application_ >> 1487247313588_0016/container_1487247313588_0016_01_000001/jobmanager.out >> 2>/home/software/hadoop-2.7.3/logs/userlogs/application_ >> 1487247313588_0016/container_1487247313588_0016_01_000001/jobmanager.err >> >> >> >> Container killed on request. Exit code is 143 >> >> Container exited with a non-zero exit code 143 >> >> Failing this attempt. Failing the application. >> >> 2017-02-17 15:52:21,160 INFO org.apache.flink.yarn. >> ApplicationClient - Notification about new leader >> address akka.tcp://flink@vip-rc-vsubu.vclound.com:55926/user/jobmanager >> with session ID null. >> >> 2017-02-17 15:52:21,163 INFO org.apache.flink.yarn. >> ApplicationClient - Sending StopCluster request to >> JobManager. >> >> 2017-02-17 15:52:21,164 INFO org.apache.flink.yarn. >> ApplicationClient - Received address of new leader >> akka.tcp://flink@vip-rc-vsubu.vclound.com:55926/user/jobmanager with >> session ID null. >> >> 2017-02-17 15:52:21,165 INFO org.apache.flink.yarn. >> ApplicationClient - Disconnect from JobManager >> null. >> >> 2017-02-17 15:52:21,168 INFO org.apache.flink.yarn.ApplicationClient >> - Trying to register at JobManager akka.tcp:// >> flink@vip-rc-vsubu.vclound.com:55926/user/jobmanager. >> >> 2017-02-17 15:52:21,684 INFO org.apache.flink.yarn. >> ApplicationClient - Trying to register at >> JobManager akka.tcp://flink@vip-rc-vsubu.vclound.com:55926/user/ >> jobmanager. >> >> 2017-02-17 15:52:22,174 INFO org.apache.flink.yarn. >> ApplicationClient - Sending StopCluster request to >> JobManager. >> >> 2017-02-17 15:52:22,704 INFO org.apache.flink.yarn. >> ApplicationClient - Trying to register at >> JobManager akka.tcp://flink@vip-rc-vsubu.vclound.com:55926/user/ >> jobmanager. >> >> 2017-02-17 15:52:23,194 INFO org.apache.flink.yarn. >> ApplicationClient - Sending StopCluster request to >> JobManager. >> >> 2017-02-17 15:52:24,214 INFO org.apache.flink.yarn. >> ApplicationClient - Sending StopCluster request to >> JobManager. >> >> 2017-02-17 15:52:24,725 INFO org.apache.flink.yarn. >> ApplicationClient - Trying to register at >> JobManager akka.tcp://flink@vip-rc-vsubu.vclound.com:55926/user/ >> jobmanager. >> >> 2017-02-17 15:52:25,234 INFO org.apache.flink.yarn. >> ApplicationClient - Sending StopCluster request to >> JobManager. >> >> 2017-02-17 15:52:26,254 INFO org.apache.flink.yarn. >> ApplicationClient - Sending StopCluster request to >> JobManager. >> >> 2017-02-17 15:52:27,274 INFO org.apache.flink.yarn. >> ApplicationClient - Sending StopCluster request to >> JobManager. >> >> 2017-02-17 15:52:28,294 INFO org.apache.flink.yarn. >> ApplicationClient - Sending StopCluster request to >> JobManager. >> >> 2017-02-17 15:52:28,744 INFO org.apache.flink.yarn. >> ApplicationClient - Trying to register at >> JobManager akka.tcp://flink@vip-rc-vsubu.vclound.com:55926/user/ >> jobmanager. >> >> 2017-02-17 15:52:29,314 INFO org.apache.flink.yarn. >> ApplicationClient - Sending StopCluster request to >> JobManager. >> >> 2017-02-17 15:52:30,334 INFO org.apache.flink.yarn. >> ApplicationClient - Sending StopCluster request to >> JobManager. >> >> 2017-02-17 15:52:31,155 WARN org.apache.flink.yarn. >> YarnClusterClient - Error while stopping YARN >> cluster. >> >> java.util.concurrent.TimeoutException: Futures timed out after [10000 >> milliseconds] >> >> at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise. >> scala:219) >> >> at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise. >> scala:153) >> >> at scala.concurrent.Await$$anonfun$ready$1.apply(package. >> scala:169) >> >> at scala.concurrent.Await$$anonfun$ready$1.apply(package. >> scala:169) >> >> at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn( >> BlockContext.scala:53) >> >> at scala.concurrent.Await$.ready(package.scala:169) >> >> at scala.concurrent.Await.ready(package.scala) >> >> at org.apache.flink.yarn.YarnClusterClient.shutdownCluster( >> YarnClusterClient.java:372) >> >> at org.apache.flink.yarn.YarnClusterClient.finalizeCluster( >> YarnClusterClient.java:342) >> >> at org.apache.flink.client.program.ClusterClient. >> shutdown(ClusterClient.java:208) >> >> at org.apache.flink.client.CliFrontend.run(CliFrontend.java:263= ) >> >> at org.apache.flink.client.CliFrontend.parseParameters( >> CliFrontend.java:1073) >> >> at org.apache.flink.client.CliFrontend$2.call( >> CliFrontend.java:1120) >> >> at org.apache.flink.client.CliFrontend$2.call( >> CliFrontend.java:1117) >> >> at org.apache.flink.runtime.security. >> HadoopSecurityContext$1.run(HadoopSecurityContext.java:43) >> >> at java.security.AccessController.doPrivileged(Native Method) >> >> at javax.security.auth.Subject.doAs(Subject.java:422) >> >> at org.apache.hadoop.security.UserGroupInformation.doAs( >> UserGroupInformation.java:1657) >> >> at org.apache.flink.runtime.security.HadoopSecurityContext. >> runSecured(HadoopSecurityContext.java:40) >> >> at org.apache.flink.client.CliFrontend.main(CliFrontend. >> java:1116) >> >> 2017-02-17 15:52:31,156 INFO org.apache.flink.yarn. >> YarnClusterClient - Deleting files in hdfs:// >> 10.199.202.161:9000/user/root/.flink/application_1487247313588_0016 >> >> 2017-02-17 15:52:31,354 INFO org.apache.flink.yarn. >> ApplicationClient - Sending StopCluster request to >> JobManager. >> >> 2017-02-17 15:52:32,163 INFO org.apache.flink.yarn. >> YarnClusterClient - Application >> application_1487247313588_0016 finished with state FAILED and final stat= e >> FAILED at 1487317906227 >> >> 2017-02-17 15:52:32,163 WARN org.apache.flink.yarn. >> YarnClusterClient - Application failed. >> Diagnostics Application application_1487247313588_0016 failed 1 times du= e >> to AM Container for appattempt_1487247313588_0016_000001 exited with >> exitCode: -103 >> >> For more detailed output, check application tracking page: >> http://vip-rc-ucsww.vclound.com:8088/cluster/app/ >> application_1487247313588_0016Then, click on links to logs of each >> attempt. >> >> Diagnostics: Container [pid=3D18590,containerID=3D >> container_1487247313588_0016_01_000001] is running beyond virtual memory >> limits. Current usage: 266.1 MB of 1 GB physical memory used; 2.2 GB of = 2.1 >> GB virtual memory used. Killing container. >> >> Dump of the process-tree for container_1487247313588_0016_01_000001 : >> >> |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) >> SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE >> >> |- 18598 18590 18590 18590 (java) 894 48 2294116352 >> <(229)%20411-6352> 67782 /home/software/jdk1.8.0_111/bin/java -Xmx424M >> -Dlog.file=3D/home/software/hadoop-2.7.3/logs/userlogs/ >> application_1487247313588_0016/container_1487247313588_0016_01_000001/jo= bmanager.log >> -Dlogback.configurationFile=3Dfile:logback.xml -Dlog4j.configuration=3Df= ile:log4j.properties >> org.apache.flink.yarn.YarnApplicationMasterRunner >> >> |- 18590 18588 18590 18590 (bash) 0 0 108605440 334 /bin/bash -= c >> /home/software/jdk1.8.0_111/bin/java -Xmx424M -Dlog.file=3D/home/softwa= re/ >> hadoop-2.7.3/logs/userlogs/application_1487247313588_ >> 0016/container_1487247313588_0016_01_000001/jobmanager.log >> -Dlogback.configurationFile=3Dfile:logback.xml -Dlog4j.configuration=3Df= ile:log4j.properties >> org.apache.flink.yarn.YarnApplicationMasterRunner >> 1>/home/software/hadoop-2.7.3/logs/userlogs/application_ >> 1487247313588_0016/container_1487247313588_0016_01_000001/jobmanager.out >> 2>/home/software/hadoop-2.7.3/logs/userlogs/application_ >> 1487247313588_0016/container_1487247313588_0016_01_000001/jobmanager.err >> >> >> >> Container killed on request. Exit code is 143 >> >> Container exited with a non-zero exit code 143 >> >> Failing this attempt. Failing the application. >> >> 2017-02-17 15:52:32,164 WARN org.apache.flink.yarn. >> YarnClusterClient - If log aggregation is >> activated in the Hadoop cluster, we recommend to retrieve the full >> application log using this command: >> >> yarn logs -applicationId application_1487247313588_0016 >> >> (It sometimes takes a few seconds until the logs are aggregated) >> >> 2017-02-17 15:52:32,164 INFO org.apache.flink.yarn. >> YarnClusterClient - YARN Client is shutting down >> >> 2017-02-17 15:52:32,267 INFO org.apache.flink.yarn. >> ApplicationClient - Stopped Application client. >> >> 2017-02-17 15:52:32,267 INFO org.apache.flink.yarn. >> ApplicationClient - Disconnect from JobManager >> null. >> >> >> >> >> >> *=E5=8F=91=E4=BB=B6=E4=BA=BA:* Bruno Aranda [mailto:brunoaranda@gmail.co= m] >> *=E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4:* 2017=E5=B9=B42=E6=9C=8817=E6=97= =A5 17:02 >> *=E6=94=B6=E4=BB=B6=E4=BA=BA:* user@flink.apache.org >> *=E4=B8=BB=E9=A2=98:* Re: Can't run flink on yarn on version 1.2.0 >> >> >> >> Hi Howard, >> >> >> >> We run Flink 1.2 in Yarn without issues. Sorry I don't have any specific >> solution, but are you sure you don't have some sort of Flink mix? In you= r >> logs I can see: >> >> >> >> *The configuration directory ('/home/software/flink-1.1.4/conf') contain= s >> both LOG4J and Logback configuration files. Please delete or rename one = of >> them.* >> >> >> >> Where it mentions 1.1.4 in the folder for the conf dir instead of 1.2. >> >> >> >> Cheers, >> >> >> >> Bruno >> >> >> >> On Fri, 17 Feb 2017 at 08:50 Howard,Li(vip.com) >> wrote: >> >> Hi, >> >> I=E2=80=99m trying to run flink on yarn by using command: bin/f= link run >> -m yarn-cluster -yn 2 -ys 4 ./examples/batch/WordCount.jar >> >> But I got the following error: >> >> >> >> 2017-02-17 15:52:40,746 INFO org.apache.flink.yarn.cli. >> FlinkYarnSessionCli - No path for the flink jar passed. >> Using the location of class org.apache.flink.yarn.YarnClusterDescriptor >> to locate the jar >> >> 2017-02-17 15:52:40,746 INFO org.apache.flink.yarn.cli. >> FlinkYarnSessionCli - No path for the flink jar passed. >> Using the location of class org.apache.flink.yarn.YarnClusterDescriptor >> to locate the jar >> >> 2017-02-17 15:52:40,775 INFO org.apache.flink.yarn. >> YarnClusterDescriptor - Using values: >> >> 2017-02-17 15:52:40,775 INFO org.apache.flink.yarn. >> YarnClusterDescriptor - TaskManager count =3D = 2 >> >> 2017-02-17 15:52:40,775 INFO org.apache.flink.yarn. >> YarnClusterDescriptor - JobManager memory =3D >> 1024 >> >> 2017-02-17 15:52:40,775 INFO org.apache.flink.yarn. >> YarnClusterDescriptor - TaskManager memory =3D >> 1024 >> >> 2017-02-17 15:52:40,796 INFO org.apache.hadoop.yarn.client. >> RMProxy - Connecting to ResourceManager at / >> 0.0.0.0:8032 >> >> 2017-02-17 15:52:41,680 WARN org.apache.flink.yarn. >> YarnClusterDescriptor - The configuration directory >> ('/home/software/flink-1.1.4/conf') contains both LOG4J and Logback >> configuration files. Please delete or rename one of them. >> >> 2017-02-17 15:52:41,702 INFO org.apache.flink.yarn.Utils >> - Copying from >> file:/home/software/flink-1.1.4/conf/logback.xml to hdfs:// >> 10.199.202.161:9000/user/root/.flink/application_ >> 1487247313588_0017/logback.xml >> >> 2017-02-17 15:52:42,025 INFO org.apache.flink.yarn.Utils >> - Copying from >> file:/home/software/flink-1.1.4/lib to hdfs://10.199.202.161:9000/ >> user/root/.flink/application_1487247313588_0017/lib >> >> 2017-02-17 15:52:42,695 INFO org.apache.flink.yarn.Utils >> - Copying from >> file:/home/software/flink-1.1.4/conf/log4j.properties to hdfs:// >> 10.199.202.161:9000/user/root/.flink/application_ >> 1487247313588_0017/log4j.properties >> >> 2017-02-17 15:52:42,722 INFO org.apache.flink.yarn.Utils >> - Copying from >> file:/home/software/flink-1.1.4/lib/flink-dist_2.10-1.1.4.jar to hdfs:// >> 10.199.202.161:9000/user/root/.flink/application_ >> 1487247313588_0017/flink-dist_2.10-1.1.4.jar >> >> 2017-02-17 15:52:43,346 INFO org.apache.flink.yarn.Utils >> - Copying from >> /home/software/flink-1.1.4/conf/flink-conf.yaml to hdfs:// >> 10.199.202.161:9000/user/root/.flink/application_ >> 1487247313588_0017/flink-conf.yaml >> >> 2017-02-17 15:52:43,386 INFO org.apache.flink.yarn. >> YarnClusterDescriptor - Submitting application master >> application_1487247313588_0017 >> >> 2017-02-17 15:52:43,425 INFO org.apache.hadoop.yarn.client. >> api.impl.YarnClientImpl - Submitted application >> application_1487247313588_0017 >> >> 2017-02-17 15:52:43,425 INFO org.apache.flink.yarn. >> YarnClusterDescriptor - Waiting for the cluster to be >> allocated >> >> 2017-02-17 15:52:43,427 INFO org.apache.flink.yarn. >> YarnClusterDescriptor - Deploying cluster, current >> state ACCEPTED >> >> 2017-02-17 15:52:48,471 INFO org.apache.flink.yarn. >> YarnClusterDescriptor - YARN application has been >> deployed successfully. >> >> Cluster started: Yarn cluster with application id >> application_1487247313588_0017 >> >> Using address 10.199.202.162:43809 to connect to JobManager. >> >> JobManager web interface address http://vip-rc-ucsww.vclound. >> com:8088/proxy/application_1487247313588_0017/ >> >> Using the parallelism provided by the remote cluster (8). To use another >> parallelism, set it at the ./bin/flink client. >> >> Starting execution of program >> >> 2017-02-17 15:52:49,278 INFO org.apache.flink.yarn. >> YarnClusterClient - Starting program in >> interactive mode >> >> Executing WordCount example with default input data set. >> >> Use --input to specify file input. >> >> Printing result to stdout. Use --output to specify output path. >> >> 2017-02-17 15:52:49,609 INFO org.apache.flink.yarn. >> YarnClusterClient - Waiting until all TaskManagers >> have connected >> >> Waiting until all TaskManagers have connected >> >> 2017-02-17 15:52:49,610 INFO org.apache.flink.yarn.YarnClusterClient >> - Starting client actor system. >> >> >> >> ------------------------------------------------------------ >> >> The program finished with the following exception: >> >> >> >> org.apache.flink.client.program.ProgramInvocationException: The main >> method caused an error. >> >> at org.apache.flink.client.program.PackagedProgram.callMainMethod( >> PackagedProgram.java:525) >> >> at org.apache.flink.client.program.PackagedProgram. >> invokeInteractiveModeForExecution(PackagedProgram.java:404) >> >> at org.apache.flink.client.program.ClusterClient.run( >> ClusterClient.java:321) >> >> at org.apache.flink.client.CliFrontend.executeProgram( >> CliFrontend.java:777) >> >> at org.apache.flink.client.CliFrontend.run(CliFrontend.java:253) >> >> at org.apache.flink.client.CliFrontend.parseParameters( >> CliFrontend.java:1005) >> >> at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1048) >> >> Caused by: java.lang.RuntimeException: Unable to get ClusterClient statu= s >> from Application Client >> >> at org.apache.flink.yarn.YarnClusterClient.getClusterStatus( >> YarnClusterClient.java:242) >> >> at org.apache.flink.yarn.YarnClusterClient.waitForClusterToBeReady( >> YarnClusterClient.java:514) >> >> at org.apache.flink.client.program.ClusterClient.run( >> ClusterClient.java:395) >> >> at org.apache.flink.yarn.YarnClusterClient.submitJob( >> YarnClusterClient.java:204) >> >> at org.apache.flink.client.program.ClusterClient.run( >> ClusterClient.java:383) >> >> at org.apache.flink.client.program.ClusterClient.run( >> ClusterClient.java:370) >> >> at org.apache.flink.client.program.ContextEnvironment. >> execute(ContextEnvironment.java:62) >> >> at org.apache.flink.api.java.ExecutionEnvironment.execute( >> ExecutionEnvironment.java:896) >> >> at org.apache.flink.api.java.DataSet.collect(DataSet.java:410) >> >> at org.apache.flink.api.java.DataSet.print(DataSet.java:1605) >> >> at org.apache.flink.examples.java.wordcount.WordCount.main( >> WordCount.java:92) >> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Meth >> >> --94eb2c0bd0f8f7e6940549388815 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi,

were both JDKs from the same vendor= ? (say OpenJDK?) both installed "vanilla" from the package manage= r?
Java is usually pretty good with backwards compatibility.
I think this issue is caused by some other effects we are overseeing = here.

= On Thu, Feb 23, 2017 at 10:43 AM, Bruno Aranda <brunoaranda@gmail.com<= /a>> wrote:
Hi= ,

Good you found a solution, but are you sure it is the = JDK version?

We are running Flink 1.2.0 on Yarn on= an AWS EMR Cluster with no issues, using JDK 8 (1.8.0_121).

=
Cheers,

Bruno

Hi All:

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 We finally find out the problem.

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 The Flink on Yarn only works on JDK= 7, but not JDK8. If you use JDK8, you may meet the problem discussed before= .

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 For more information: OS: CentOS 6.= 6. JDK7 version: 1.7.0u75 JDK8 version: 1.8.0u111.

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 This problem may have some relation= ship with akka.

=C2=A0

=E5=8F=91=E4=BB=B6=E4=BA=BA: = Till Rohrmann [mailto:trohrmann@apache.org]
=E5=8F=91=E9=80=81=E6= =97=B6=E9=97=B4: 2017=E5=B9=B42=E6=9C=8817=E6=97=A5 18:33

=E6=94=B6=E4=BB=B6=E4=BA=BA: user@fl= ink.apache.org
=E4=B8=BB=E9=A2=98: Re: Can't ru= n flink on yarn on version 1.2.0

=C2=A0

Hi Howard,

=C2=A0

could you check whether the JobM= anager's actor system was bound to "vip-rc-vsubu.vclound.com:55926"? You should see that in = the job manager logs. Furthermore, have you checked that you Yarn cluster nodes are actually reachable from the node w= here you start the Flink application? If so, the logs of the cli client as = well as the JobManager logs (both on debug level) would be tremendously hel= pful.

=C2=A0

Cheers,

Till

=C2=A0

On Fri, Feb 17, 2017 at 10:41 AM= , Howard,Li(vip.com) <howard.l= i@vipshop.com> wrote:

Sorry for t= he confusion I made. I do copy the wrong log, but we do meet this problem on 1.2.0.

for version= 1.1.4 however, we meet this in one cluster but not in another. We are stil= l trying to figure out what happened.

=C2=A0

The followi= ng is the log for 1.2.0 version:

=C2=A0

2017-02-17 = 15:51:37,775 INFO=C2=A0 org.apache.flink.yarn.cli.FlinkYarnSessionCli= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 - No path for the flink jar passed. Using the location of class org.apache= .flink.yarn.YarnClusterDescriptor to locate the jar

2017-02-17 = 15:51:37,775 INFO=C2=A0 org.apache.flink.yarn.cli.FlinkYarnSessionCli= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 - No path for the flink jar passed. Using the location of class org.apache= .flink.yarn.YarnClusterDescriptor to locate the jar

2017-02-17 = 15:51:37,803 INFO=C2=A0 org.apache.flink.yarn.YarnClusterDescriptor=C2= =A0=C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0- Using values:

2017-02-17 = 15:51:37,804 INFO=C2=A0 org.apache.flink.yarn.YarnClusterDescriptor=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - =C2=A0=C2=A0 TaskManager count =3D 2

2017-02-17 = 15:51:37,804 INFO=C2=A0 org.apache.flink.yarn.YarnClusterDescriptor=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - =C2=A0=C2=A0 JobManager memory =3D 1024

2017-02-17 = 15:51:37,804 INFO=C2=A0 org.apache.flink.yarn.YarnClusterDescriptor=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - =C2=A0=C2=A0 TaskManager memory =3D 1024

2017-02-17 = 15:51:37,827 INFO=C2=A0 org.apache.hadoop.yarn.client.RMProxy=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Connecting to ResourceManager at /0.0.0.0:8032

2017-02-17 = 15:51:38,672 WARN=C2=A0 org.apache.flink.yarn.YarnClusterDescriptor=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - The configuration directory ('/home/software/flink-1.2.0/conf&#= 39;) contains both LOG4J and Logback configuration files. Please delete or = rename one of them.

2017-02-17 = 15:51:38,685 INFO=C2=A0 org.apache.flink.yarn.Utils=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Copying from file:/home/software/flink-1.2.0/examples/batch/WordCou= nt.jar to hdfs://10.199.202.161:9000/user/root/.flink/appli= cation_1487247313588_0016/WordCount.jar

2017-02-17 = 15:51:38,992 INFO=C2=A0 org.apache.flink.yarn.Utils=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Copying from file:/home/software/flink-1.2.0/conf/log4j.properties = to hdfs://10.199.202.161:9000/user/root/.flink/applicati= on_1487247313588_0016/log4j.properties

2017-02-17 = 15:51:39,058 INFO=C2=A0 org.apache.flink.yarn.Utils=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Copying from file:/home/software/flink-1.2.0/conf/logback.xml to hd= fs://10.199.202.161:9000/user/root/.flink/application_14= 87247313588_0016/logback.xml

2017-02-17 = 15:51:39,085 INFO=C2=A0 org.apache.flink.yarn.Utils=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Copying from file:/home/software/flink-1.2.0/lib to hdfs://10.199= .202.161:9000/user/root/.flink/application_1487247313588_0016/lib= <= u class=3D"m_-4120454232305343109gmail_msg">

2017-02-17 = 15:51:39,695 INFO=C2=A0 org.apache.flink.yarn.Utils=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Copying from file:/home/software/flink-1.2.0/lib/flink-dist_2.11-1.= 2.0.jar to hdfs://10.199.202.161:9000/user= /root/.flink/application_1487247313588_0016/flink-dist_2.11-1.2.0= .jar

2017-02-17 = 15:51:40,493 INFO=C2=A0 org.apache.flink.yarn.Utils=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Copying from /home/software/flink-1.2.0/conf/flink-conf.yaml to hdf= s://10.199.202.161:9000/user/root/.flink/application_1487247313588_0016/flink-conf.yaml

2017-02-17 = 15:51:40,547 INFO=C2=A0 org.apache.flink.yarn.YarnClusterDescriptor=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Submitting application master application_1487247313588_0016

2017-02-17 = 15:51:40,585 INFO=C2=A0 org.apache.hadoop.yarn.client.api.impl.YarnCli= entImpl=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Submitted application application_1487247313588_0016

2017-02-17 = 15:51:40,585 INFO=C2=A0 org.apache.flink.yarn.YarnClusterDescriptor=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Waiting for the cluster to be allocated

2017-02-17 = 15:51:40,587 INFO=C2=A0 org.apache.flink.yarn.YarnClusterDescriptor=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Deploying cluster, current state ACCEPTED

2017-02-17 = 15:51:45,879 INFO=C2=A0 org.apache.flink.yarn.YarnClusterDescriptor=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - YARN application has been deployed successfully.

Cluster sta= rted: Yarn cluster with application id application_1487247313588_0016

Using addre= ss vip-rc-vsubu.vclound.com:55926 to co= nnect to JobManager.

JobManager = web interface address http://vip-rc-ucsww.vclound.com:8088/proxy/application_1487247313= 588_0016/

Using the p= arallelism provided by the remote cluster (8). To use another parallelism, set it at the ./bin/flink client.<= /u>

Starting ex= ecution of program

2017-02-17 = 15:51:46,704 INFO=C2=A0 org.apache.flink.yarn.YarnClusterClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Starting program in interactive mode

Executing W= ordCount example with default input data set.

Use --input= to specify file input.

Printing re= sult to stdout. Use --output to specify output path.

2017-02-17 = 15:51:47,029 INFO=C2=A0 org.apache.flink.yarn.YarnClusterClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Waiting until all TaskManagers have connected

Waiting unt= il all TaskManagers have connected<= /u>

2017-02-17 = 15:51:47,029 INFO=C2=A0 org.apache.flink.yarn.YarnClusterClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Starting client actor system.

=C2=A0

-----------= -------------------------------------------------

The program= finished with the following exception:

=C2=A0

org.apache.= flink.client.program.ProgramInvocationException: The main method = caused an error.

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.pro= gram.PackagedProgram.callMainMethod(PackagedProgram.java:545)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.pro= gram.PackagedProgram.invokeInteractiveModeForExecution(PackagedPr= ogram.java:419)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.pro= gram.ClusterClient.run(ClusterClient.java:339)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.Cli= Frontend.executeProgram(CliFrontend.java:831)=

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.Cli= Frontend.run(CliFrontend.java:256)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.Cli= Frontend.parseParameters(CliFrontend.java:1073)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.Cli= Frontend$2.call(CliFrontend.java:1120)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.Cli= Frontend$2.call(CliFrontend.java:1117)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.runtime.se= curity.HadoopSecurityContext$1.run(HadoopSecurityContext.java:43)=

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.security.AccessControl= ler.doPrivileged(Native Method)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at javax.security.auth.Subject.doAs(Subject.java:422)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.security.= UserGroupInformation.doAs(UserGroupInformation.java:1657)<= span lang=3D"EN-US" class=3D"m_-4120454232305343109gmail_msg">

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.runtime.se= curity.HadoopSecurityContext.runSecured(HadoopSecurityContex= t.java:40)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.Cli= Frontend.main(CliFrontend.java:1116)

Caused by: = java.lang.RuntimeException: Unable to get ClusterClient status from Applica= tion Client

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.yarn.YarnC= lusterClient.getClusterStatus(YarnClusterClient.java:248)<= span lang=3D"EN-US" class=3D"m_-4120454232305343109gmail_msg">

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.yarn.YarnC= lusterClient.waitForClusterToBeReady(YarnClusterClient.java:520)<= /span>

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.pro= gram.ClusterClient.run(ClusterClient.java:412)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.yarn.YarnC= lusterClient.submitJob(YarnClusterClient.java:210)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.pro= gram.ClusterClient.run(ClusterClient.java:400)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.pro= gram.ClusterClient.run(ClusterClient.java:387)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.pro= gram.ContextEnvironment.execute(ContextEnvironment.java:62)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.api.java.E= xecutionEnvironment.execute(ExecutionEnvironment.java:926)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.api.java.D= ataSet.collect(DataSet.java:410)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.api.java.D= ataSet.print(DataSet.java:1605)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.examples.j= ava.wordcount.WordCount.main(WordCount.java:92)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at sun.reflect.NativeMethodAcc= essorImpl.invoke0(Native Method)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at sun.reflect.NativeMethodAcc= essorImpl.invoke(NativeMethodAccessorImpl.java:62)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at sun.reflect.DelegatingMetho= dAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.lang.reflect.Method.in= voke(Method.java:498)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.pro= gram.PackagedProgram.callMainMethod(PackagedProgram.java:528)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ... 13 more

Caused by: = org.apache.flink.runtime.leaderretrieval.LeaderRetrievalException= : Could not retrieve the leader gateway

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.runtime.util.LeaderRetrievalUtils.retrieveLeaderGateway(LeaderRetrievalUtil= s.java:141)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.pro= gram.ClusterClient.getJobManagerGateway(ClusterClient.java:691)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.yarn.YarnC= lusterClient.getClusterStatus(YarnClusterClient.java:242)<= span lang=3D"EN-US" class=3D"m_-4120454232305343109gmail_msg">

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ... 28 more

Caused by: = java.util.concurrent.TimeoutException: Futures timed out after [10000 = milliseconds]

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.concurrent.Await$$ano= nfun$result$1.apply(package.scala:190)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.concurrent.Await$.res= ult(package.scala:190)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.concurrent.Await.result(package.scala)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.runtime.util.LeaderRetrievalUtils.retrieveLeaderGateway(LeaderRetrievalUtil= s.java:139)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ... 30 more

2017-02-17 = 15:52:21,145 INFO=C2=A0 org.apache.flink.yarn.YarnClusterClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Sending shutdown request to the Application Master

2017-02-17 = 15:52:21,145 INFO=C2=A0 org.apache.flink.yarn.YarnClusterClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Start application client.

2017-02-17 = 15:52:21,151 WARN=C2=A0 org.apache.flink.yarn.YarnClusterClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - YARN reported application state FAILED

2017-02-17 = 15:52:21,152 WARN=C2=A0 org.apache.flink.yarn.YarnClusterClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Diagnostics: Application application_1487247313588_0016 failed 1 times d= ue to AM Container for appattempt_1487247313588_0016_000001 exited wit= h=C2=A0 exitCode: -103

For more de= tailed output, check application tracking page:http://vip-rc-ucsww.= vclound.com:8088/cluster/app/application_1487247313588_0016T= hen, click on links to logs of each attempt.

Diagnostics= : Container [pid=3D18590,containerID=3Dcontainer_1487247313588_0016_01_000001] is running beyond virtual memory limits. Current usage: 266.1 MB of 1 GB p= hysical memory used; 2.2 GB of 2.1 GB virtual memory used. Killing containe= r.

Dump of the= process-tree for container_1487247313588_0016_01_000001 :

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |- PID PPID PGRPID SESSID CMD_NAME = USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |- 18598 18590 18590 18590 (java) 8= 94 48 2294116352 67782 /home/software/jdk1.8.0_111/bin= /java -Xmx424M -Dlog.file=3D/home/software/hadoop-2.7.3/logs/userlogs/= application_1487247313588_0016/container_1487247313588_0016_= 01_000001/jobmanager.log -Dlogback.configurationFile=3Dfile:logback.xm= l -Dlog4j.configuration=3Dfile:log4j.properties org.apache.flink.yarn.<= wbr>YarnApplicationMasterRunner

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |- 18590 18588 18590 18590 (bash) 0= 0 108605440 334 /bin/bash -c /home/software/jdk1.8.0_111/bin/java -Xmx424M=C2=A0 -Dlog.file=3D/home/software/hadoop-2.7.3/logs/userlogs= /application_1487247313588_0016/container_1487247313588_0016= _01_000001/jobmanager.log -Dlogback.configurationFile=3Dfile:logback.x= ml -Dlog4j.configuration=3Dfile:log4j.properties org.apache.flink.yarn= .YarnApplicationMasterRunner=C2=A0 1>/home/software/hadoop-2.7.3/logs/userlogs/application_14872= 47313588_0016/container_1487247313588_0016_01_000001/jobmanager.o= ut 2>/home/software/hadoop-2.7.3/logs/userlogs/application_148= 7247313588_0016/container_1487247313588_0016_01_000001/jobmanager= .err

=C2=A0

Container k= illed on request. Exit code is 143<= /u>

Container e= xited with a non-zero exit code 143=

Failing thi= s attempt. Failing the application.=

2017-02-17 = 15:52:21,160 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Notification about new leader address akka.tcp://flink@vip-rc-vsubu.vclound.com:55926= /user/jobmanager with session ID null.

2017-02-17 = 15:52:21,163 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Sending StopCluster request to JobManager.

2017-02-17 = 15:52:21,164 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Received address of new leader akka.tcp://flink@vip-rc-vsubu.vclound.com:55926/user/<= wbr>jobmanager with session ID null.

2017-02-17 = 15:52:21,165 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Disconnect from JobManager null.=

2017-02-17 = 15:52:21,168 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0 = =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0- Trying to register at JobManager akka.tcp://flink@vip-rc-vsubu.vclound.com:55926/user/<= wbr>jobmanager.

2017-02-17 = 15:52:21,684 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Trying to register at JobManager akka.tcp://flink@vip-rc-vsubu.vclound.com:55926/user= /jobmanager.

2017-02-17 = 15:52:22,174 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Sending StopCluster request to JobManager.

2017-02-17 = 15:52:22,704 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Trying to register at JobManager akka.tcp://flink@vip-rc-vsubu.vclound.com:55926/user= /jobmanager.

2017-02-17 = 15:52:23,194 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Sending StopCluster request to JobManager.

2017-02-17 = 15:52:24,214 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Sending StopCluster request to JobManager.

2017-02-17 = 15:52:24,725 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Trying to register at JobManager akka.tcp://flink@vip-rc-vsubu.vclound.com:55926/user= /jobmanager.

2017-02-17 = 15:52:25,234 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Sending StopCluster request to JobManager.

2017-02-17 = 15:52:26,254 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Sending StopCluster request to JobManager.

2017-02-17 = 15:52:27,274 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Sending StopCluster request to JobManager.

2017-02-17 = 15:52:28,294 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Sending StopCluster request to JobManager.

2017-02-17 = 15:52:28,744 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Trying to register at JobManager akka.tcp://flink@vip-rc-vsubu.vclound.com:55926/user= /jobmanager.

2017-02-17 = 15:52:29,314 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Sending StopCluster request to JobManager.

2017-02-17 = 15:52:30,334 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Sending StopCluster request to JobManager.

2017-02-17 = 15:52:31,155 WARN=C2=A0 org.apache.flink.yarn.YarnClusterClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Error while stopping YARN cluster.

java.util.c= oncurrent.TimeoutException: Futures timed out after [10000 millisecond= s]

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.concurrent.Await$$ano= nfun$ready$1.apply(package.scala:169)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.concurrent.Await$$ano= nfun$ready$1.apply(package.scala:169)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.concurrent.Await$.ready(package.scala:169)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at scala.concurrent.Await.ready(package.scala)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.yarn.YarnC= lusterClient.shutdownCluster(YarnClusterClient.java:372)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.yarn.YarnC= lusterClient.finalizeCluster(YarnClusterClient.java:342)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.pro= gram.ClusterClient.shutdown(ClusterClient.java:208)=

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.Cli= Frontend.run(CliFrontend.java:263)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.Cli= Frontend.parseParameters(CliFrontend.java:1073)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.Cli= Frontend$2.call(CliFrontend.java:1120)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.Cli= Frontend$2.call(CliFrontend.java:1117)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.runtime.se= curity.HadoopSecurityContext$1.run(HadoopSecurityContext.java:43)=

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.security.AccessControl= ler.doPrivileged(Native Method)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at javax.security.auth.Subject.doAs(Subject.java:422)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.hadoop.security.= UserGroupInformation.doAs(UserGroupInformation.java:1657)<= span lang=3D"EN-US" class=3D"m_-4120454232305343109gmail_msg">

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.runtime.se= curity.HadoopSecurityContext.runSecured(HadoopSecurityContex= t.java:40)

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.flink.client.Cli= Frontend.main(CliFrontend.java:1116)

2017-02-17 = 15:52:31,156 INFO=C2=A0 org.apache.flink.yarn.YarnClusterClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Deleting files in hdfs://10.199.202.161:9000/user/root/.flink/applicat= ion_1487247313588_0016=

2017-02-17 = 15:52:31,354 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Sending StopCluster request to JobManager.

2017-02-17 = 15:52:32,163 INFO=C2=A0 org.apache.flink.yarn.YarnClusterClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Application application_1487247313588_0016 finished with state FAILED an= d final state FAILED at 1487317906227

2017-02-17 = 15:52:32,163 WARN=C2=A0 org.apache.flink.yarn.YarnClusterClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Application failed. Diagnostics Application application_1487247313588_00= 16 failed 1 times due to AM Container for appattempt_1487247313588_0016_000001 exited with=C2=A0 exitCode: -103

For more de= tailed output, check application tracking page:http://vip-rc-ucsww.= vclound.com:8088/cluster/app/application_1487247313588_0016T= hen, click on links to logs of each attempt.

Diagnostics= : Container [pid=3D18590,containerID=3Dcontainer_1487247313588_0016_01_000001] is running beyond virtual memory limits. Current usage: 266.1 MB of 1 GB p= hysical memory used; 2.2 GB of 2.1 GB virtual memory used. Killing containe= r.

Dump of the= process-tree for container_1487247313588_0016_01_000001 :

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |- PID PPID PGRPID SESSID CMD_NAME = USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |- 18598 18590 18590 18590 (java) 8= 94 48 2294116352 67782 /home/software/jdk1.8.0_111/bin= /java -Xmx424M -Dlog.file=3D/home/software/hadoop-2.7.3/logs/userlogs/= application_1487247313588_0016/container_1487247313588_0016_= 01_000001/jobmanager.log -Dlogback.configurationFile=3Dfile:logback.xm= l -Dlog4j.configuration=3Dfile:log4j.properties org.apache.flink.yarn.<= wbr>YarnApplicationMasterRunner

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |- 18590 18588 18590 18590 (bash) 0= 0 108605440 334 /bin/bash -c /home/software/jdk1.8.0_111/bin/java -Xmx424M=C2=A0 -Dlog.file=3D/home/software/hadoop-2.7.3/logs/userlogs= /application_1487247313588_0016/container_1487247313588_0016= _01_000001/jobmanager.log -Dlogback.configurationFile=3Dfile:logback.x= ml -Dlog4j.configuration=3Dfile:log4j.properties org.apache.flink.yarn= .YarnApplicationMasterRunner=C2=A0 1>/home/software/hadoop-2.7.3/logs/userlogs/application_14872= 47313588_0016/container_1487247313588_0016_01_000001/jobmanager.o= ut 2>/home/software/hadoop-2.7.3/logs/userlogs/application_148= 7247313588_0016/container_1487247313588_0016_01_000001/jobmanager= .err

=C2=A0

Container k= illed on request. Exit code is 143<= /u>

Container e= xited with a non-zero exit code 143=

Failing thi= s attempt. Failing the application.=

2017-02-17 = 15:52:32,164 WARN=C2=A0 org.apache.flink.yarn.YarnClusterClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - If log aggregation is activated in the Hadoop cluster, we recommend to r= etrieve the full application log using this command:

=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 yarn logs -applicationId applicatio= n_1487247313588_0016

(It sometim= es takes a few seconds until the logs are aggregated)

2017-02-17 = 15:52:32,164 INFO=C2=A0 org.apache.flink.yarn.YarnClusterClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - YARN Client is shutting down=

2017-02-17 = 15:52:32,267 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Stopped Application client.<= u class=3D"m_-4120454232305343109gmail_msg">

2017-02-17 = 15:52:32,267 INFO=C2=A0 org.apache.flink.yarn.ApplicationClient=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Disconnect from JobManager null.=

=C2=A0

=C2=A0

=E5=8F=91=E4=BB=B6=E4=BA=BA: = Bruno Aranda [mailto:brunoaranda@gmail.com]
=E5=8F=91=E9=80=81=E6= =97=B6=E9=97=B4: 2017=E5=B9=B42=E6=9C=8817=E6=97=A5 17:02
=E6=94=B6=E4=BB=B6=E4= =BA=BA: user@flink.apache.org
=E4=B8=BB=E9=A2=98: Re: Can't ru= n flink on yarn on version 1.2.0

=C2=A0

Hi Howard,

=C2=A0

We run Flink 1.2 in Yarn without= issues. Sorry I don't have any specific solution, but are you sure you= don't have some sort of Flink mix? In your logs I can see:

=C2=A0

The configuration directory ('/home/software/flink-= 1.1.4/conf') contains both LOG4J and Logback configuration files. Please delete or rename one of them.

=C2=A0

Where it mentions 1.1.4 in the f= older for the conf dir instead of 1.2.

=C2=A0

Cheers,

=C2=A0

Bruno

=C2=A0

On Fri, 17 Feb 2017 at 08:50 How= ard,Li(vip.com) <howard.li@vip= shop.com> wrote:

Hi,

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 I=E2=80=99m = trying to run flink on yarn by using command: bin/flink run -m yarn-cluster= -yn 2 -ys 4 ./examples/batch/WordCount.jar=

=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 But I got th= e following error:

=C2=A0

2017-02-17 15:52:40,746 INFO=C2=A0 org.ap= ache.flink.yarn.cli.FlinkYarnSessionCli=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - N= o path for the flink jar passed. Using the location of class org.apache.fli= nk.yarn.YarnClusterDescriptor to locate the jar

2017-02-17 15:52:40,746 INFO=C2=A0 org.ap= ache.flink.yarn.cli.FlinkYarnSessionCli=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - N= o path for the flink jar passed. Using the location of class org.apache.fli= nk.yarn.YarnClusterDescriptor to locate the jar

2017-02-17 15:52:40,775 INFO=C2=A0 org.ap= ache.flink.yarn.YarnClusterDescriptor=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 - Using values:

2017-02-17 15:52:40,775 INFO=C2=A0 org.ap= ache.flink.yarn.YarnClusterDescriptor=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 - =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 TaskManager count =3D 2=

2017-02-17 15:52:40,775 INFO=C2=A0 org.ap= ache.flink.yarn.YarnClusterDescriptor=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 - =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 JobManager memory =3D 1= 024

2017-02-17 15:52:40,775 INFO=C2=A0 org.ap= ache.flink.yarn.YarnClusterDescriptor=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 - =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 TaskManager memory =3D = 1024

2017-02-17 15:52:40,796 INFO=C2=A0 org.ap= ache.hadoop.yarn.client.RMProxy=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - Connecting to ResourceManager at /0.0.0.0:8032

2017-02-17 15:52:41,680 WARN=C2=A0 org.ap= ache.flink.yarn.YarnClusterDescriptor=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 - The configuration directory ('/home/software/flink-1.1.4/= conf') contains both LOG4J and Logback configuration files. Please delete or rename one of them.

2017-02-17 15:52:41,702 INFO=C2=A0 org.ap= ache.flink.yarn.Utils=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 - Copying from file:/home/software/flink-1.1.4/conf/logbac= k.xml to hdfs://10.199.202.161:9000/user/root/.flink/applicat= ion_1487247313588_0017/logback.xml=

2017-02-17 15:52:42,025 INFO=C2=A0 org.ap= ache.flink.yarn.Utils=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 - Copying from file:/home/software/flink-1.1.4/lib to hdfs= ://10.199.202.161:9000/user/root/.flink/application_148724731358= 8_0017/lib

2017-02-17 15:52:42,695 INFO=C2=A0 org.ap= ache.flink.yarn.Utils=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 - Copying from file:/home/software/flink-1.1.4/conf/log4j.= properties to hdfs://10.199.202.161:9000/user/root/.flin= k/application_1487247313588_0017/log4j.properties

2017-02-17 15:52:42,722 INFO=C2=A0 org.ap= ache.flink.yarn.Utils=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 - Copying from file:/home/software/flink-1.1.4/lib/flink-d= ist_2.10-1.1.4.jar to hdfs://10.199.202.161:900= 0/user/root/.flink/application_1487247313588_0017/flink-dist_2.10-1.1.4.jar

2017-02-17 15:52:43,346 INFO=C2=A0 org.ap= ache.flink.yarn.Utils=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 - Copying from /home/software/flink-1.1.4/conf/flink-conf.= yaml to hdfs://10.199.202.161:9000/user/root/.flink/appli= cation_1487247313588_0017/flink-conf.yaml

2017-02-17 15:52:43,386 INFO=C2=A0 org.ap= ache.flink.yarn.YarnClusterDescriptor=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 - Submitting application master application_1487247313588_0017

2017-02-17 15:52:43,425 INFO=C2=A0 org.ap= ache.hadoop.yarn.client.api.impl.YarnClientImpl=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 - Submitted application application_148724= 7313588_0017

2017-02-17 15:52:43,425 INFO=C2=A0 org.ap= ache.flink.yarn.YarnClusterDescriptor=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 - Waiting for the cluster to be allocated

2017-02-17 15:52:43,427 INFO=C2=A0 org.ap= ache.flink.yarn.YarnClusterDescriptor=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 - Deploying cluster, current state ACCEPTED

2017-02-17 15:52:48,471 INFO=C2=A0 org.ap= ache.flink.yarn.YarnClusterDescriptor=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 - YARN application has been deployed successfully.

Cluster started: Yarn cluster with applic= ation id application_1487247313588_0017

Using address 10.199.202.162:43809 to connect to JobManager.

JobManager web interface address http://vip-rc-ucsww.vclound.com:8088/proxy/application_1487247313= 588_0017/

Using the parallelism provided by the rem= ote cluster (8). To use another parallelism, set it at the ./bin/flink clie= nt.

Starting execution of program

2017-02-17 15:52:49,278 INFO=C2=A0 org.ap= ache.flink.yarn.YarnClusterClient=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 - Starting program in interactive mode

Executing WordCount example with default = input data set.

Use --input to specify file input.=

Printing result to stdout. Use --output t= o specify output path.

2017-02-17 15:52:49,609 INFO=C2=A0 org.ap= ache.flink.yarn.YarnClusterClient=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 - Waiting until all TaskManagers have connected=

Waiting until all TaskManagers have conne= cted

2017-02-17 15:52:49,610 INFO=C2=A0 org.ap= ache.flink.yarn.YarnClusterClient =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0- Starting client actor system.

=C2=A0

------------------------------------= ------------------------=

The program finished with the following e= xception:

=C2=A0

org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.=

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.client.program.PackagedProgram.callMainMethod(PackagedPr= ogram.java:525)

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.client.program.PackagedProgram.invokeInteractiveModeForExecut= ion(PackagedProgram.java:404)

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.client.program.ClusterClient.run(ClusterClient.java:321)

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.client.CliFrontend.executeProgram(CliFrontend.java:777)

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.client.CliFrontend.run(CliFrontend.java:253)

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.client.CliFrontend.parseParameters(CliFrontend.java:1005)

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.client.CliFrontend.main(CliFrontend.java:1048)<= span lang=3D"EN-US" class=3D"m_-4120454232305343109gmail_msg">

Caused by: java.lang.RuntimeException: Un= able to get ClusterClient status from Application Client

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.yarn.YarnClusterClient.getClusterStatus(YarnClusterClien= t.java:242)

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.yarn.YarnClusterClient.waitForClusterToBeReady(YarnClust= erClient.java:514)

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.client.program.ClusterClient.run(ClusterClient.java:395)

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.yarn.YarnClusterClient.submitJob(YarnClusterClient.java:204)<= /span>

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.client.program.ClusterClient.run(ClusterClient.java:383)

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.client.program.ClusterClient.run(ClusterClient.java:370)

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.client.program.ContextEnvironment.execute(ContextEnvironment.= java:62)

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.ja= va:896)

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.api.java.DataSet.collect(DataSet.java:410)

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.api.java.DataSet.print(DataSet.java:1605)

=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.fl= ink.examples.java.wordcount.WordCount.main(WordCount.java:92)

=C2=A0=C2=A0=C2=A0=C2=A0 at sun.reflect.<= wbr>NativeMethodAccessorImpl.invoke0(Native Meth

=

--94eb2c0bd0f8f7e6940549388815--