flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: Container running beyond physical memory limits when processing DataStream
Date Tue, 02 Aug 2016 12:21:58 GMT
Your job creates a lot of String objects which need to be garbage
collected. It could be that the JVM is not fast enough and Yarn kills
the JVM for consuming too much memory.

You can try two things:

1) Give the task manager more memory
2) Increase the Yarn heap cutoff ratio (e.g yarn.heap-cutoff-ratio: 0.4)

If the error still occurs then we need to investigate further.

Thanks,
Max

On Mon, Aug 1, 2016 at 7:48 PM, Jack Huang <jackhuang@mz.com> wrote:
> Hi Max, here is the YARN task log. Searching for "limit" will bring you to
> the error line.
>
> Container: container_e05_1467433388200_0136_01_000002 on
> node03.mycluster.com_45454
> ============================================================================================
> LogType:taskmanager.err
> Log Upload Time:Thu Jul 28 22:34:43 +0000 2016
> LogLength:0
> Log Contents:
> End of LogType:taskmanager.err
>
> LogType:taskmanager.log
> Log Upload Time:Thu Jul 28 22:34:43 +0000 2016
> LogLength:49515
> Log Contents:
> 2016-07-28 22:34:12,861 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -
> --------------------------------------------------------------------------------
> 2016-07-28 22:34:12,871 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -  Starting YARN TaskManager (Version: 1.0.1, Rev:4afa401, Date:31.03.2016 @
> 13:40:33 UTC)
> 2016-07-28 22:34:12,871 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -  Current user: yarn
> 2016-07-28 22:34:12,871 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -  JVM: Java HotSpot(TM) 64-Bit Server VM - Oracle Corporation -
> 1.8/25.60-b23
> 2016-07-28 22:34:12,872 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -  Maximum heap size: 1388 MiBytes
> 2016-07-28 22:34:12,872 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -  JAVA_HOME: /usr/jdk64/jdk1.8.0_60
> 2016-07-28 22:34:12,874 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -  Hadoop version: 2.7.2
> 2016-07-28 22:34:12,874 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -  JVM Options:
> 2016-07-28 22:34:12,874 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -     -Xms1448m
> 2016-07-28 22:34:12,874 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -     -Xmx1448m
> 2016-07-28 22:34:12,874 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -     -XX:MaxDirectMemorySize=1448m
> 2016-07-28 22:34:12,874 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -
> -Dlog.file=/mnt/a/hadoop/yarn/log/application_1467433388200_0136/container_e05_1467433388200_0136_01_000002/taskmanager.log
> 2016-07-28 22:34:12,875 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -     -Dlogback.configurationFile=file:logback.xml
> 2016-07-28 22:34:12,875 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -     -Dlog4j.configuration=file:log4j.properties
> 2016-07-28 22:34:12,875 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -  Program Arguments:
> 2016-07-28 22:34:12,875 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -     --configDir
> 2016-07-28 22:34:12,875 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -     .
> 2016-07-28 22:34:12,875 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -  Classpath:
> /mnt/c/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/container_e05_1467433388200_0136_01_000002/flink.jar:/usr/hdp/current/hadoop-client/conf:/usr/hdp/current/hadoop-client/hadoop-nfs.jar:/usr/hdp/current/hadoop-client/hadoop-annotations.jar:/usr/hdp/current/hadoop-client/hadoop-common-tests.jar:/usr/hdp/current/hadoop-client/hadoop-annotations-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-auth-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-common.jar:/usr/hdp/current/hadoop-client/hadoop-auth.jar:/usr/hdp/current/hadoop-client/hadoop-aws-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-nfs-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-azure.jar:/usr/hdp/current/hadoop-client/hadoop-common-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-azure-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-common-2.7.1.2.3.4.0-3485-tests.jar:/usr/hdp/current/hadoop-client/hadoop-aws.jar:/usr/hdp/current/hadoop-client/lib/jettison-1.1.jar:/usr/hdp/current/hadoop-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-client/lib/commons-collections-3.2.2.jar:/usr/hdp/current/hadoop-client/lib/jets3t-0.9.0.jar:/usr/hdp/current/hadoop-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-client/lib/jsp-api-2.1.jar:/usr/hdp/current/hadoop-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-client/lib/xz-1.0.jar:/usr/hdp/current/hadoop-client/lib/ranger-yarn-plugin-shim-0.5.0.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/hadoop-lzo-0.6.0.2.3.4.0-3485-test.jar:/usr/hdp/current/hadoop-client/lib/hadoop-lzo-0.6.0.2.3.4.0-3485-sources.jar:/usr/hdp/current/hadoop-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-client/lib/paranamer-2.3.jar:/usr/hdp/current/hadoop-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-client/lib/junit-4.11.jar:/usr/hdp/current/hadoop-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-client/lib/azure-storage-2.2.0.jar:/usr/hdp/current/hadoop-client/lib/slf4j-log4j12-1.7.10.jar:/usr/hdp/current/hadoop-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-client/lib/ranger-plugin-classloader-0.5.0.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-client/lib/hamcrest-core-1.3.jar:/usr/hdp/current/hadoop-client/lib/commons-configuration-1.6.jar:/usr/hdp/current/hadoop-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/stax-api-1.0-2.jar:/usr/hdp/current/hadoop-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-client/lib/commons-beanutils-1.7.0.jar:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-1.7.4.jar:/usr/hdp/current/hadoop-client/lib/gson-2.2.4.jar:/usr/hdp/current/hadoop-client/lib/ojdbc6.jar:/usr/hdp/current/hadoop-client/lib/snappy-java-1.0.4.1.jar:/usr/hdp/current/hadoop-client/lib/api-util-1.0.0-M20.jar:/usr/hdp/current/hadoop-client/lib/jackson-databind-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/commons-compress-1.4.1.jar:/usr/hdp/current/hadoop-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-client/lib/jaxb-api-2.2.2.jar:/usr/hdp/current/hadoop-client/lib/jackson-annotations-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/current/hadoop-client/lib/commons-digester-1.8.jar:/usr/hdp/current/hadoop-client/lib/activation-1.1.jar:/usr/hdp/current/hadoop-client/lib/hadoop-lzo-0.6.0.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/current/hadoop-client/lib/java-xmlbuilder-0.4.jar:/usr/hdp/current/hadoop-client/lib/commons-httpclient-3.1.jar:/usr/hdp/current/hadoop-client/lib/commons-math3-3.1.1.jar:/usr/hdp/current/hadoop-client/lib/jersey-json-1.9.jar:/usr/hdp/current/hadoop-client/lib/ranger-hdfs-plugin-shim-0.5.0.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/microsoft-windowsazure-storage-sdk-0.6.0.jar:/usr/hdp/current/hadoop-client/lib/slf4j-api-1.7.10.jar:/usr/hdp/current/hadoop-client/lib/avro-1.7.4.jar:/usr/hdp/current/hadoop-client/lib/mockito-all-1.8.5.jar:/usr/hdp/current/hadoop-client/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hdp/current/hadoop-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-client/lib/commons-beanutils-core-1.8.0.jar:/usr/hdp/current/hadoop-client/lib/commons-net-3.1.jar:/usr/hdp/current/hadoop-client/lib/curator-recipes-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/jsch-0.1.42.jar:/usr/hdp/current/hadoop-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-client/lib/jackson-core-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/curator-framework-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/jackson-xc-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/httpclient-4.2.5.jar:/usr/hdp/current/hadoop-client/lib/curator-client-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/api-asn1-api-1.0.0-M20.jar:/usr/hdp/current/hadoop-client/lib/httpcore-4.2.5.jar:/usr/hdp/current/hadoop-client/lib/zookeeper-3.4.6.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-nfs.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-2.7.1.2.3.4.0-3485-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-nfs-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-hdfs-client/lib/okhttp-2.4.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xml-apis-1.3.04.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-daemon-1.0.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-hdfs-client/lib/okio-1.4.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-hdfs-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-hdfs-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-hdfs-client/lib/leveldbjni-all-1.8.jar:/usr/hdp/current/hadoop-hdfs-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xercesImpl-2.9.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/netty-all-4.0.23.Final.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-hdfs-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-applicationhistoryservice.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-registry-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-web-proxy.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-api.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-resourcemanager-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-nodemanager-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-common-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-sharedcachemanager-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-registry.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-client.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-timeline-plugins-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-timeline-plugins.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-api-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-web-proxy-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-common.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-applicationhistoryservice-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-sharedcachemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-client-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-resourcemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-common.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-unmanaged-am-launcher-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-nodemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-tests-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-common-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-tests.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-client-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/javassist-3.18.1-GA.jar:/usr/hdp/current/hadoop-yarn-client/lib/jettison-1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/aopalliance-1.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-collections-3.2.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/javax.inject-1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jets3t-0.9.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsp-api-2.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-yarn-client/lib/xz-1.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/paranamer-2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-yarn-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-yarn-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-yarn-client/lib/guice-3.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-configuration-1.6.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/stax-api-1.0-2.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/fst-2.24.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/zookeeper-3.4.6.2.3.4.0-3485-tests.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-beanutils-1.7.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/gson-2.2.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/leveldbjni-all-1.8.jar:/usr/hdp/current/hadoop-yarn-client/lib/snappy-java-1.0.4.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/api-util-1.0.0-M20.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-databind-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-compress-1.4.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/jaxb-api-2.2.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-annotations-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-digester-1.8.jar:/usr/hdp/current/hadoop-yarn-client/lib/activation-1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-guice-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/java-xmlbuilder-0.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-httpclient-3.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-math3-3.1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-json-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/microsoft-windowsazure-storage-sdk-0.6.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/guice-servlet-3.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/avro-1.7.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hdp/current/hadoop-yarn-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-yarn-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-beanutils-core-1.8.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-net-3.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-recipes-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsch-0.1.42.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/objenesis-2.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-core-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-framework-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-xc-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/httpclient-4.2.5.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-client-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/api-asn1-api-1.0.0-M20.jar:/usr/hdp/current/hadoop-yarn-client/lib/httpcore-4.2.5.jar:/usr/hdp/current/hadoop-yarn-client/lib/zookeeper-3.4.6.2.3.4.0-3485.jar
> 2016-07-28 22:34:12,877 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> -
> --------------------------------------------------------------------------------
> 2016-07-28 22:34:12,878 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> - Registered UNIX signal handlers for [TERM, HUP, INT]
> 2016-07-28 22:34:13,098 INFO
> org.apache.flink.runtime.taskmanager.TaskManager              - Loading
> configuration from .
> 2016-07-28 22:34:13,111 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> - Setting directories for temporary file
> /mnt/a/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136,/mnt/b/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136,/mnt/c/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136,/mnt/d/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136
> 2016-07-28 22:34:13,111 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> - YARN daemon runs as 'yarn' setting user to execute Flink TaskManager to
> 'user'
> 2016-07-28 22:34:13,126 INFO  org.apache.flink.runtime.util.ZooKeeperUtils
> - Using '/flink' as root namespace.
> 2016-07-28 22:34:13,224 INFO
> org.apache.flink.shaded.org.apache.curator.framework.imps.CuratorFrameworkImpl
> - Starting
> 2016-07-28 22:34:13,242 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014
> 09:09 GMT
> 2016-07-28 22:34:13,242 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:host.name=node03.mycluster.com
> 2016-07-28 22:34:13,242 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:java.version=1.8.0_60
> 2016-07-28 22:34:13,242 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:java.vendor=Oracle Corporation
> 2016-07-28 22:34:13,242 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:java.home=/usr/jdk64/jdk1.8.0_60/jre
> 2016-07-28 22:34:13,243 INFO  org.apache.zookeeper.ZooKeeper
> - Client
> environment:java.class.path=/mnt/c/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/container_e05_1467433388200_0136_01_000002/flink.jar:/usr/hdp/current/hadoop-client/conf:/usr/hdp/current/hadoop-client/hadoop-nfs.jar:/usr/hdp/current/hadoop-client/hadoop-annotations.jar:/usr/hdp/current/hadoop-client/hadoop-common-tests.jar:/usr/hdp/current/hadoop-client/hadoop-annotations-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-auth-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-common.jar:/usr/hdp/current/hadoop-client/hadoop-auth.jar:/usr/hdp/current/hadoop-client/hadoop-aws-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-nfs-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-azure.jar:/usr/hdp/current/hadoop-client/hadoop-common-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-azure-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-common-2.7.1.2.3.4.0-3485-tests.jar:/usr/hdp/current/hadoop-client/hadoop-aws.jar:/usr/hdp/current/hadoop-client/lib/jettison-1.1.jar:/usr/hdp/current/hadoop-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-client/lib/commons-collections-3.2.2.jar:/usr/hdp/current/hadoop-client/lib/jets3t-0.9.0.jar:/usr/hdp/current/hadoop-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-client/lib/jsp-api-2.1.jar:/usr/hdp/current/hadoop-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-client/lib/xz-1.0.jar:/usr/hdp/current/hadoop-client/lib/ranger-yarn-plugin-shim-0.5.0.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/hadoop-lzo-0.6.0.2.3.4.0-3485-test.jar:/usr/hdp/current/hadoop-client/lib/hadoop-lzo-0.6.0.2.3.4.0-3485-sources.jar:/usr/hdp/current/hadoop-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-client/lib/paranamer-2.3.jar:/usr/hdp/current/hadoop-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-client/lib/junit-4.11.jar:/usr/hdp/current/hadoop-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-client/lib/azure-storage-2.2.0.jar:/usr/hdp/current/hadoop-client/lib/slf4j-log4j12-1.7.10.jar:/usr/hdp/current/hadoop-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-client/lib/ranger-plugin-classloader-0.5.0.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-client/lib/hamcrest-core-1.3.jar:/usr/hdp/current/hadoop-client/lib/commons-configuration-1.6.jar:/usr/hdp/current/hadoop-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/stax-api-1.0-2.jar:/usr/hdp/current/hadoop-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-client/lib/commons-beanutils-1.7.0.jar:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-1.7.4.jar:/usr/hdp/current/hadoop-client/lib/gson-2.2.4.jar:/usr/hdp/current/hadoop-client/lib/ojdbc6.jar:/usr/hdp/current/hadoop-client/lib/snappy-java-1.0.4.1.jar:/usr/hdp/current/hadoop-client/lib/api-util-1.0.0-M20.jar:/usr/hdp/current/hadoop-client/lib/jackson-databind-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/commons-compress-1.4.1.jar:/usr/hdp/current/hadoop-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-client/lib/jaxb-api-2.2.2.jar:/usr/hdp/current/hadoop-client/lib/jackson-annotations-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/current/hadoop-client/lib/commons-digester-1.8.jar:/usr/hdp/current/hadoop-client/lib/activation-1.1.jar:/usr/hdp/current/hadoop-client/lib/hadoop-lzo-0.6.0.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/current/hadoop-client/lib/java-xmlbuilder-0.4.jar:/usr/hdp/current/hadoop-client/lib/commons-httpclient-3.1.jar:/usr/hdp/current/hadoop-client/lib/commons-math3-3.1.1.jar:/usr/hdp/current/hadoop-client/lib/jersey-json-1.9.jar:/usr/hdp/current/hadoop-client/lib/ranger-hdfs-plugin-shim-0.5.0.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/microsoft-windowsazure-storage-sdk-0.6.0.jar:/usr/hdp/current/hadoop-client/lib/slf4j-api-1.7.10.jar:/usr/hdp/current/hadoop-client/lib/avro-1.7.4.jar:/usr/hdp/current/hadoop-client/lib/mockito-all-1.8.5.jar:/usr/hdp/current/hadoop-client/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hdp/current/hadoop-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-client/lib/commons-beanutils-core-1.8.0.jar:/usr/hdp/current/hadoop-client/lib/commons-net-3.1.jar:/usr/hdp/current/hadoop-client/lib/curator-recipes-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/jsch-0.1.42.jar:/usr/hdp/current/hadoop-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-client/lib/jackson-core-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/curator-framework-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/jackson-xc-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/httpclient-4.2.5.jar:/usr/hdp/current/hadoop-client/lib/curator-client-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/api-asn1-api-1.0.0-M20.jar:/usr/hdp/current/hadoop-client/lib/httpcore-4.2.5.jar:/usr/hdp/current/hadoop-client/lib/zookeeper-3.4.6.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-nfs.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-2.7.1.2.3.4.0-3485-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-nfs-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-hdfs-client/lib/okhttp-2.4.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xml-apis-1.3.04.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-daemon-1.0.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-hdfs-client/lib/okio-1.4.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-hdfs-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-hdfs-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-hdfs-client/lib/leveldbjni-all-1.8.jar:/usr/hdp/current/hadoop-hdfs-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xercesImpl-2.9.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/netty-all-4.0.23.Final.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-hdfs-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-applicationhistoryservice.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-registry-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-web-proxy.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-api.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-resourcemanager-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-nodemanager-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-common-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-sharedcachemanager-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-registry.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-client.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-timeline-plugins-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-timeline-plugins.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-api-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-web-proxy-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-common.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-applicationhistoryservice-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-sharedcachemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-client-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-resourcemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-common.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-unmanaged-am-launcher-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-nodemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-tests-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-common-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-tests.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-client-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/javassist-3.18.1-GA.jar:/usr/hdp/current/hadoop-yarn-client/lib/jettison-1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/aopalliance-1.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-collections-3.2.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/javax.inject-1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jets3t-0.9.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsp-api-2.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-yarn-client/lib/xz-1.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/paranamer-2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-yarn-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-yarn-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-yarn-client/lib/guice-3.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-configuration-1.6.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/stax-api-1.0-2.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/fst-2.24.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/zookeeper-3.4.6.2.3.4.0-3485-tests.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-beanutils-1.7.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/gson-2.2.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/leveldbjni-all-1.8.jar:/usr/hdp/current/hadoop-yarn-client/lib/snappy-java-1.0.4.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/api-util-1.0.0-M20.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-databind-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-compress-1.4.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/jaxb-api-2.2.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-annotations-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-digester-1.8.jar:/usr/hdp/current/hadoop-yarn-client/lib/activation-1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-guice-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/java-xmlbuilder-0.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-httpclient-3.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-math3-3.1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-json-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/microsoft-windowsazure-storage-sdk-0.6.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/guice-servlet-3.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/avro-1.7.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hdp/current/hadoop-yarn-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-yarn-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-beanutils-core-1.8.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-net-3.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-recipes-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsch-0.1.42.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/objenesis-2.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-core-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-framework-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-xc-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/httpclient-4.2.5.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-client-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/api-asn1-api-1.0.0-M20.jar:/usr/hdp/current/hadoop-yarn-client/lib/httpcore-4.2.5.jar:/usr/hdp/current/hadoop-yarn-client/lib/zookeeper-3.4.6.2.3.4.0-3485.jar
> 2016-07-28 22:34:13,244 INFO  org.apache.zookeeper.ZooKeeper
> - Client
> environment:java.library.path=::/usr/hdp/2.3.4.0-3485/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.3.4.0-3485/hadoop/lib/native::/usr/hdp/2.3.4.0-3485/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.3.4.0-3485/hadoop/lib/native:/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir:/usr/hdp/2.3.4.0-3485/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.3.4.0-3485/hadoop/lib/native:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 2016-07-28 22:34:13,244 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:java.io.tmpdir=/tmp
> 2016-07-28 22:34:13,244 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:java.compiler=<NA>
> 2016-07-28 22:34:13,244 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:os.name=Linux
> 2016-07-28 22:34:13,244 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:os.arch=amd64
> 2016-07-28 22:34:13,244 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:os.version=3.13.0-32-generic
> 2016-07-28 22:34:13,244 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:user.name=yarn
> 2016-07-28 22:34:13,244 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:user.home=/home/yarn
> 2016-07-28 22:34:13,244 INFO  org.apache.zookeeper.ZooKeeper
> - Client
> environment:user.dir=/mnt/c/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/container_e05_1467433388200_0136_01_000002
> 2016-07-28 22:34:13,245 INFO  org.apache.zookeeper.ZooKeeper
> - Initiating client connection,
> connectString=node01:2181,node02:2181,node03:2181 sessionTimeout=60000
> watcher=org.apache.flink.shaded.org.apache.curator.ConnectionState@5f8e8a9d
> 2016-07-28 22:34:13,314 INFO  org.apache.zookeeper.ClientCnxn
> - Opening socket connection to server
> node03.mycluster.com/38.111.30.146:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2016-07-28 22:34:13,327 INFO  org.apache.zookeeper.ClientCnxn
> - Socket connection established to node03.mycluster.com/38.111.30.146:2181,
> initiating session
> 2016-07-28 22:34:13,354 INFO  org.apache.zookeeper.ClientCnxn
> - Session establishment complete on server
> node03.mycluster.com/38.111.30.146:2181, sessionid = 0x3556f328b260300,
> negotiated timeout = 40000
> 2016-07-28 22:34:13,360 INFO
> org.apache.flink.shaded.org.apache.curator.framework.state.ConnectionStateManager
> - State change: CONNECTED
> 2016-07-28 22:34:14,430 INFO
> org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService  -
> Starting ZooKeeperLeaderRetrievalService.
> 2016-07-28 22:34:14,436 INFO
> org.apache.flink.runtime.util.LeaderRetrievalUtils            - Trying to
> select the network interface and address to use by connecting to the leading
> JobManager.
> 2016-07-28 22:34:14,437 INFO
> org.apache.flink.runtime.util.LeaderRetrievalUtils            - TaskManager
> will try to connect for 10000 milliseconds before falling back to heuristics
> 2016-07-28 22:34:14,549 INFO  org.apache.flink.runtime.net.ConnectionUtils
> - Retrieved new target address /38.111.30.148:59857.
> 2016-07-28 22:34:14,578 INFO
> org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService  -
> Stopping ZooKeeperLeaderRetrievalService.
> 2016-07-28 22:34:14,590 INFO  org.apache.zookeeper.ZooKeeper
> - Session: 0x3556f328b260300 closed
> 2016-07-28 22:34:14,590 INFO  org.apache.zookeeper.ClientCnxn
> - EventThread shut down
> 2016-07-28 22:34:14,597 INFO
> org.apache.flink.runtime.taskmanager.TaskManager              - TaskManager
> will use hostname/address 'node03' (38.111.30.146) for communication.
> 2016-07-28 22:34:14,598 INFO
> org.apache.flink.runtime.taskmanager.TaskManager              - Starting
> TaskManager
> 2016-07-28 22:34:14,599 INFO
> org.apache.flink.runtime.taskmanager.TaskManager              - Starting
> TaskManager actor system at 38.111.30.146:0
> 2016-07-28 22:34:15,001 INFO  akka.event.slf4j.Slf4jLogger
> - Slf4jLogger started
> 2016-07-28 22:34:15,046 INFO  Remoting
> - Starting remoting
> 2016-07-28 22:34:15,353 INFO  Remoting
> - Remoting started; listening on addresses
> :[akka.tcp://flink@38.111.30.146:48700]
> 2016-07-28 22:34:15,360 INFO
> org.apache.flink.runtime.taskmanager.TaskManager              - Starting
> TaskManager actor
> 2016-07-28 22:34:15,369 INFO
> org.apache.flink.runtime.io.network.netty.NettyConfig         - NettyConfig
> [server address: node03/38.111.30.146, server port: 40641, memory segment
> size (bytes): 32768, transport type: NIO, number of server threads: 1
> (manual), number of client threads: 1 (manual), server connect backlog: 0
> (use Netty's default), client connect timeout (sec): 120, send/receive
> buffer size (bytes): 0 (use Netty's default)]
> 2016-07-28 22:34:15,372 INFO
> org.apache.flink.runtime.taskmanager.TaskManager              - Messages
> between TaskManager and JobManager have a max timeout of 10000 milliseconds
> 2016-07-28 22:34:15,378 INFO
> org.apache.flink.runtime.taskmanager.TaskManager              - Temporary
> file directory
> '/mnt/a/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136':
> total 2734 GB, usable 2055 GB (75.16% usable)
> 2016-07-28 22:34:15,379 INFO
> org.apache.flink.runtime.taskmanager.TaskManager              - Temporary
> file directory
> '/mnt/b/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136':
> total 2750 GB, usable 2064 GB (75.05% usable)
> 2016-07-28 22:34:15,379 INFO
> org.apache.flink.runtime.taskmanager.TaskManager              - Temporary
> file directory
> '/mnt/c/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136':
> total 2750 GB, usable 2233 GB (81.20% usable)
> 2016-07-28 22:34:15,380 INFO
> org.apache.flink.runtime.taskmanager.TaskManager              - Temporary
> file directory
> '/mnt/d/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136':
> total 2750 GB, usable 2466 GB (89.67% usable)
> 2016-07-28 22:34:15,416 INFO
> org.apache.flink.runtime.io.network.buffer.NetworkBufferPool  - Allocated 64
> MB for network buffer pool (number of memory segments: 2048, bytes per
> segment: 32768).
> 2016-07-28 22:34:15,599 INFO
> org.apache.flink.runtime.taskmanager.TaskManager              - Limiting
> managed memory to 0.7 of the currently free heap space (922 MB), memory will
> be allocated lazily.
> 2016-07-28 22:34:15,626 INFO
> org.apache.flink.runtime.io.disk.iomanager.IOManager          - I/O manager
> uses directory
> /mnt/a/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/flink-io-96e84170-f59d-4316-bda1-d2c2810fdcdf
> for spill files.
> 2016-07-28 22:34:15,626 INFO
> org.apache.flink.runtime.io.disk.iomanager.IOManager          - I/O manager
> uses directory
> /mnt/b/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/flink-io-996962ff-d621-4285-a4d2-7eb9e846ee11
> for spill files.
> 2016-07-28 22:34:15,626 INFO
> org.apache.flink.runtime.io.disk.iomanager.IOManager          - I/O manager
> uses directory
> /mnt/c/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/flink-io-4b2cc5e6-110c-43aa-a1b1-620058e12a94
> for spill files.
> 2016-07-28 22:34:15,627 INFO
> org.apache.flink.runtime.io.disk.iomanager.IOManager          - I/O manager
> uses directory
> /mnt/d/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/flink-io-a9c9d249-da5c-4c3c-a208-06603df1e966
> for spill files.
> 2016-07-28 22:34:15,630 INFO  org.apache.flink.runtime.util.ZooKeeperUtils
> - Using '/flink' as root namespace.
> 2016-07-28 22:34:15,630 INFO
> org.apache.flink.shaded.org.apache.curator.framework.imps.CuratorFrameworkImpl
> - Starting
> 2016-07-28 22:34:15,631 INFO  org.apache.zookeeper.ZooKeeper
> - Initiating client connection,
> connectString=node01:2181,node02:2181,node03:2181 sessionTimeout=60000
> watcher=org.apache.flink.shaded.org.apache.curator.ConnectionState@4678a2eb
> 2016-07-28 22:34:15,634 INFO  org.apache.zookeeper.ClientCnxn
> - Opening socket connection to server
> node02.mycluster.com/38.111.30.145:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2016-07-28 22:34:15,634 INFO  org.apache.zookeeper.ClientCnxn
> - Socket connection established to node02.mycluster.com/38.111.30.145:2181,
> initiating session
> 2016-07-28 22:34:15,656 INFO  org.apache.zookeeper.ClientCnxn
> - Session establishment complete on server
> node02.mycluster.com/38.111.30.145:2181, sessionid = 0x255bc6698ff025f,
> negotiated timeout = 40000
> 2016-07-28 22:34:15,656 INFO
> org.apache.flink.shaded.org.apache.curator.framework.state.ConnectionStateManager
> - State change: CONNECTED
> 2016-07-28 22:34:15,667 INFO  org.apache.flink.runtime.filecache.FileCache
> - User file cache uses directory
> /mnt/a/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/flink-dist-cache-cba06958-4240-4070-b611-d4687e1bbea0
> 2016-07-28 22:34:15,667 INFO  org.apache.flink.runtime.filecache.FileCache
> - User file cache uses directory
> /mnt/b/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/flink-dist-cache-cba06958-4240-4070-b611-d4687e1bbea0
> 2016-07-28 22:34:15,667 INFO  org.apache.flink.runtime.filecache.FileCache
> - User file cache uses directory
> /mnt/c/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/flink-dist-cache-cba06958-4240-4070-b611-d4687e1bbea0
> 2016-07-28 22:34:15,667 INFO  org.apache.flink.runtime.filecache.FileCache
> - User file cache uses directory
> /mnt/d/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/flink-dist-cache-cba06958-4240-4070-b611-d4687e1bbea0
> 2016-07-28 22:34:15,964 INFO  org.apache.flink.yarn.YarnTaskManager
> - Starting TaskManager actor at akka://flink/user/taskmanager#-1203652007.
> 2016-07-28 22:34:15,964 INFO  org.apache.flink.yarn.YarnTaskManager
> - TaskManager data connection information: node03.mycluster.com
> (dataPort=40641)
> 2016-07-28 22:34:15,965 INFO  org.apache.flink.yarn.YarnTaskManager
> - TaskManager has 1 task slot(s).
> 2016-07-28 22:34:15,967 INFO  org.apache.flink.yarn.YarnTaskManager
> - Memory usage stats: [HEAP: 123/1388/1388 MB, NON HEAP: 33/34/-1 MB
> (used/committed/max)]
> 2016-07-28 22:34:15,967 INFO
> org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService  -
> Starting ZooKeeperLeaderRetrievalService.
> 2016-07-28 22:34:15,977 INFO  org.apache.flink.yarn.YarnTaskManager
> - Trying to register at JobManager
> akka.tcp://flink@38.111.30.148:59857/user/jobmanager (attempt 1, timeout:
> 500 milliseconds)
> 2016-07-28 22:34:16,187 INFO  org.apache.flink.yarn.YarnTaskManager
> - Successful registration at JobManager
> (akka.tcp://flink@38.111.30.148:59857/user/jobmanager), starting network
> stack and library cache.
> 2016-07-28 22:34:16,445 INFO
> org.apache.flink.runtime.io.network.netty.NettyClient         - Successful
> initialization (took 77 ms).
> 2016-07-28 22:34:16,520 INFO
> org.apache.flink.runtime.io.network.netty.NettyServer         - Successful
> initialization (took 74 ms). Listening on SocketAddress
> /38.111.30.146:40641.
> 2016-07-28 22:34:16,521 INFO  org.apache.flink.yarn.YarnTaskManager
> - Determined BLOB server address to be /38.111.30.148:46326. Starting BLOB
> cache.
> 2016-07-28 22:34:16,551 INFO  org.apache.flink.runtime.blob.BlobCache
> - Created BLOB cache storage directory
> /tmp/blobStore-6505c99f-30f2-4a6f-a986-6bffccc0ddf9
> 2016-07-28 22:34:23,537 INFO  org.apache.flink.yarn.YarnTaskManager
> - Received task Source: Custom Source -> Sink: Unnamed (1/1)
> 2016-07-28 22:34:23,545 INFO  org.apache.flink.runtime.taskmanager.Task
> - Loading JAR files for task Source: Custom Source -> Sink: Unnamed (1/1)
> 2016-07-28 22:34:23,546 INFO  org.apache.flink.runtime.blob.BlobCache
> - Downloading d44873ce558492ab4d65fb4f9ca3d2da3090041f from
> /38.111.30.148:46326
> 2016-07-28 22:34:24,528 INFO  org.apache.flink.runtime.taskmanager.Task
> - Registering task at network: Source: Custom Source -> Sink: Unnamed (1/1)
> [DEPLOYING]
> 2016-07-28 22:34:24,531 INFO  org.apache.flink.runtime.taskmanager.Task
> - Source: Custom Source -> Sink: Unnamed (1/1) switched to RUNNING
> 2016-07-28 22:34:24,664 INFO
> org.apache.flink.streaming.runtime.tasks.StreamTask           - Using
> user-defined state backend: MemoryStateBackend (data in heap memory /
> checkpoints to JobManager)
> 2016-07-28 22:34:24,675 INFO
> org.apache.flink.streaming.runtime.tasks.StreamTask           - Using
> user-defined state backend: MemoryStateBackend (data in heap memory /
> checkpoints to JobManager)
> 2016-07-28 22:34:24,743 INFO
> org.apache.kafka.clients.consumer.ConsumerConfig              -
> ConsumerConfig values:
>     metric.reporters = []
>     metadata.max.age.ms = 300000
>     value.deserializer = class
> org.apache.kafka.common.serialization.ByteArrayDeserializer
>     group.id =
>     partition.assignment.strategy =
> [org.apache.kafka.clients.consumer.RangeAssignor]
>     reconnect.backoff.ms = 50
>     sasl.kerberos.ticket.renew.window.factor = 0.8
>     max.partition.fetch.bytes = 1048576
>     bootstrap.servers = [node01:6667]
>     retry.backoff.ms = 100
>     sasl.kerberos.kinit.cmd = /usr/bin/kinit
>     sasl.kerberos.service.name = null
>     sasl.kerberos.ticket.renew.jitter = 0.05
>     ssl.keystore.type = JKS
>     ssl.trustmanager.algorithm = PKIX
>     enable.auto.commit = true
>     ssl.key.password = null
>     fetch.max.wait.ms = 500
>     sasl.kerberos.min.time.before.relogin = 60000
>     connections.max.idle.ms = 540000
>     ssl.truststore.password = null
>     session.timeout.ms = 30000
>     metrics.num.samples = 2
>     client.id =
>     ssl.endpoint.identification.algorithm = null
>     key.deserializer = class
> org.apache.kafka.common.serialization.ByteArrayDeserializer
>     ssl.protocol = TLS
>     check.crcs = true
>     request.timeout.ms = 40000
>     ssl.provider = null
>     ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
>     ssl.keystore.location = null
>     heartbeat.interval.ms = 3000
>     auto.commit.interval.ms = 5000
>     receive.buffer.bytes = 32768
>     ssl.cipher.suites = null
>     ssl.truststore.type = JKS
>     security.protocol = PLAINTEXT
>     ssl.truststore.location = null
>     ssl.keystore.password = null
>     ssl.keymanager.algorithm = SunX509
>     metrics.sample.window.ms = 30000
>     fetch.min.bytes = 1
>     send.buffer.bytes = 131072
>     auto.offset.reset = earliest
>
> 2016-07-28 22:34:24,886 INFO  org.apache.kafka.common.utils.AppInfoParser
> - Kafka version : 0.9.0.1
> 2016-07-28 22:34:24,886 INFO  org.apache.kafka.common.utils.AppInfoParser
> - Kafka commitId : 23c69d62a0cabf06
> 2016-07-28 22:34:39,491 INFO  org.apache.flink.yarn.YarnTaskManagerRunner
> - RECEIVED SIGNAL 15: SIGTERM. Shutting down as requested.
> 2016-07-28 22:34:39,524 INFO  org.apache.flink.runtime.blob.BlobCache
> - Shutting down BlobCache
> 2016-07-28 22:34:39,655 INFO
> org.apache.flink.runtime.io.disk.iomanager.IOManager          - I/O manager
> removed spill file directory
> /mnt/a/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/flink-io-96e84170-f59d-4316-bda1-d2c2810fdcdf
> 2016-07-28 22:34:39,656 INFO
> org.apache.flink.runtime.io.disk.iomanager.IOManager          - I/O manager
> removed spill file directory
> /mnt/b/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/flink-io-996962ff-d621-4285-a4d2-7eb9e846ee11
> 2016-07-28 22:34:39,656 INFO
> org.apache.flink.runtime.io.disk.iomanager.IOManager          - I/O manager
> removed spill file directory
> /mnt/c/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/flink-io-4b2cc5e6-110c-43aa-a1b1-620058e12a94
> 2016-07-28 22:34:39,656 INFO
> org.apache.flink.runtime.io.disk.iomanager.IOManager          - I/O manager
> removed spill file directory
> /mnt/d/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/flink-io-a9c9d249-da5c-4c3c-a208-06603df1e966
> End of LogType:taskmanager.log
>
> LogType:taskmanager.out
> Log Upload Time:Thu Jul 28 22:34:43 +0000 2016
> LogLength:0
> Log Contents:
> End of LogType:taskmanager.out
>
> Container: container_e05_1467433388200_0136_01_000001 on
> node05.mycluster.com_45454
> ============================================================================================
> LogType:jobmanager.err
> Log Upload Time:Thu Jul 28 22:34:43 +0000 2016
> LogLength:0
> Log Contents:
> End of LogType:jobmanager.err
>
> LogType:jobmanager.log
> Log Upload Time:Thu Jul 28 22:34:43 +0000 2016
> LogLength:53981
> Log Contents:
> 2016-07-28 22:33:59,613 INFO  org.apache.flink.yarn.ApplicationMaster
> - YARN daemon runs as yarn setting user to execute Flink
> ApplicationMaster/JobManager to user
> 2016-07-28 22:33:59,631 INFO  org.apache.flink.yarn.ApplicationMaster
> -
> --------------------------------------------------------------------------------
> 2016-07-28 22:33:59,631 INFO  org.apache.flink.yarn.ApplicationMaster
> -  Starting YARN ApplicationMaster/JobManager (Version: 1.0.1, Rev:4afa401,
> Date:31.03.2016 @ 13:40:33 UTC)
> 2016-07-28 22:33:59,631 INFO  org.apache.flink.yarn.ApplicationMaster
> -  Current user: yarn
> 2016-07-28 22:33:59,631 INFO  org.apache.flink.yarn.ApplicationMaster
> -  JVM: Java HotSpot(TM) 64-Bit Server VM - Oracle Corporation -
> 1.8/25.60-b23
> 2016-07-28 22:33:59,631 INFO  org.apache.flink.yarn.ApplicationMaster
> -  Maximum heap size: 377 MiBytes
> 2016-07-28 22:33:59,631 INFO  org.apache.flink.yarn.ApplicationMaster
> -  JAVA_HOME: /usr/jdk64/jdk1.8.0_60
> 2016-07-28 22:33:59,632 INFO  org.apache.flink.yarn.ApplicationMaster
> -  Hadoop version: 2.7.2
> 2016-07-28 22:33:59,632 INFO  org.apache.flink.yarn.ApplicationMaster
> -  JVM Options:
> 2016-07-28 22:33:59,633 INFO  org.apache.flink.yarn.ApplicationMaster
> -     -Xmx424M
> 2016-07-28 22:33:59,633 INFO  org.apache.flink.yarn.ApplicationMaster
> -
> -Dlog.file=/mnt/d/hadoop/yarn/log/application_1467433388200_0136/container_e05_1467433388200_0136_01_000001/jobmanager.log
> 2016-07-28 22:33:59,633 INFO  org.apache.flink.yarn.ApplicationMaster
> -     -Dlogback.configurationFile=file:logback.xml
> 2016-07-28 22:33:59,633 INFO  org.apache.flink.yarn.ApplicationMaster
> -     -Dlog4j.configuration=file:log4j.properties
> 2016-07-28 22:33:59,633 INFO  org.apache.flink.yarn.ApplicationMaster
> -  Program Arguments: (none)
> 2016-07-28 22:33:59,633 INFO  org.apache.flink.yarn.ApplicationMaster
> -  Classpath:
> /mnt/b/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/container_e05_1467433388200_0136_01_000001/flink.jar:/usr/hdp/current/hadoop-client/conf:/usr/hdp/current/hadoop-client/hadoop-annotations-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-azure.jar:/usr/hdp/current/hadoop-client/hadoop-common-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-nfs-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-aws.jar:/usr/hdp/current/hadoop-client/hadoop-common-tests.jar:/usr/hdp/current/hadoop-client/hadoop-annotations.jar:/usr/hdp/current/hadoop-client/hadoop-auth.jar:/usr/hdp/current/hadoop-client/hadoop-azure-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-nfs.jar:/usr/hdp/current/hadoop-client/hadoop-common-2.7.1.2.3.4.0-3485-tests.jar:/usr/hdp/current/hadoop-client/hadoop-aws-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-common.jar:/usr/hdp/current/hadoop-client/hadoop-auth-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-client/lib/paranamer-2.3.jar:/usr/hdp/current/hadoop-client/lib/commons-httpclient-3.1.jar:/usr/hdp/current/hadoop-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-client/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hdp/current/hadoop-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-client/lib/httpclient-4.2.5.jar:/usr/hdp/current/hadoop-client/lib/api-asn1-api-1.0.0-M20.jar:/usr/hdp/current/hadoop-client/lib/slf4j-log4j12-1.7.10.jar:/usr/hdp/current/hadoop-client/lib/hadoop-lzo-0.6.0.2.3.4.0-3485-test.jar:/usr/hdp/current/hadoop-client/lib/slf4j-api-1.7.10.jar:/usr/hdp/current/hadoop-client/lib/java-xmlbuilder-0.4.jar:/usr/hdp/current/hadoop-client/lib/mockito-all-1.8.5.jar:/usr/hdp/current/hadoop-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-client/lib/ojdbc6.jar:/usr/hdp/current/hadoop-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/jsp-api-2.1.jar:/usr/hdp/current/hadoop-client/lib/activation-1.1.jar:/usr/hdp/current/hadoop-client/lib/jaxb-api-2.2.2.jar:/usr/hdp/current/hadoop-client/lib/commons-compress-1.4.1.jar:/usr/hdp/current/hadoop-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-client/lib/commons-digester-1.8.jar:/usr/hdp/current/hadoop-client/lib/azure-storage-2.2.0.jar:/usr/hdp/current/hadoop-client/lib/commons-net-3.1.jar:/usr/hdp/current/hadoop-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/commons-configuration-1.6.jar:/usr/hdp/current/hadoop-client/lib/ranger-plugin-classloader-0.5.0.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-client/lib/hamcrest-core-1.3.jar:/usr/hdp/current/hadoop-client/lib/httpcore-4.2.5.jar:/usr/hdp/current/hadoop-client/lib/commons-collections-3.2.2.jar:/usr/hdp/current/hadoop-client/lib/jets3t-0.9.0.jar:/usr/hdp/current/hadoop-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-client/lib/jackson-annotations-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/hadoop-lzo-0.6.0.2.3.4.0-3485-sources.jar:/usr/hdp/current/hadoop-client/lib/curator-client-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/ranger-hdfs-plugin-shim-0.5.0.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/avro-1.7.4.jar:/usr/hdp/current/hadoop-client/lib/jackson-xc-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/junit-4.11.jar:/usr/hdp/current/hadoop-client/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/current/hadoop-client/lib/jackson-core-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/ranger-yarn-plugin-shim-0.5.0.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-client/lib/jettison-1.1.jar:/usr/hdp/current/hadoop-client/lib/stax-api-1.0-2.jar:/usr/hdp/current/hadoop-client/lib/curator-framework-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/current/hadoop-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-client/lib/zookeeper-3.4.6.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/jsch-0.1.42.jar:/usr/hdp/current/hadoop-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-client/lib/jersey-json-1.9.jar:/usr/hdp/current/hadoop-client/lib/commons-beanutils-1.7.0.jar:/usr/hdp/current/hadoop-client/lib/hadoop-lzo-0.6.0.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/jackson-databind-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/api-util-1.0.0-M20.jar:/usr/hdp/current/hadoop-client/lib/xz-1.0.jar:/usr/hdp/current/hadoop-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-client/lib/curator-recipes-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/commons-math3-3.1.1.jar:/usr/hdp/current/hadoop-client/lib/snappy-java-1.0.4.1.jar:/usr/hdp/current/hadoop-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-client/lib/microsoft-windowsazure-storage-sdk-0.6.0.jar:/usr/hdp/current/hadoop-client/lib/commons-beanutils-core-1.8.0.jar:/usr/hdp/current/hadoop-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-1.7.4.jar:/usr/hdp/current/hadoop-client/lib/gson-2.2.4.jar:/usr/hdp/current/hadoop-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-2.7.1.2.3.4.0-3485-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-nfs.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-nfs-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-hdfs-client/lib/netty-all-4.0.23.Final.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-hdfs-client/lib/leveldbjni-all-1.8.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/okhttp-2.4.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-daemon-1.0.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xercesImpl-2.9.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-hdfs-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-hdfs-client/lib/okio-1.4.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xml-apis-1.3.04.jar:/usr/hdp/current/hadoop-hdfs-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-client.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-api-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-sharedcachemanager-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-registry.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-common.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-timeline-plugins.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-timeline-plugins-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-applicationhistoryservice.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-client-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-resourcemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-nodemanager-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-web-proxy.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-common-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-registry-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-web-proxy-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-unmanaged-am-launcher-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-common-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-sharedcachemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-tests-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-applicationhistoryservice-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-tests.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-resourcemanager-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-common.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-nodemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-api.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/paranamer-2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/guice-3.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-httpclient-3.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-yarn-client/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/httpclient-4.2.5.jar:/usr/hdp/current/hadoop-yarn-client/lib/api-asn1-api-1.0.0-M20.jar:/usr/hdp/current/hadoop-yarn-client/lib/objenesis-2.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/java-xmlbuilder-0.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsp-api-2.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/activation-1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-client-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/jaxb-api-2.2.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-compress-1.4.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-yarn-client/lib/javassist-3.18.1-GA.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-digester-1.8.jar:/usr/hdp/current/hadoop-yarn-client/lib/fst-2.24.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-net-3.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-configuration-1.6.jar:/usr/hdp/current/hadoop-yarn-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-yarn-client/lib/httpcore-4.2.5.jar:/usr/hdp/current/hadoop-yarn-client/lib/zookeeper-3.4.6.2.3.4.0-3485-tests.jar:/usr/hdp/current/hadoop-yarn-client/lib/guice-servlet-3.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-collections-3.2.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/leveldbjni-all-1.8.jar:/usr/hdp/current/hadoop-yarn-client/lib/jets3t-0.9.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-annotations-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/javax.inject-1.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-client-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/avro-1.7.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-xc-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-core-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-yarn-client/lib/jettison-1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/stax-api-1.0-2.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-framework-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/current/hadoop-yarn-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/zookeeper-3.4.6.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsch-0.1.42.jar:/usr/hdp/current/hadoop-yarn-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-json-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-beanutils-1.7.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-databind-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/api-util-1.0.0-M20.jar:/usr/hdp/current/hadoop-yarn-client/lib/xz-1.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-recipes-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-math3-3.1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/snappy-java-1.0.4.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-guice-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-yarn-client/lib/microsoft-windowsazure-storage-sdk-0.6.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-beanutils-core-1.8.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/aopalliance-1.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/gson-2.2.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-logging-1.1.3.jar
> 2016-07-28 22:33:59,634 INFO  org.apache.flink.yarn.ApplicationMaster
> -
> --------------------------------------------------------------------------------
> 2016-07-28 22:33:59,635 INFO  org.apache.flink.yarn.ApplicationMaster
> - Registered UNIX signal handlers for [TERM, HUP, INT]
> 2016-07-28 22:33:59,649 INFO  org.apache.flink.yarn.ApplicationMaster
> - Loading config from:
> /mnt/b/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/container_e05_1467433388200_0136_01_000001.
> 2016-07-28 22:33:59,720 INFO  org.apache.flink.runtime.jobmanager.JobManager
> - Starting JobManager
> 2016-07-28 22:33:59,722 INFO  org.apache.flink.runtime.jobmanager.JobManager
> - Starting JobManager actor system at 38.111.30.148:59857
> 2016-07-28 22:34:00,262 INFO  akka.event.slf4j.Slf4jLogger
> - Slf4jLogger started
> 2016-07-28 22:34:00,326 INFO  Remoting
> - Starting remoting
> 2016-07-28 22:34:00,534 INFO  Remoting
> - Remoting started; listening on addresses
> :[akka.tcp://flink@38.111.30.148:59857]
> 2016-07-28 22:34:00,541 INFO  org.apache.flink.runtime.jobmanager.JobManager
> - Starting JobManger web frontend
> 2016-07-28 22:34:00,545 INFO  org.apache.flink.runtime.util.ZooKeeperUtils
> - Using '/flink' as root namespace.
> 2016-07-28 22:34:00,674 INFO
> org.apache.flink.shaded.org.apache.curator.framework.imps.CuratorFrameworkImpl
> - Starting
> 2016-07-28 22:34:00,684 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014
> 09:09 GMT
> 2016-07-28 22:34:00,684 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:host.name=node05.mycluster.com
> 2016-07-28 22:34:00,684 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:java.version=1.8.0_60
> 2016-07-28 22:34:00,684 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:java.vendor=Oracle Corporation
> 2016-07-28 22:34:00,684 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:java.home=/usr/jdk64/jdk1.8.0_60/jre
> 2016-07-28 22:34:00,685 INFO  org.apache.zookeeper.ZooKeeper
> - Client
> environment:java.class.path=/mnt/b/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/container_e05_1467433388200_0136_01_000001/flink.jar:/usr/hdp/current/hadoop-client/conf:/usr/hdp/current/hadoop-client/hadoop-annotations-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-azure.jar:/usr/hdp/current/hadoop-client/hadoop-common-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-nfs-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-aws.jar:/usr/hdp/current/hadoop-client/hadoop-common-tests.jar:/usr/hdp/current/hadoop-client/hadoop-annotations.jar:/usr/hdp/current/hadoop-client/hadoop-auth.jar:/usr/hdp/current/hadoop-client/hadoop-azure-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-nfs.jar:/usr/hdp/current/hadoop-client/hadoop-common-2.7.1.2.3.4.0-3485-tests.jar:/usr/hdp/current/hadoop-client/hadoop-aws-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/hadoop-common.jar:/usr/hdp/current/hadoop-client/hadoop-auth-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-client/lib/paranamer-2.3.jar:/usr/hdp/current/hadoop-client/lib/commons-httpclient-3.1.jar:/usr/hdp/current/hadoop-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-client/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hdp/current/hadoop-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-client/lib/httpclient-4.2.5.jar:/usr/hdp/current/hadoop-client/lib/api-asn1-api-1.0.0-M20.jar:/usr/hdp/current/hadoop-client/lib/slf4j-log4j12-1.7.10.jar:/usr/hdp/current/hadoop-client/lib/hadoop-lzo-0.6.0.2.3.4.0-3485-test.jar:/usr/hdp/current/hadoop-client/lib/slf4j-api-1.7.10.jar:/usr/hdp/current/hadoop-client/lib/java-xmlbuilder-0.4.jar:/usr/hdp/current/hadoop-client/lib/mockito-all-1.8.5.jar:/usr/hdp/current/hadoop-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-client/lib/ojdbc6.jar:/usr/hdp/current/hadoop-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/jsp-api-2.1.jar:/usr/hdp/current/hadoop-client/lib/activation-1.1.jar:/usr/hdp/current/hadoop-client/lib/jaxb-api-2.2.2.jar:/usr/hdp/current/hadoop-client/lib/commons-compress-1.4.1.jar:/usr/hdp/current/hadoop-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-client/lib/commons-digester-1.8.jar:/usr/hdp/current/hadoop-client/lib/azure-storage-2.2.0.jar:/usr/hdp/current/hadoop-client/lib/commons-net-3.1.jar:/usr/hdp/current/hadoop-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/commons-configuration-1.6.jar:/usr/hdp/current/hadoop-client/lib/ranger-plugin-classloader-0.5.0.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-client/lib/hamcrest-core-1.3.jar:/usr/hdp/current/hadoop-client/lib/httpcore-4.2.5.jar:/usr/hdp/current/hadoop-client/lib/commons-collections-3.2.2.jar:/usr/hdp/current/hadoop-client/lib/jets3t-0.9.0.jar:/usr/hdp/current/hadoop-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-client/lib/jackson-annotations-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/hadoop-lzo-0.6.0.2.3.4.0-3485-sources.jar:/usr/hdp/current/hadoop-client/lib/curator-client-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/ranger-hdfs-plugin-shim-0.5.0.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/avro-1.7.4.jar:/usr/hdp/current/hadoop-client/lib/jackson-xc-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/junit-4.11.jar:/usr/hdp/current/hadoop-client/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/current/hadoop-client/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/current/hadoop-client/lib/jackson-core-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/ranger-yarn-plugin-shim-0.5.0.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-client/lib/jettison-1.1.jar:/usr/hdp/current/hadoop-client/lib/stax-api-1.0-2.jar:/usr/hdp/current/hadoop-client/lib/curator-framework-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/current/hadoop-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-client/lib/zookeeper-3.4.6.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/jsch-0.1.42.jar:/usr/hdp/current/hadoop-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-client/lib/jersey-json-1.9.jar:/usr/hdp/current/hadoop-client/lib/commons-beanutils-1.7.0.jar:/usr/hdp/current/hadoop-client/lib/hadoop-lzo-0.6.0.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-client/lib/jackson-databind-2.2.3.jar:/usr/hdp/current/hadoop-client/lib/api-util-1.0.0-M20.jar:/usr/hdp/current/hadoop-client/lib/xz-1.0.jar:/usr/hdp/current/hadoop-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-client/lib/curator-recipes-2.7.1.jar:/usr/hdp/current/hadoop-client/lib/commons-math3-3.1.1.jar:/usr/hdp/current/hadoop-client/lib/snappy-java-1.0.4.1.jar:/usr/hdp/current/hadoop-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-client/lib/microsoft-windowsazure-storage-sdk-0.6.0.jar:/usr/hdp/current/hadoop-client/lib/commons-beanutils-core-1.8.0.jar:/usr/hdp/current/hadoop-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-1.7.4.jar:/usr/hdp/current/hadoop-client/lib/gson-2.2.4.jar:/usr/hdp/current/hadoop-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-2.7.1.2.3.4.0-3485-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-nfs.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-tests.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs-nfs-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-hdfs-client/hadoop-hdfs.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-hdfs-client/lib/netty-all-4.0.23.Final.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-hdfs-client/lib/leveldbjni-all-1.8.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/okhttp-2.4.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-daemon-1.0.13.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xercesImpl-2.9.1.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-hdfs-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-hdfs-client/lib/okio-1.4.0.jar:/usr/hdp/current/hadoop-hdfs-client/lib/xml-apis-1.3.04.jar:/usr/hdp/current/hadoop-hdfs-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-hdfs-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-hdfs-client/lib/commons-logging-1.1.3.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-client.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-api-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-sharedcachemanager-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-registry.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-common.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-timeline-plugins.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-timeline-plugins-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-applicationhistoryservice.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-client-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-resourcemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-nodemanager-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-web-proxy.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-common-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-registry-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-web-proxy-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-unmanaged-am-launcher-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-common-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-sharedcachemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-tests-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-applicationhistoryservice-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-tests.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-resourcemanager-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell-2.7.1.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-common.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-server-nodemanager.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-api.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-io-2.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/paranamer-2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/guice-3.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-httpclient-3.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/xmlenc-0.52.jar:/usr/hdp/current/hadoop-yarn-client/lib/apacheds-i18n-2.0.0-M15.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-server-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/httpclient-4.2.5.jar:/usr/hdp/current/hadoop-yarn-client/lib/api-asn1-api-1.0.0-M20.jar:/usr/hdp/current/hadoop-yarn-client/lib/objenesis-2.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/java-xmlbuilder-0.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/jetty-6.1.26.hwx.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsr305-3.0.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-codec-1.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-mapper-asl-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsp-api-2.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/activation-1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-client-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/jaxb-api-2.2.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-compress-1.4.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/asm-3.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/servlet-api-2.5.jar:/usr/hdp/current/hadoop-yarn-client/lib/javassist-3.18.1-GA.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-digester-1.8.jar:/usr/hdp/current/hadoop-yarn-client/lib/fst-2.24.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-net-3.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-core-asl-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-configuration-1.6.jar:/usr/hdp/current/hadoop-yarn-client/lib/netty-3.6.2.Final.jar:/usr/hdp/current/hadoop-yarn-client/lib/httpcore-4.2.5.jar:/usr/hdp/current/hadoop-yarn-client/lib/zookeeper-3.4.6.2.3.4.0-3485-tests.jar:/usr/hdp/current/hadoop-yarn-client/lib/guice-servlet-3.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-collections-3.2.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/leveldbjni-all-1.8.jar:/usr/hdp/current/hadoop-yarn-client/lib/jets3t-0.9.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-cli-1.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-annotations-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/javax.inject-1.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-client-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/avro-1.7.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-xc-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-jaxrs-1.9.13.jar:/usr/hdp/current/hadoop-yarn-client/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-core-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-lang-2.6.jar:/usr/hdp/current/hadoop-yarn-client/lib/jettison-1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/stax-api-1.0-2.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-framework-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jaxb-impl-2.2.3-1.jar:/usr/hdp/current/hadoop-yarn-client/lib/protobuf-java-2.5.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/zookeeper-3.4.6.2.3.4.0-3485.jar:/usr/hdp/current/hadoop-yarn-client/lib/jsch-0.1.42.jar:/usr/hdp/current/hadoop-yarn-client/lib/log4j-1.2.17.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-json-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-beanutils-1.7.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jackson-databind-2.2.3.jar:/usr/hdp/current/hadoop-yarn-client/lib/api-util-1.0.0-M20.jar:/usr/hdp/current/hadoop-yarn-client/lib/xz-1.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/guava-11.0.2.jar:/usr/hdp/current/hadoop-yarn-client/lib/curator-recipes-2.7.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-math3-3.1.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/snappy-java-1.0.4.1.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-guice-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/jetty-util-6.1.26.hwx.jar:/usr/hdp/current/hadoop-yarn-client/lib/microsoft-windowsazure-storage-sdk-0.6.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-beanutils-core-1.8.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/jersey-core-1.9.jar:/usr/hdp/current/hadoop-yarn-client/lib/aopalliance-1.0.jar:/usr/hdp/current/hadoop-yarn-client/lib/gson-2.2.4.jar:/usr/hdp/current/hadoop-yarn-client/lib/commons-logging-1.1.3.jar
> 2016-07-28 22:34:00,685 INFO  org.apache.zookeeper.ZooKeeper
> - Client
> environment:java.library.path=::/usr/hdp/2.3.4.0-3485/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.3.4.0-3485/hadoop/lib/native::/usr/hdp/2.3.4.0-3485/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.3.4.0-3485/hadoop/lib/native:/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir:/usr/hdp/2.3.4.0-3485/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.3.4.0-3485/hadoop/lib/native:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 2016-07-28 22:34:00,685 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:java.io.tmpdir=/tmp
> 2016-07-28 22:34:00,685 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:java.compiler=<NA>
> 2016-07-28 22:34:00,685 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:os.name=Linux
> 2016-07-28 22:34:00,685 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:os.arch=amd64
> 2016-07-28 22:34:00,686 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:os.version=3.13.0-32-generic
> 2016-07-28 22:34:00,686 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:user.name=yarn
> 2016-07-28 22:34:00,686 INFO  org.apache.zookeeper.ZooKeeper
> - Client environment:user.home=/home/yarn
> 2016-07-28 22:34:00,686 INFO  org.apache.zookeeper.ZooKeeper
> - Client
> environment:user.dir=/mnt/b/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/container_e05_1467433388200_0136_01_000001
> 2016-07-28 22:34:00,687 INFO  org.apache.zookeeper.ZooKeeper
> - Initiating client connection,
> connectString=node01:2181,node02:2181,node03:2181 sessionTimeout=60000
> watcher=org.apache.flink.shaded.org.apache.curator.ConnectionState@3a1dd365
> 2016-07-28 22:34:00,714 INFO  org.apache.zookeeper.ClientCnxn
> - Opening socket connection to server
> node01.mycluster.com/38.111.30.144:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2016-07-28 22:34:00,720 INFO  org.apache.zookeeper.ClientCnxn
> - Socket connection established to node01.mycluster.com/38.111.30.144:2181,
> initiating session
> 2016-07-28 22:34:00,739 INFO  org.apache.zookeeper.ClientCnxn
> - Session establishment complete on server
> node01.mycluster.com/38.111.30.144:2181, sessionid = 0x15624f916ae08d2,
> negotiated timeout = 40000
> 2016-07-28 22:34:00,744 INFO
> org.apache.flink.shaded.org.apache.curator.framework.state.ConnectionStateManager
> - State change: CONNECTED
> 2016-07-28 22:34:01,773 INFO
> org.apache.flink.runtime.webmonitor.WebMonitorUtils           - Determined
> location of JobManager log file:
> /mnt/d/hadoop/yarn/log/application_1467433388200_0136/container_e05_1467433388200_0136_01_000001/jobmanager.log
> 2016-07-28 22:34:01,773 INFO
> org.apache.flink.runtime.webmonitor.WebMonitorUtils           - Determined
> location of JobManager stdout file:
> /mnt/d/hadoop/yarn/log/application_1467433388200_0136/container_e05_1467433388200_0136_01_000001/jobmanager.out
> 2016-07-28 22:34:01,802 INFO
> org.apache.flink.runtime.webmonitor.WebRuntimeMonitor         - Using
> directory /tmp/flink-web-fbe7069f-8e03-4535-b8d9-71db4f51b298 for the web
> interface files
> 2016-07-28 22:34:01,812 INFO
> org.apache.flink.runtime.webmonitor.WebRuntimeMonitor         - Using
> directory /tmp/flink-web-upload-1fecf32c-3b86-4d95-ae7b-25def122c3d4 for web
> frontend JAR file uploads
> 2016-07-28 22:34:02,113 INFO
> org.apache.flink.runtime.webmonitor.WebRuntimeMonitor         - Web frontend
> listening at 0:0:0:0:0:0:0:0:48590
> 2016-07-28 22:34:02,114 INFO  org.apache.flink.runtime.jobmanager.JobManager
> - Starting JobManager actor
> 2016-07-28 22:34:02,125 INFO  org.apache.flink.runtime.blob.BlobServer
> - Created BLOB server storage directory
> /tmp/blobStore-de21570f-ca07-45a0-ae82-af4041390d86
> 2016-07-28 22:34:03,079 INFO
> org.apache.flink.runtime.blob.FileSystemBlobStore             - Created blob
> directory hdfs:///flink/recovery/blob.
> 2016-07-28 22:34:03,080 INFO  org.apache.flink.runtime.blob.BlobServer
> - Started BLOB server at 0.0.0.0:46326 - max concurrent requests: 50 - max
> backlog: 1000
> 2016-07-28 22:34:03,092 INFO  org.apache.flink.runtime.util.ZooKeeperUtils
> - Using '/flink' as root namespace.
> 2016-07-28 22:34:03,094 INFO
> org.apache.flink.shaded.org.apache.curator.framework.imps.CuratorFrameworkImpl
> - Starting
> 2016-07-28 22:34:03,095 INFO  org.apache.zookeeper.ZooKeeper
> - Initiating client connection,
> connectString=node01:2181,node02:2181,node03:2181 sessionTimeout=60000
> watcher=org.apache.flink.shaded.org.apache.curator.ConnectionState@14fc5d40
> 2016-07-28 22:34:03,097 INFO  org.apache.zookeeper.ClientCnxn
> - Opening socket connection to server
> node03.mycluster.com/38.111.30.146:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2016-07-28 22:34:03,099 INFO  org.apache.zookeeper.ClientCnxn
> - Socket connection established to node03.mycluster.com/38.111.30.146:2181,
> initiating session
> 2016-07-28 22:34:03,119 INFO  org.apache.zookeeper.ClientCnxn
> - Session establishment complete on server
> node03.mycluster.com/38.111.30.146:2181, sessionid = 0x3556f328b2602ff,
> negotiated timeout = 40000
> 2016-07-28 22:34:03,119 INFO
> org.apache.flink.shaded.org.apache.curator.framework.state.ConnectionStateManager
> - State change: CONNECTED
> 2016-07-28 22:34:03,147 INFO
> org.apache.flink.runtime.checkpoint.SavepointStoreFactory     - Using
> filesystem savepoint backend (root path: hdfs:///flink/savepoints).
> 2016-07-28 22:34:03,158 INFO
> org.apache.flink.runtime.jobmanager.MemoryArchivist           - Started
> memory archivist akka://flink/user/archive
> 2016-07-28 22:34:03,161 INFO  org.apache.flink.yarn.YarnJobManager
> - Starting JobManager at
> akka.tcp://flink@38.111.30.148:59857/user/jobmanager.
> 2016-07-28 22:34:03,161 INFO
> org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService  -
> Starting ZooKeeperLeaderElectionService.
> 2016-07-28 22:34:03,166 INFO
> org.apache.flink.runtime.webmonitor.WebRuntimeMonitor         - Starting
> with JobManager akka.tcp://flink@38.111.30.148:59857/user/jobmanager on port
> 48590
> 2016-07-28 22:34:03,167 INFO
> org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService  -
> Starting ZooKeeperLeaderRetrievalService.
> 2016-07-28 22:34:03,176 INFO  org.apache.flink.yarn.ApplicationMaster
> - Generate configuration file for application master.
> 2016-07-28 22:34:03,206 INFO  org.apache.flink.yarn.ApplicationMaster
> - Starting YARN session on Job Manager.
> 2016-07-28 22:34:03,206 INFO  org.apache.flink.yarn.ApplicationMaster
> - Application Master properly initiated. Awaiting termination of actor
> system.
> 2016-07-28 22:34:03,214 INFO  org.apache.flink.yarn.YarnJobManager
> - Start yarn session.
> 2016-07-28 22:34:03,273 INFO  org.apache.flink.yarn.YarnJobManager
> - Yarn session with 1 TaskManagers. Tolerating 1 failed TaskManagers
> 2016-07-28 22:34:03,302 INFO  org.apache.hadoop.yarn.client.RMProxy
> - Connecting to ResourceManager at node02.mycluster.com/38.111.30.145:8030
> 2016-07-28 22:34:03,324 INFO
> org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy  -
> yarn.client.max-cached-nodemanagers-proxies : 0
> 2016-07-28 22:34:03,325 INFO  org.apache.flink.yarn.YarnJobManager
> - Registering ApplicationMaster with tracking url
> http://node05.mycluster.com:48590.
> 2016-07-28 22:34:03,771 INFO  org.apache.flink.yarn.YarnJobManager
> - Retrieved 0 TaskManagers from previous attempts.
> 2016-07-28 22:34:03,776 INFO  org.apache.flink.yarn.YarnJobManager
> - Requesting initial TaskManager container 0.
> 2016-07-28 22:34:03,811 INFO  org.apache.flink.yarn.Utils
> - Copying from
> file:/mnt/b/hadoop/yarn/local/usercache/user/appcache/application_1467433388200_0136/container_e05_1467433388200_0136_01_000001/flink-conf-modified.yaml
> to
> hdfs://node01.mycluster.com:8020/user/user/.flink/application_1467433388200_0136/flink-conf-modified.yaml
> 2016-07-28 22:34:04,443 INFO  org.apache.flink.yarn.YarnJobManager
> - Prepared local resource for modified yaml: resource { scheme: "hdfs" host:
> "node01.mycluster.com" port: 8020 file:
> "/user/user/.flink/application_1467433388200_0136/flink-conf-modified.yaml"
> } size: 5593 timestamp: 1469745244302 type: FILE visibility: APPLICATION
> 2016-07-28 22:34:04,449 INFO  org.apache.flink.yarn.YarnJobManager
> - Create container launch context.
> 2016-07-28 22:34:04,459 INFO  org.apache.flink.yarn.YarnJobManager
> - Starting TM with command=$JAVA_HOME/bin/java -Xms1448m -Xmx1448m
> -XX:MaxDirectMemorySize=1448m  -Dlog.file="<LOG_DIR>/taskmanager.log"
> -Dlogback.configurationFile=file:logback.xml
> -Dlog4j.configuration=file:log4j.properties
> org.apache.flink.yarn.YarnTaskManagerRunner --configDir . 1>
> <LOG_DIR>/taskmanager.out 2> <LOG_DIR>/taskmanager.err
> 2016-07-28 22:34:04,463 INFO  org.apache.flink.yarn.YarnJobManager
> - JobManager akka.tcp://flink@38.111.30.148:59857/user/jobmanager was
> granted leadership with leader session ID
> Some(3d2a8629-081e-44bc-8c38-446b510d5909).
> 2016-07-28 22:34:04,504 INFO  org.apache.flink.yarn.YarnJobManager
> - Delaying recovery of all jobs by 10000 milliseconds.
> 2016-07-28 22:34:04,511 INFO
> org.apache.flink.runtime.webmonitor.JobManagerRetriever       - New leader
> reachable under
> akka.tcp://flink@38.111.30.148:59857/user/jobmanager:3d2a8629-081e-44bc-8c38-446b510d5909.
> 2016-07-28 22:34:04,878 INFO
> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Received new
> token for : node03.mycluster.com:45454
> 2016-07-28 22:34:04,900 INFO  org.apache.flink.yarn.YarnJobManager
> - Got new container for allocation:
> container_e05_1467433388200_0136_01_000002
> 2016-07-28 22:34:04,900 INFO  org.apache.flink.yarn.YarnJobManager
> - The user requested 1 containers, 0 running. 1 containers missing
> 2016-07-28 22:34:04,901 INFO  org.apache.flink.yarn.YarnJobManager
> - 1 containers already allocated by YARN. Starting...
> 2016-07-28 22:34:04,902 INFO
> org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy  -
> Opening proxy : node03.mycluster.com:45454
> 2016-07-28 22:34:05,149 INFO  org.apache.flink.yarn.YarnJobManager
> - Launching container (container_e05_1467433388200_0136_01_000002 on host
> node03.mycluster.com).
> 2016-07-28 22:34:07,375 INFO  org.apache.flink.yarn.YarnJobManager
> - Register akka.tcp://flink@38.111.30.144:45229/user/applicationClient as
> client.
> 2016-07-28 22:34:14,524 INFO  org.apache.flink.yarn.YarnJobManager
> - Attempting to recover all jobs.
> 2016-07-28 22:34:14,527 INFO
> org.apache.flink.runtime.jobmanager.ZooKeeperSubmittedJobGraphStore  - No
> job graph to recover.
> 2016-07-28 22:34:14,528 INFO  org.apache.flink.yarn.YarnJobManager
> - Re-submitting 0 job graphs.
> 2016-07-28 22:34:16,168 INFO
> org.apache.flink.runtime.instance.InstanceManager             - Registered
> TaskManager at node03
> (akka.tcp://flink@38.111.30.146:48700/user/taskmanager) as
> 1d52e723ce440d6181b6e0e98627bfd2. Current number of registered hosts is 1.
> Current number of alive task slots is 1.
> 2016-07-28 22:34:23,098 INFO  org.apache.flink.yarn.YarnJobManager
> - Submitting job 9cdd920b36bec849689a72026402a333 (Flink Streaming Job).
> 2016-07-28 22:34:23,100 INFO  org.apache.flink.yarn.YarnJobManager
> - Using restart strategy NoRestartStrategy for
> 9cdd920b36bec849689a72026402a333.
> 2016-07-28 22:34:23,387 INFO
> org.apache.flink.runtime.jobmanager.ZooKeeperSubmittedJobGraphStore  - Added
> SubmittedJobGraph(JobGraph(jobId: 9cdd920b36bec849689a72026402a333),
> JobInfo(client: Actor[akka.tcp://flink@127.0.0.1:38212/user/$a#-2098822004]
> (EXECUTION_RESULT_AND_STATE_CHANGES), start: 1469745263098)) to ZooKeeper.
> 2016-07-28 22:34:23,395 INFO  org.apache.flink.yarn.YarnJobManager
> - Scheduling job 9cdd920b36bec849689a72026402a333 (Flink Streaming Job).
> 2016-07-28 22:34:23,397 INFO  org.apache.flink.yarn.YarnJobManager
> - Status of job 9cdd920b36bec849689a72026402a333 (Flink Streaming Job)
> changed to RUNNING.
> 2016-07-28 22:34:23,400 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> Custom Source -> Sink: Unnamed (1/1) (fcae1e4dbfa221cff020dd8140e0b282)
> switched from CREATED to SCHEDULED
> 2016-07-28 22:34:23,423 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> Custom Source -> Sink: Unnamed (1/1) (fcae1e4dbfa221cff020dd8140e0b282)
> switched from SCHEDULED to DEPLOYING
> 2016-07-28 22:34:23,424 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
> Source: Custom Source -> Sink: Unnamed (1/1) (attempt #0) to node03
> 2016-07-28 22:34:24,554 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> Custom Source -> Sink: Unnamed (1/1) (fcae1e4dbfa221cff020dd8140e0b282)
> switched from DEPLOYING to RUNNING
> 2016-07-28 22:34:40,021 WARN  akka.remote.ReliableDeliverySupervisor
> - Association with remote system [akka.tcp://flink@38.111.30.146:48700] has
> failed, address is now gated for [5000] ms. Reason is: [Disassociated].
> 2016-07-28 22:34:40,397 INFO  org.apache.flink.yarn.YarnJobManager
> - Container container_e05_1467433388200_0136_01_000002 is completed with
> diagnostics: Container
> [pid=5832,containerID=container_e05_1467433388200_0136_01_000002] is running
> beyond physical memory limits. Current usage: 2.3 GB of 2 GB physical memory
> used; 6.1 GB of 4.2 GB virtual memory used. Killing container.
> Dump of the process-tree for container_e05_1467433388200_0136_01_000002 :
>     |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
> SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
>     |- 5838 5832 5832 5832 (java) 2817 481 6553391104 592779
> /usr/jdk64/jdk1.8.0_60/bin/java -Xms1448m -Xmx1448m
> -XX:MaxDirectMemorySize=1448m
> -Dlog.file=/mnt/a/hadoop/yarn/log/application_1467433388200_0136/container_e05_1467433388200_0136_01_000002/taskmanager.log
> -Dlogback.configurationFile=file:logback.xml
> -Dlog4j.configuration=file:log4j.properties
> org.apache.flink.yarn.YarnTaskManagerRunner --configDir .
>     |- 5832 5830 5832 5832 (bash) 0 0 12759040 357 /bin/bash -c
> /usr/jdk64/jdk1.8.0_60/bin/java -Xms1448m -Xmx1448m
> -XX:MaxDirectMemorySize=1448m
> -Dlog.file=/mnt/a/hadoop/yarn/log/application_1467433388200_0136/container_e05_1467433388200_0136_01_000002/taskmanager.log
> -Dlogback.configurationFile=file:logback.xml
> -Dlog4j.configuration=file:log4j.properties
> org.apache.flink.yarn.YarnTaskManagerRunner --configDir . 1>
> /mnt/a/hadoop/yarn/log/application_1467433388200_0136/container_e05_1467433388200_0136_01_000002/taskmanager.out
> 2>
> /mnt/a/hadoop/yarn/log/application_1467433388200_0136/container_e05_1467433388200_0136_01_000002/taskmanager.err
>
> Container killed on request. Exit code is 143
> Container exited with a non-zero exit code 143
>
> 2016-07-28 22:34:40,400 INFO  org.apache.flink.yarn.YarnJobManager
> - Container container_e05_1467433388200_0136_01_000002 was a running
> container. Total failed containers 1.
> 2016-07-28 22:34:40,404 ERROR org.apache.flink.yarn.YarnJobManager
> - Stopping YARN session because the number of failed containers (1) exceeded
> the maximum failed container count (1). This number is controlled by the
> 'yarn.maximum-failed-containers' configuration setting. By default its the
> number of requested containers
> 2016-07-28 22:34:40,405 INFO  org.apache.flink.yarn.YarnJobManager
> - The user requested 1 containers, 0 running. 1 containers missing
> 2016-07-28 22:34:40,406 INFO  org.apache.flink.yarn.YarnJobManager
> - There are 1 containers missing. 0 are already requested. Requesting 1
> additional container(s) from YARN. Reallocation of failed containers is
> enabled=true ('yarn.reallocate-failed')
> 2016-07-28 22:34:40,408 INFO  org.apache.flink.yarn.YarnJobManager
> - Requested additional container from YARN. Pending requests 1.
> 2016-07-28 22:34:40,410 INFO  org.apache.flink.yarn.YarnJobManager
> - Stopping YARN JobManager with status FAILED and diagnostic Stopping YARN
> session because the number of failed containers (1) exceeded the maximum
> failed container count (1). This number is controlled by the
> 'yarn.maximum-failed-containers' configuration setting. By default its the
> number of requested containers.
> 2016-07-28 22:34:40,419 INFO
> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Waiting for
> application to be successfully unregistered.
> 2016-07-28 22:34:40,525 INFO
> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl  -
> Interrupted while waiting for queue
> java.lang.InterruptedException
>     at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
>     at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
>     at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
>     at
> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:287)
> 2016-07-28 22:34:40,528 INFO
> org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy  -
> Opening proxy : node03.mycluster.com:45454
> 2016-07-28 22:34:40,647 INFO  org.apache.flink.yarn.YarnJobManager
> - Stopping JobManager akka.tcp://flink@38.111.30.148:59857/user/jobmanager.
> 2016-07-28 22:34:40,883 INFO
> org.apache.flink.runtime.jobmanager.ZooKeeperSubmittedJobGraphStore  -
> Removed job graph 9cdd920b36bec849689a72026402a333 from ZooKeeper.
> 2016-07-28 22:34:40,911 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> Custom Source -> Sink: Unnamed (1/1) (fcae1e4dbfa221cff020dd8140e0b282)
> switched from RUNNING to CANCELING
> 2016-07-28 22:34:40,923 INFO
> org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService  -
> Stopping ZooKeeperLeaderElectionService.
> 2016-07-28 22:34:40,955 INFO  org.apache.zookeeper.ClientCnxn
> - EventThread shut down
> 2016-07-28 22:34:40,955 INFO  org.apache.zookeeper.ZooKeeper
> - Session: 0x3556f328b2602ff closed
> 2016-07-28 22:34:40,961 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> Custom Source -> Sink: Unnamed (1/1) (fcae1e4dbfa221cff020dd8140e0b282)
> switched from CANCELING to CANCELED
> 2016-07-28 22:34:41,145 INFO  org.apache.flink.runtime.blob.BlobServer
> - Stopped BLOB server at 0.0.0.0:46326
> 2016-07-28 22:34:41,152 INFO
> akka.remote.RemoteActorRefProvider$RemotingTerminator         - Shutting
> down remote daemon.
> 2016-07-28 22:34:41,153 INFO
> akka.remote.RemoteActorRefProvider$RemotingTerminator         - Remote
> daemon shut down; proceeding with flushing remote transports.
> 2016-07-28 22:34:41,185 INFO
> akka.remote.RemoteActorRefProvider$RemotingTerminator         - Remoting
> shut down.
> 2016-07-28 22:34:41,199 INFO
> org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService  -
> Stopping ZooKeeperLeaderRetrievalService.
> 2016-07-28 22:34:41,202 INFO  org.apache.flink.yarn.YarnJobManager
> - Shutdown completed. Stopping JVM.
> 2016-07-28 22:34:41,208 INFO  org.apache.zookeeper.ZooKeeper
> - Session: 0x15624f916ae08d2 closed
> 2016-07-28 22:34:41,208 INFO  org.apache.zookeeper.ClientCnxn
> - EventThread shut down
> 2016-07-28 22:34:41,209 INFO
> org.apache.flink.runtime.webmonitor.WebRuntimeMonitor         - Removing web
> dashboard root cache directory
> /tmp/flink-web-fbe7069f-8e03-4535-b8d9-71db4f51b298
> 2016-07-28 22:34:41,209 INFO
> org.apache.flink.runtime.webmonitor.WebRuntimeMonitor         - Removing web
> dashboard jar upload directory
> /tmp/flink-web-upload-1fecf32c-3b86-4d95-ae7b-25def122c3d4
> 2016-07-28 22:34:41,217 INFO
> org.apache.flink.runtime.webmonitor.StackTraceSampleCoordinator  - Shutting
> down stack trace sample coordinator.
> End of LogType:jobmanager.log
>
> LogType:jobmanager.out
> Log Upload Time:Thu Jul 28 22:34:43 +0000 2016
> LogLength:0
> Log Contents:
> End of LogType:jobmanager.out
>
>
>
>
> On Fri, Jul 29, 2016 at 11:19 AM, Jack Huang <jackhuang@mz.com> wrote:
>>
>> Hi Max,
>>
>> Each events are only a few hundred bytes. I am reading from a Kafka topic
>> from some offset in the past, so the events should be flowing in as fast as
>> Flink can process them.
>>
>> The entire YARN task log, which contains both JobManager and TaskManager
>> outputs, is attached.
>>
>> Thanks a lot,
>> Jack
>>
>>
>> On Fri, Jul 29, 2016 at 2:04 AM, Maximilian Michels <mxm@apache.org>
>> wrote:
>>>
>>> Hi Jack,
>>>
>>> Considering the type of job you're running, you shouldn't run out of
>>> memory. Could it be that the events are quite large strings? It could
>>> be that the TextOutputFormat doesn't write to disk fast enough and
>>> accumulates memory. Actually, it doesn't perform regular flushing
>>> which could be an issue.
>>>
>>> I'm just guessing, we need to investigate further. Could you please
>>> supply the entire JobManager log file output?
>>>
>>> Thanks,
>>> Max
>>>
>>> On Fri, Jul 29, 2016 at 12:59 AM, Jack Huang <jackhuang@mz.com> wrote:
>>> > Hi all,
>>> >
>>> > I am running a test Flink streaming task under YARN. It reads messages
>>> > from
>>> > a Kafka topic and writes them to local file system.
>>> >
>>> > object PricerEvent {
>>> >     def main(args:Array[String]) {
>>> >         val kafkaProp = new Properties()
>>> >         kafkaProp.setProperty("bootstrap.servers", "localhost:6667")
>>> >         kafkaProp.setProperty("auto.offset.reset", "earliest")
>>> >
>>> >         val env = StreamExecutionEnvironment.getExecutionEnvironment
>>> >         env.setStateBackend(new MemoryStateBackend)
>>> >
>>> >         val wins = env.addSource(new
>>> > FlinkKafkaConsumer09[String]("events",
>>> > new SimpleStringSchema, kafkaProp))
>>> >         wins.writeAsText("/home/user/flink_out/" + new
>>> > SimpleDateFormat("yyyy-MM-dd_HH-mm-ss").format(new Date))
>>> >
>>> >         env.execute
>>> >     }
>>> > }
>>> >
>>> > With the command
>>> >
>>> > flink run -m yarn-cluster -yn 1 -ytm 2048 -c PricerEvent
>>> > /home/user/flink-example/build/libs/flink-example-1.0-all.jar
>>> >
>>> >
>>> > The task runs fine for a moment and then terminates. I looked into the
>>> > error
>>> > log and found following out-of-memory error message:
>>> >
>>> > 2016-07-28 22:34:40,397 INFO  org.apache.flink.yarn.YarnJobManager
>>> > - Container container_e05_1467433388200_0136_01_000002 is completed
>>> > with
>>> > diagnostics: Container
>>> > [pid=5832,containerID=container_e05_1467433388200_0136_01_000002] is
>>> > running
>>> > beyond physical memory limits. Current usage: 2.3 GB of 2 GB physical
>>> > memory
>>> > used; 6.1 GB of 4.2 GB virtual memory used. Killing container.
>>> > Dump of the process-tree for container_e05_1467433388200_0136_01_000002
>>> > :
>>> >     |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
>>> > SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
>>> >     |- 5838 5832 5832 5832 (java) 2817 481 6553391104 592779
>>> > /usr/jdk64/jdk1.8.0_60/bin/java -Xms1448m -Xmx1448m
>>> > -XX:MaxDirectMemorySize=1448m
>>> >
>>> > -Dlog.file=/mnt/a/hadoop/yarn/log/application_1467433388200_0136/container_e05_1467433388200_0136_01_000002/taskmanager.log
>>> > -Dlogback.configurationFile=file:logback.xml
>>> > -Dlog4j.configuration=file:log4j.properties
>>> > org.apache.flink.yarn.YarnTaskManagerRunner --configDir .
>>> >     |- 5832 5830 5832 5832 (bash) 0 0 12759040 357 /bin/bash -c
>>> > /usr/jdk64/jdk1.8.0_60/bin/java -Xms1448m -Xmx1448m
>>> > -XX:MaxDirectMemorySize=1448m
>>> >
>>> > -Dlog.file=/mnt/a/hadoop/yarn/log/application_1467433388200_0136/container_e05_1467433388200_0136_01_000002/taskmanager.log
>>> > -Dlogback.configurationFile=file:logback.xml
>>> > -Dlog4j.configuration=file:log4j.properties
>>> > org.apache.flink.yarn.YarnTaskManagerRunner --configDir . 1>
>>> >
>>> > /mnt/a/hadoop/yarn/log/application_1467433388200_0136/container_e05_1467433388200_0136_01_000002/taskmanager.out
>>> > 2>
>>> >
>>> > /mnt/a/hadoop/yarn/log/application_1467433388200_0136/container_e05_1467433388200_0136_01_000002/taskmanager.err
>>> >
>>> > Container killed on request. Exit code is 143
>>> > Container exited with a non-zero exit code 143
>>> >
>>> >
>>> > I don't understand how it should fail due to out-of-memory error. I
>>> > would
>>> > expect the task to slow down if there are too many messages to process,
>>> > but
>>> > not failing altogether. I am not storing any states either.
>>> >
>>> > Does anyone know the reason and the way to fix/avoid this issue?
>>> >
>>> >
>>> > Thanks,
>>> > Jack
>>> >
>>> >
>>> >
>>> >
>>
>>
>

Mime
View raw message