flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <trohrm...@apache.org>
Subject Re: OutofMemoryError: Java heap space & Loss of Taskmanager
Date Tue, 15 Mar 2016 15:48:18 GMT
Have you checked the logs?

Cheers,
Till

On Tue, Mar 15, 2016 at 4:42 PM, Ravinder Kaur <neetu0404@gmail.com> wrote:

> Hello,
>
> Can anyone suggest the cause for the TaskManagers not starting up? I have
> checked all the cluster configurations and nothing has changed nor wrong.
> I'm unable to find out why the taskmanagers do not start up. The startup
> script also runs normally saying that all TM processes were started up. But
> there is no TM process running on worker nodes.
>
> Only when cluster is stopped there is info that "No taskmanager daemon is
> running anymore on worker node"
>
> Kind Regards,
> Ravinder
>
>
>
> On Tue, Mar 15, 2016 at 12:35 PM, Ravinder Kaur <neetu0404@gmail.com>
> wrote:
>
>> Update:
>>
>> I restarted the cluster by assinging 50GB memmory to taskmanager.heap.mb
>> and now 3 of the TMs that previously did not get stopped now do not start
>> up.
>>
>> Kind Regards,
>> Ravinder
>>
>>
>> On Tue, Mar 15, 2016 at 12:32 PM, Ravinder Kaur <neetu0404@gmail.com>
>> wrote:
>>
>>> Hi Till,
>>>
>>> I had stopped the cluster using bin/stop-cluster.sh after my jobs got
>>> cancelled. After this I checked all TM machines if the processes were
>>> killed but on 4 machines it was still running. I ran the shell script again
>>> but it said No taskmanager is running on the machines. Then I had to
>>> manually kill the TaskManager processes on the each of the 4 machines
>>> explicitly.
>>>
>>> I have now stumbled upon the correct log files. Sorry I had not noticed
>>> this before. This is the log file of one of the Taskmanagers that got
>>> disassociated from the JM.
>>>
>>> 08:51:26,916 WARN  org.apache.hadoop.util.NativeCodeLoader
>>>         - Unable to load native-hadoop library for your platform... using
>>> builtin-java classes where applicable
>>> 08:51:27,331 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -
>>> --------------------------------------------------------------------------------
>>> 08:51:27,331 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -  Starting TaskManager (Version: 0.10.1, Rev:2e9b231,
>>> Date:22.11.2015 @ 12:41:12 CET)
>>> 08:51:27,332 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -  Current user: flink
>>> 08:51:27,332 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -  JVM: OpenJDK 64-Bit Server VM - Oracle Corporation -
>>> 1.7/24.91-b01
>>> 08:51:27,332 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -  Maximum heap size: 990 MiBytes
>>> 08:51:27,332 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -  JAVA_HOME: /usr/lib/jvm/java-1.7.0-openjdk-amd64
>>> 08:51:27,337 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -  Hadoop version: 2.7.0
>>> 08:51:27,341 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -  JVM Options:
>>> 08:51:27,341 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -     -XX:+UseConcMarkSweepGC
>>> 08:51:27,341 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -     -XX:+CMSClassUnloadingEnabled
>>> 08:51:27,342 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -     -Xms1024M
>>> 08:51:27,342 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -     -Xmx1024M
>>> 08:51:27,342 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -     -XX:MaxDirectMemorySize=8388607T
>>> 08:51:27,342 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -     -XX:MaxPermSize=256m
>>> 08:51:27,342 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -
>>> -Dlog.file=/home/flink/flink-0.10.1/log/flink-flink-taskmanager-0-vm-10-155-208-137.cloud.mwn.de.log
>>> 08:51:27,342 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -
>>> -Dlog4j.configuration=file:/home/flink/flink-0.10.1/conf/log4j.properties
>>> 08:51:27,342 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -
>>> -Dlogback.configurationFile=file:/home/flink/flink-0.10.1/conf/logback.xml
>>> 08:51:27,342 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -  Program Arguments:
>>> 08:51:27,342 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -     --configDir
>>> 08:51:27,342 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -     /home/flink/flink-0.10.1/conf
>>> 08:51:27,342 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -     --streamingMode
>>> 08:51:27,342 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -     streaming
>>> 08:51:27,342 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -  Classpath:
>>> /home/flink/flink-0.10.1/lib/flink-dist_2.11-0.10.1.jar:/home/flink/flink-0.10.1/lib/flink-python_2.11-0.10.1.jar:/home/flink/flink-0.10.1/lib/log4j-1.2.17.jar:/home/flink/flink-0.10.1/lib/slf4j-log4j12-1.7.7.jar:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar::
>>> 08:51:27,342 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          -
>>> --------------------------------------------------------------------------------
>>> 08:51:27,349 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Maximum number of open file descriptors is 4096
>>> 08:51:27,378 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Loading configuration from /home/flink/flink-0.10.1/conf
>>> 08:51:27,457 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Security is not enabled. Starting non-authenticated TaskManager.
>>> 08:51:27,573 INFO  org.apache.flink.runtime.util.LeaderRetrievalUtils
>>>          - Trying to select the network interface and address to use by
>>> connecting to the leading JobManager.
>>> 08:51:27,574 INFO  org.apache.flink.runtime.util.LeaderRetrievalUtils
>>>          - TaskManager will try to connect for 10000 milliseconds before
>>> falling back to heuristics
>>> 08:51:27,577 INFO  org.apache.flink.runtime.net.ConnectionUtils
>>>          - Retrieved new target address /10.155.208.156:6123.
>>> 08:51:27,606 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - TaskManager will use hostname/address '
>>> vm-10-155-208-137.cloud.mwn.de' (10.155.208.137) for communication.
>>> 08:51:27,607 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Starting TaskManager in streaming mode STREAMING
>>> 08:51:27,607 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Starting TaskManager actor system at 10.155.208.137:0
>>> 08:51:28,499 INFO  akka.event.slf4j.Slf4jLogger
>>>          - Slf4jLogger started
>>> 08:51:28,587 INFO  Remoting
>>>          - Starting remoting
>>> 08:51:29,079 INFO  Remoting
>>>          - Remoting started; listening on addresses :[akka.tcp://
>>> flink@10.155.208.137:41096]
>>> 08:51:29,084 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Starting TaskManager actor
>>> 08:51:29,114 INFO  org.apache.flink.runtime.io.network.netty.NettyConfig
>>>         - NettyConfig [server address:
>>> vm-10-155-208-137.cloud.mwn.de/10.155.208.137, server port: 42988,
>>> memory segment size (bytes): 32768, transport type: NIO, number of server
>>> threads: 0 (use Netty's default), number of client threads: 0 (use Netty's
>>> default), server connect backlog: 0 (use Netty's default), client connect
>>> timeout (sec): 120, send/receive buffer size (bytes): 0 (use Netty's
>>> default)]
>>> 08:51:29,118 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Messages between TaskManager and JobManager have a max timeout
>>> of 100000 milliseconds
>>> 08:51:29,150 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Temporary file directory '/tmp': total 4 GB, usable 1 GB (25.00%
>>> usable)
>>> 08:51:29,191 INFO
>>>  org.apache.flink.runtime.io.network.buffer.NetworkBufferPool  - Allocated
>>> 64 MB for network buffer pool (number of memory segments: 2048, bytes per
>>> segment: 32768).
>>> 08:51:29,286 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Using 0.5 of the currently free heap space for Flink managed
>>> heap memory (455 MB).
>>> 08:51:29,433 INFO  org.apache.flink.runtime.io.disk.iomanager.IOManager
>>>          - I/O manager uses directory
>>> /tmp/flink-io-6892942e-0f4a-4564-87b3-eab3406022c0 for spill files.
>>> 08:51:29,446 INFO  org.apache.flink.runtime.filecache.FileCache
>>>          - User file cache uses directory
>>> /tmp/flink-dist-cache-2b1b9ee0-714e-455c-b405-011b8b5a2bdc
>>> 08:51:29,836 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Starting TaskManager actor at
>>> akka://flink/user/taskmanager#-215796751.
>>> 08:51:29,836 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - TaskManager data connection information:
>>> vm-10-155-208-137.cloud.mwn.de (dataPort=42988)
>>> 08:51:29,836 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - TaskManager has 4 task slot(s).
>>> 08:51:29,838 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Memory usage stats: [HEAP: 102/990/990 MB, NON HEAP: 24/37/304
>>> MB (used/committed/max)]
>>> 08:51:29,850 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Trying to register at JobManager akka.tcp://
>>> flink@10.155.208.156:6123/user/jobmanager (attempt 1, timeout: 500
>>> milliseconds)
>>> 08:51:30,042 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Successful registration at JobManager (akka.tcp://
>>> flink@10.155.208.156:6123/user/jobmanager), starting network stack and
>>> library cache.
>>> 08:51:30,682 INFO  org.apache.flink.runtime.io.network.netty.NettyClient
>>>         - Successful initialization (took 136 ms).
>>> 08:51:30,763 INFO  org.apache.flink.runtime.io.network.netty.NettyServer
>>>         - Successful initialization (took 81 ms). Listening on
>>> SocketAddress /10.155.208.137:42988.
>>> 08:51:30,764 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Determined BLOB server address to be /10.155.208.156:36914.
>>> Starting BLOB cache.
>>> 08:51:30,770 INFO  org.apache.flink.runtime.blob.BlobCache
>>>         - Created BLOB cache storage directory
>>> /tmp/blobStore-f40e39a7-f4ef-49e4-b082-a07a69f8b5c5
>>> 08:52:33,190 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Received task Source: Read Text File Source -> Flat Map (6/25)
>>> 08:52:33,191 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Loading JAR files for task Source: Read Text File Source -> Flat
>>> Map (6/25)
>>> 08:52:33,192 INFO  org.apache.flink.runtime.blob.BlobCache
>>>         - Downloading 4c0984b81f71e4a8d15b9162415af4c237f7b1be from /
>>> 10.155.208.156:36914
>>> 08:52:33,195 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Received task Source: Read Text File Source -> Flat Map (13/25)
>>> 08:52:33,195 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Loading JAR files for task Source: Read Text File Source -> Flat
>>> Map (13/25)
>>> 08:52:33,198 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Received task Source: Read Text File Source -> Flat Map (20/25)
>>> 08:52:33,198 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Loading JAR files for task Source: Read Text File Source -> Flat
>>> Map (20/25)
>>> 08:52:33,200 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Received task Source: Read Text File Source -> Flat Map (24/25)
>>> 08:52:33,200 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Loading JAR files for task Source: Read Text File Source -> Flat
>>> Map (24/25)
>>> 08:52:33,261 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Registering task at network: Source: Read Text File Source ->
>>> Flat Map (6/25) [DEPLOYING]
>>> 08:52:33,261 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Registering task at network: Source: Read Text File Source ->
>>> Flat Map (24/25) [DEPLOYING]
>>> 08:52:33,261 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Registering task at network: Source: Read Text File Source ->
>>> Flat Map (13/25) [DEPLOYING]
>>> 08:52:33,261 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Registering task at network: Source: Read Text File Source ->
>>> Flat Map (20/25) [DEPLOYING]
>>> 08:52:33,275 WARN  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - No state backend has been specified, using default state backend
>>> (Memory / JobManager)
>>> 08:52:33,276 INFO  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - State backend is set to heap memory (checkpoint to jobmanager)
>>> 08:52:33,276 WARN  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - No state backend has been specified, using default state backend
>>> (Memory / JobManager)
>>> 08:52:33,277 INFO  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - State backend is set to heap memory (checkpoint to jobmanager)
>>> 08:52:33,276 WARN  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - No state backend has been specified, using default state backend
>>> (Memory / JobManager)
>>> 08:52:33,277 INFO  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - State backend is set to heap memory (checkpoint to jobmanager)
>>> 08:52:33,276 WARN  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - No state backend has been specified, using default state backend
>>> (Memory / JobManager)
>>> 08:52:33,277 INFO  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - State backend is set to heap memory (checkpoint to jobmanager)
>>> 08:52:33,502 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Source: Read Text File Source -> Flat Map (24/25) switched to
>>> RUNNING
>>> 08:52:33,502 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Source: Read Text File Source -> Flat Map (20/25) switched to
>>> RUNNING
>>> 08:52:33,505 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Source: Read Text File Source -> Flat Map (13/25) switched to
>>> RUNNING
>>> 08:52:33,505 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Source: Read Text File Source -> Flat Map (6/25) switched to
>>> RUNNING
>>> 08:52:34,118 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Received task Keyed Aggregation -> Sink: Unnamed (24/25)
>>> 08:52:34,121 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Loading JAR files for task Keyed Aggregation -> Sink: Unnamed
>>> (24/25)
>>> 08:52:34,149 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Registering task at network: Keyed Aggregation -> Sink: Unnamed
>>> (24/25) [DEPLOYING]
>>> 08:52:34,150 WARN  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - No state backend has been specified, using default state backend
>>> (Memory / JobManager)
>>> 08:52:34,150 INFO  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - State backend is set to heap memory (checkpoint to jobmanager)
>>> 08:52:34,166 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Received task Keyed Aggregation -> Sink: Unnamed (21/25)
>>> 08:52:34,186 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Loading JAR files for task Keyed Aggregation -> Sink: Unnamed
>>> (21/25)
>>> 08:52:34,186 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Registering task at network: Keyed Aggregation -> Sink: Unnamed
>>> (21/25) [DEPLOYING]
>>> 08:52:34,208 WARN  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - No state backend has been specified, using default state backend
>>> (Memory / JobManager)
>>> 08:52:34,226 INFO  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - State backend is set to heap memory (checkpoint to jobmanager)
>>> 08:52:34,239 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Received task Keyed Aggregation -> Sink: Unnamed (23/25)
>>> 08:52:34,250 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Loading JAR files for task Keyed Aggregation -> Sink: Unnamed
>>> (23/25)
>>> 08:52:34,272 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Registering task at network: Keyed Aggregation -> Sink: Unnamed
>>> (23/25) [DEPLOYING]
>>> 08:52:34,150 WARN  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - No state backend has been specified, using default state backend
>>> (Memory / JobManager)
>>> 08:52:34,150 INFO  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - State backend is set to heap memory (checkpoint to jobmanager)
>>> 08:52:34,166 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Received task Keyed Aggregation -> Sink: Unnamed (21/25)
>>> 08:52:34,186 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Loading JAR files for task Keyed Aggregation -> Sink: Unnamed
>>> (21/25)
>>> 08:52:34,186 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Registering task at network: Keyed Aggregation -> Sink: Unnamed
>>> (21/25) [DEPLOYING]
>>> 08:52:34,208 WARN  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - No state backend has been specified, using default state backend
>>> (Memory / JobManager)
>>> 08:52:34,226 INFO  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - State backend is set to heap memory (checkpoint to jobmanager)
>>> 08:52:34,239 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Received task Keyed Aggregation -> Sink: Unnamed (23/25)
>>> 08:52:34,250 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Loading JAR files for task Keyed Aggregation -> Sink: Unnamed
>>> (23/25)
>>> 08:52:34,272 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Registering task at network: Keyed Aggregation -> Sink: Unnamed
>>> (23/25) [DEPLOYING]
>>> 08:52:34,272 WARN  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - No state backend has been specified, using default state backend
>>> (Memory / JobManager)
>>> 08:52:34,272 INFO  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - State backend is set to heap memory (checkpoint to jobmanager)
>>> 08:52:34,366 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>          - Received task Keyed Aggregation -> Sink: Unnamed (25/25)
>>> 08:52:34,374 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Loading JAR files for task Keyed Aggregation -> Sink: Unnamed
>>> (25/25)
>>> 08:52:34,389 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Registering task at network: Keyed Aggregation -> Sink: Unnamed
>>> (25/25) [DEPLOYING]
>>> 08:52:34,389 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Keyed Aggregation -> Sink: Unnamed (24/25) switched to RUNNING
>>> 08:52:34,389 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Keyed Aggregation -> Sink: Unnamed (21/25) switched to RUNNING
>>> 08:52:34,389 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Keyed Aggregation -> Sink: Unnamed (23/25) switched to RUNNING
>>> 08:52:34,423 WARN  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - No state backend has been specified, using default state backend
>>> (Memory / JobManager)
>>> 08:52:34,424 INFO  org.apache.flink.streaming.runtime.tasks.StreamTask
>>>         - State backend is set to heap memory (checkpoint to jobmanager)
>>> 08:52:34,451 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Keyed Aggregation -> Sink: Unnamed (25/25) switched to RUNNING
>>> 09:13:01,822 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Keyed Aggregation -> Sink: Unnamed (24/25) switched to FAILED
>>> with exception.
>>> 09:22:56,768 ERROR org.apache.flink.runtime.taskmanager.Task
>>>         - FATAL - exception in task exception handler
>>> 09:23:27,729 INFO  org.apache.flink.runtime.taskmanager.Task
>>>         - Freeing task resources for Keyed Aggregation -> Sink: Unnamed
>>> (24/25)
>>>
>>> Log of JobManager during Job Execution
>>>
>>> 08:51:11,911 WARN  org.apache.hadoop.util.NativeCodeLoader
>>>         - Unable to load native-hadoop library for your platform... using
>>> builtin-java classes where applicable
>>> 08:51:12,228 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -
>>> --------------------------------------------------------------------------------
>>> 08:51:12,228 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -  Starting JobManager (Version: 0.10.1, Rev:2e9b231,
>>> Date:22.11.2015 @ 12:41:12 CET)
>>> 08:51:12,228 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -  Current user: flink
>>> 08:51:12,228 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -  JVM: OpenJDK 64-Bit Server VM - Oracle Corporation -
>>> 1.7/24.95-b01
>>> 08:51:12,228 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -  Maximum heap size: 246 MiBytes
>>> 08:51:12,228 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -  JAVA_HOME: /usr/lib/jvm/java-1.7.0-openjdk-amd64
>>> 08:51:12,241 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -  Hadoop version: 2.7.0
>>> 08:51:12,241 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -  JVM Options:
>>> 08:51:12,241 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -     -Xms256m
>>> 08:51:12,241 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -     -Xmx256m
>>> 08:51:12,241 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -     -XX:MaxPermSize=256m
>>> 08:51:12,241 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -
>>> -Dlog.file=/home/flink/flink-0.10.1/log/flink-flink-jobmanager-0-vm-10-155-208-156.cloud.mwn.de.log
>>> 08:51:12,241 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -
>>> -Dlog4j.configuration=file:/home/flink/flink-0.10.1/conf/log4j.properties
>>> 08:51:12,241 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -
>>> -Dlogback.configurationFile=file:/home/flink/flink-0.10.1/conf/logback.xml
>>> 08:51:12,241 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -  Program Arguments:
>>> 08:51:12,241 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -     --configDir
>>> 08:51:12,241 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -     /home/flink/flink-0.10.1/conf
>>> 08:51:12,242 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -     --executionMode
>>> 08:51:12,242 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -     cluster
>>> 08:51:12,242 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -     --streamingMode
>>> 08:51:12,242 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -     streaming
>>> 08:51:12,242 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -  Classpath:
>>> /home/flink/flink-0.10.1/lib/flink-dist_2.11-0.10.1.jar:/home/flink/flink-0.10.1/lib/flink-python_2.11-0.10.1.jar:/home/flink/flink-0.10.1/lib/log4j-1.2.17.jar:/home/flink/flink-0.10.1/lib/slf4j-log4j12-1.7.7.jar:::
>>> 08:51:12,242 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          -
>>> --------------------------------------------------------------------------------
>>> 08:51:12,430 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Loading configuration from /home/flink/flink-0.10.1/conf
>>> 08:51:12,451 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Staring JobManager without high-availability
>>> 08:51:12,464 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Starting JobManager on 10.155.208.156:6123 with execution
>>> mode CLUSTER and streaming mode STREAMING
>>> 08:51:12,574 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Security is not enabled. Starting non-authenticated JobManager.
>>> 08:51:12,575 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Starting JobManager
>>> 08:51:12,575 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Starting JobManager actor system at 10.155.208.156:6123
>>> 08:51:13,253 INFO  akka.event.slf4j.Slf4jLogger
>>>          - Slf4jLogger started
>>> 08:51:13,362 INFO  Remoting
>>>          - Starting remoting
>>> 08:51:13,668 INFO  Remoting
>>>          - Remoting started; listening on addresses :[akka.tcp://
>>> flink@10.155.208.156:6123]
>>> 08:51:13,681 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Starting JobManger web frontend
>>> 08:51:13,735 INFO  org.apache.flink.runtime.webmonitor.WebRuntimeMonitor
>>>         - Using directory
>>> /tmp/flink-web-e5a124e6-2c92-4f19-803c-7e2f4a9e398d for the web interface
>>> files
>>> 08:51:13,736 INFO  org.apache.flink.runtime.webmonitor.WebRuntimeMonitor
>>>         - Serving job manager log from
>>> /home/flink/flink-0.10.1/log/flink-flink-jobmanager-0-vm-10-155-208-156.cloud.mwn.de.log
>>> 08:51:13,737 INFO  org.apache.flink.runtime.webmonitor.WebRuntimeMonitor
>>>         - Serving job manager stdout from
>>> /home/flink/flink-0.10.1/log/flink-flink-jobmanager-0-vm-10-155-208-156.cloud.mwn.de.out
>>> 08:51:14,169 INFO  org.apache.flink.runtime.webmonitor.WebRuntimeMonitor
>>>         - Web frontend listening at 0:0:0:0:0:0:0:0:8081
>>> 08:51:14,170 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Starting JobManager actor
>>> 08:51:14,178 INFO  org.apache.flink.runtime.blob.BlobServer
>>>          - Created BLOB server storage directory
>>> /tmp/blobStore-a2f6d78f-c650-430d-8cd2-dde7d65b8463
>>> 08:51:14,180 INFO  org.apache.flink.runtime.blob.BlobServer
>>>          - Started BLOB server at 0.0.0.0:36914 - max concurrent
>>> requests: 50 - max backlog: 1000
>>> 08:51:14,227 INFO  org.apache.flink.runtime.webmonitor.WebRuntimeMonitor
>>>         - Starting with JobManager akka.tcp://
>>> flink@10.155.208.156:6123/user/jobmanager on port 8081
>>> 08:51:14,227 INFO
>>>  org.apache.flink.runtime.webmonitor.JobManagerRetriever       - New leader
>>> reachable under akka.tcp://
>>> flink@10.155.208.156:6123/user/jobmanager:null.
>>> 08:51:14,230 INFO  org.apache.flink.runtime.jobmanager.MemoryArchivist
>>>         - Started memory archivist akka://flink/user/archive
>>> 08:51:14,232 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Starting JobManager at akka.tcp://
>>> flink@10.155.208.156:6123/user/jobmanager.
>>> 08:51:14,238 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - JobManager akka.tcp://
>>> flink@10.155.208.156:6123/user/jobmanager was granted leadership with
>>> leader session ID None.
>>> 08:51:15,659 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>         - Registered TaskManager at vm-10-155-208-156 (akka.tcp://
>>> flink@10.155.208.156:33728/user/taskmanager) as
>>> 749bdc28313ae125f292377f3bc46925. Current number of registered hosts is 1.
>>> Current number of alive task slots is 3.
>>> 08:51:17,670 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>         - Registered TaskManager at vm-10-155-208-157 (akka.tcp://
>>> flink@10.155.208.157:33728/user/taskmanager) as
>>> 62f658c5f1e690b1dd6c2f0610553916. Current number of registered hosts is 2.
>>> Current number of alive task slots is 6.
>>> 08:51:18,513 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>         - Registered TaskManager at vm-10-155-208-158 (akka.tcp://
>>> flink@10.155.208.158:33728/user/taskmanager) as
>>> e5e5d4a5a015b0a4636e5a6c1d16be84. Current number of registered hosts is 3.
>>> Current number of alive task slots is 9.
>>> 08:51:23,684 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>         - Registered TaskManager at slave3 (akka.tcp://
>>> flink@10.155.208.135:54516/user/taskmanager) as
>>> fbd10bca3c9f82460b4b75bd66810aee. Current number of registered hosts is 4.
>>> Current number of alive task slots is 13.
>>> 08:51:29,639 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>         - Registered TaskManager at slave2 (akka.tcp://
>>> flink@10.155.208.136:43058/user/taskmanager) as
>>> c4230ed259af95525dc316040048e7fd. Current number of registered hosts is 5.
>>> Current number of alive task slots is 17.
>>> 08:51:30,029 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>         - Registered TaskManager at vm-10-155-208-137 (akka.tcp://
>>> flink@10.155.208.137:41096/user/taskmanager) as
>>> 8ebde78fbee2d2fb7d94f1fd71378431. Current number of registered hosts is 6.
>>> Current number of alive task slots is 21.
>>> 08:51:40,700 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>         - Registered TaskManager at vm-10-155-208-138 (akka.tcp://
>>> flink@10.155.208.138:42222/user/taskmanager) as
>>> 831a72dad6fbb533b193820f45bdc5bc. Current number of registered hosts is 7.
>>> Current number of alive task slots is 25.
>>> 08:52:33,050 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Submitting job aeb1eca3a4feefa6fd00b599c0c783d4 (WordCount from
>>> SocketTextStream Example).
>>> 08:52:33,117 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Scheduling job aeb1eca3a4feefa6fd00b599c0c783d4 (WordCount from
>>> SocketTextStream Example).
>>> 08:52:33,119 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Status of job aeb1eca3a4feefa6fd00b599c0c783d4 (WordCount from
>>> SocketTextStream Example) changed to RUNNING.
>>> 08:52:33,122 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (1/25) (bce7a3459fd80414266a6ac891b04551)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,130 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (1/25) (bce7a3459fd80414266a6ac891b04551)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,131 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (1/25) (attempt #0) to
>>> vm-10-155-208-156
>>> 08:52:33,134 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (2/25) (b67073687787b804d628eb6482fa2a30)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,136 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (2/25) (b67073687787b804d628eb6482fa2a30)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,137 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (2/25) (attempt #0) to
>>> vm-10-155-208-157
>>> 08:52:33,142 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (3/25) (07a2adef7d57cdd5081dd88533bb6631)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,143 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (3/25) (07a2adef7d57cdd5081dd88533bb6631)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,143 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (3/25) (attempt #0) to
>>> vm-10-155-208-158
>>> 08:52:33,144 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (4/25) (da342b5a369f9da0deaffac409640210)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,145 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (4/25) (da342b5a369f9da0deaffac409640210)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,145 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (4/25) (attempt #0) to slave3
>>> 08:52:33,151 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (5/25) (0edf387c2b8f7f3e6925192bc7e295ee)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,152 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (5/25) (0edf387c2b8f7f3e6925192bc7e295ee)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,152 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (5/25) (attempt #0) to slave2
>>> 08:52:33,154 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (6/25) (de554cfcbc7f7ecf924fd3bfd1f951ce)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,155 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (6/25) (de554cfcbc7f7ecf924fd3bfd1f951ce)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,155 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (6/25) (attempt #0) to
>>> vm-10-155-208-137
>>> 08:52:33,156 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (7/25) (386211d919b02137f42e4df3fefb9619)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,158 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (7/25) (386211d919b02137f42e4df3fefb9619)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,158 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (7/25) (attempt #0) to
>>> vm-10-155-208-138
>>> 08:52:33,159 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (8/25) (38f8b7ded3af389ccfc3a92f74761c1b)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,160 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (8/25) (38f8b7ded3af389ccfc3a92f74761c1b)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,160 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (8/25) (attempt #0) to
>>> vm-10-155-208-156
>>> 08:52:33,161 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (9/25) (88259e334ed2c8ea47b64f4f8083ae83)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,162 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (9/25) (88259e334ed2c8ea47b64f4f8083ae83)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,162 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (9/25) (attempt #0) to
>>> vm-10-155-208-157
>>> 08:52:33,164 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (10/25)
>>> (5f1e48cb807bfc4f837d0d2587d071e7) switched from CREATED to SCHEDULED
>>> 08:52:33,165 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (10/25)
>>> (5f1e48cb807bfc4f837d0d2587d071e7) switched from SCHEDULED to DEPLOYING
>>> 08:52:33,165 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (10/25) (attempt #0) to
>>> vm-10-155-208-158
>>> 08:52:33,166 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (11/25)
>>> (39e9a6f6c11427bf9024fcf6b88dbd3e) switched from CREATED to SCHEDULED
>>> 08:52:33,167 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (11/25)
>>> (39e9a6f6c11427bf9024fcf6b88dbd3e) switched from SCHEDULED to DEPLOYING
>>> 08:52:33,168 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (11/25) (attempt #0) to slave3
>>> 08:52:33,169 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (12/25)
>>> (f25aa9d229f5df70aafa40fa59d88264) switched from CREATED to SCHEDULED
>>> 08:52:33,170 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (12/25)
>>> (f25aa9d229f5df70aafa40fa59d88264) switched from SCHEDULED to DEPLOYING
>>> 08:52:33,170 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (12/25) (attempt #0) to slave2
>>> 08:52:33,171 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (13/25)
>>> (1e572ad1eab41b5f219d8ddd3d544699) switched from CREATED to SCHEDULED
>>> 08:52:33,172 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (13/25)
>>> (1e572ad1eab41b5f219d8ddd3d544699) switched from SCHEDULED to DEPLOYING
>>> 08:52:33,172 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (13/25) (attempt #0) to
>>> vm-10-155-208-137
>>> 08:52:33,173 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (14/25)
>>> (10a5223d3d716bca1a50853f0efcab02) switched from CREATED to SCHEDULED
>>> 08:52:33,174 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (14/25)
>>> (10a5223d3d716bca1a50853f0efcab02) switched from SCHEDULED to DEPLOYING
>>> 08:52:33,175 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (14/25) (attempt #0) to
>>> vm-10-155-208-138
>>> 08:52:33,176 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (15/25)
>>> (c8b82db126393db46682e036fd6f8527) switched from CREATED to SCHEDULED
>>> 08:52:33,177 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (15/25)
>>> (c8b82db126393db46682e036fd6f8527) switched from SCHEDULED to DEPLOYING
>>> 08:52:33,177 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (15/25) (attempt #0) to
>>> vm-10-155-208-156
>>> 08:52:33,178 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (16/25)
>>> (7339669af58d2f65f8ea69cb7b318773) switched from CREATED to SCHEDULED
>>> 08:52:33,179 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (16/25)
>>> (7339669af58d2f65f8ea69cb7b318773) switched from SCHEDULED to DEPLOYING
>>> 08:52:33,179 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (16/25) (attempt #0) to
>>> vm-10-155-208-157
>>> 08:52:33,181 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (17/25)
>>> (da4388c09a9874e7b7afc979a7af79b8) switched from CREATED to SCHEDULED
>>> 08:52:33,184 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (17/25)
>>> (da4388c09a9874e7b7afc979a7af79b8) switched from SCHEDULED to DEPLOYING
>>> 08:52:33,184 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (17/25) (attempt #0) to
>>> vm-10-155-208-158
>>> 08:52:33,187 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (18/25)
>>> (498d68b92dbb4c8a2a7a1db344fe01a8) switched from CREATED to SCHEDULED
>>> 08:52:33,188 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (18/25)
>>> (498d68b92dbb4c8a2a7a1db344fe01a8) switched from SCHEDULED to DEPLOYING
>>> 08:52:33,189 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (18/25) (attempt #0) to slave3
>>> 08:52:33,193 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (19/25)
>>> (f7fcba6ec0608360f363665afd99fb5d) switched from CREATED to SCHEDULED
>>> 08:52:33,194 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (19/25)
>>> (f7fcba6ec0608360f363665afd99fb5d) switched from SCHEDULED to DEPLOYING
>>> 08:52:33,194 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (19/25) (attempt #0) to slave2
>>> 08:52:33,195 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (20/25)
>>> (fd5b27a36b4f09c2191fbf4c23ab61d1) switched from CREATED to SCHEDULED
>>> 08:52:33,195 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (20/25)
>>> (fd5b27a36b4f09c2191fbf4c23ab61d1) switched from SCHEDULED to DEPLOYING
>>> 08:52:33,196 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (20/25) (attempt #0) to
>>> vm-10-155-208-137
>>> 08:52:33,196 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (21/25)
>>> (3827b667c8e997c9e7e16ad0018cc767) switched from CREATED to SCHEDULED
>>> 08:52:33,200 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (21/25)
>>> (3827b667c8e997c9e7e16ad0018cc767) switched from SCHEDULED to DEPLOYING
>>> 08:52:33,201 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (21/25) (attempt #0) to
>>> vm-10-155-208-138
>>> 08:52:33,202 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (22/25)
>>> (4cbe7fa6850e61d695c3ac2a49995e60) switched from CREATED to SCHEDULED
>>> 08:52:33,202 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (22/25)
>>> (4cbe7fa6850e61d695c3ac2a49995e60) switched from SCHEDULED to DEPLOYING
>>> 08:52:33,203 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (22/25) (attempt #0) to slave3
>>> 08:52:33,203 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (23/25)
>>> (1450b59a7088dd0ae6c6720d31839167) switched from CREATED to SCHEDULED
>>> 08:52:33,204 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (23/25)
>>> (1450b59a7088dd0ae6c6720d31839167) switched from SCHEDULED to DEPLOYING
>>> 08:52:33,204 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (23/25) (attempt #0) to slave2
>>> 08:52:33,205 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (24/25)
>>> (6766fa50924793cec3a4d1234bb62c06) switched from CREATED to SCHEDULED
>>> 08:52:33,206 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (24/25)
>>> (6766fa50924793cec3a4d1234bb62c06) switched from SCHEDULED to DEPLOYING
>>> 08:52:33,206 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (24/25) (attempt #0) to
>>> vm-10-155-208-137
>>> 08:52:33,207 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (25/25)
>>> (2ce8063004c80a4d16104e873f82132e) switched from CREATED to SCHEDULED
>>> 08:52:33,208 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (25/25)
>>> (2ce8063004c80a4d16104e873f82132e) switched from SCHEDULED to DEPLOYING
>>> 08:52:33,208 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Source: Read Text File Source -> Flat Map (25/25) (attempt #0) to
>>> vm-10-155-208-138
>>> 08:52:33,209 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (1/25) (9dee3c4d890f01233d5c95f15a9b3c96)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,210 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (1/25) (9dee3c4d890f01233d5c95f15a9b3c96)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,211 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (1/25) (attempt #0) to vm-10-155-208-157
>>> 08:52:33,245 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (2/25) (fbdd9e6f0a6ab2f5441afa4b171a8db3)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,256 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (2/25) (fbdd9e6f0a6ab2f5441afa4b171a8db3)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,256 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (2/25) (attempt #0) to vm-10-155-208-157
>>> 08:52:33,316 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (3/25) (bbd7f74a8a57288a2e8a5c459418d6b3)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,316 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (3/25) (bbd7f74a8a57288a2e8a5c459418d6b3)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,317 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (3/25) (attempt #0) to vm-10-155-208-157
>>> 08:52:33,333 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (4/25) (6b71db98a7a0f2fa53da59f2f8a21cce)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,334 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (4/25) (6b71db98a7a0f2fa53da59f2f8a21cce)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,335 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (4/25) (attempt #0) to vm-10-155-208-156
>>> 08:52:33,354 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (5/25) (011b449fae3ff4b9769c7d75b1cc34ef)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,359 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (5/25) (011b449fae3ff4b9769c7d75b1cc34ef)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,360 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (5/25) (attempt #0) to vm-10-155-208-156
>>> 08:52:33,380 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (6/25) (61dec00d3e2736d3178d22f7c5c9e499)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,381 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (6/25) (61dec00d3e2736d3178d22f7c5c9e499)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,381 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (6/25) (attempt #0) to vm-10-155-208-156
>>> 08:52:33,400 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (7/25) (fa434906c2e64e78743049e21494ad08)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,401 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (7/25) (fa434906c2e64e78743049e21494ad08)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,401 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (7/25) (attempt #0) to slave2
>>> 08:52:33,414 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (8/25) (504a1e8018bd16bbc1cdc5474bc5a496)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,415 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (8/25) (504a1e8018bd16bbc1cdc5474bc5a496)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,415 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (8/25) (attempt #0) to slave2
>>> 08:52:33,428 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (9/25) (ca432acdde8ac296ee1b7fa169d80260)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,428 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (9/25) (ca432acdde8ac296ee1b7fa169d80260)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,428 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (9/25) (attempt #0) to slave2
>>> 08:52:33,446 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (10/25) (4ec7d078930d8b31e89bc5bf63964029)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,447 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (10/25) (4ec7d078930d8b31e89bc5bf63964029)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,447 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (10/25) (attempt #0) to slave2
>>> 08:52:33,464 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (11/25) (b6d53055b7787e63cda0ccfe6f1cf67d)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,465 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (11/25) (b6d53055b7787e63cda0ccfe6f1cf67d)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,465 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (11/25) (attempt #0) to vm-10-155-208-158
>>> 08:52:33,489 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (12/25) (78fc3cd38db931784d45e8b6a0ecd719)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,502 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> local split to host vm-10-155-208-156
>>> 08:52:33,516 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> local split to host vm-10-155-208-156
>>> 08:52:33,517 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> local split to host vm-10-155-208-156
>>> 08:52:33,517 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (12/25) (78fc3cd38db931784d45e8b6a0ecd719)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,517 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (12/25) (attempt #0) to vm-10-155-208-158
>>> 08:52:33,539 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host vm-10-155-208-138
>>> 08:52:33,540 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (21/25)
>>> (3827b667c8e997c9e7e16ad0018cc767) switched from DEPLOYING to RUNNING
>>> 08:52:33,541 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (7/25) (386211d919b02137f42e4df3fefb9619)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,542 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (14/25)
>>> (10a5223d3d716bca1a50853f0efcab02) switched from DEPLOYING to RUNNING
>>> 08:52:33,546 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (25/25)
>>> (2ce8063004c80a4d16104e873f82132e) switched from DEPLOYING to RUNNING
>>> 08:52:33,579 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (13/25) (3d2f2e956e1f5c3518c222f68b37698a)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,580 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (13/25) (3d2f2e956e1f5c3518c222f68b37698a)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,580 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (13/25) (attempt #0) to vm-10-155-208-158
>>> 08:52:33,590 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host vm-10-155-208-138
>>> 08:52:33,590 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host vm-10-155-208-138
>>> 08:52:33,591 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host vm-10-155-208-138
>>> 08:52:33,592 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (16/25)
>>> (7339669af58d2f65f8ea69cb7b318773) switched from DEPLOYING to RUNNING
>>> 08:52:33,592 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (9/25) (88259e334ed2c8ea47b64f4f8083ae83)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,603 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host vm-10-155-208-157
>>> 08:52:33,603 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host vm-10-155-208-157
>>> 08:52:33,604 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host vm-10-155-208-157
>>> 08:52:33,606 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (3/25) (bbd7f74a8a57288a2e8a5c459418d6b3)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,607 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (1/25) (9dee3c4d890f01233d5c95f15a9b3c96)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,608 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (2/25) (fbdd9e6f0a6ab2f5441afa4b171a8db3)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,618 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host vm-10-155-208-137
>>> 08:52:33,619 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (2/25) (b67073687787b804d628eb6482fa2a30)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,621 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (24/25)
>>> (6766fa50924793cec3a4d1234bb62c06) switched from DEPLOYING to RUNNING
>>> 08:52:33,622 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (13/25)
>>> (1e572ad1eab41b5f219d8ddd3d544699) switched from DEPLOYING to RUNNING
>>> 08:52:33,635 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (6/25) (de554cfcbc7f7ecf924fd3bfd1f951ce)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,637 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (20/25)
>>> (fd5b27a36b4f09c2191fbf4c23ab61d1) switched from DEPLOYING to RUNNING
>>> 08:52:33,645 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host vm-10-155-208-137
>>> 08:52:33,647 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host vm-10-155-208-137
>>> 08:52:33,648 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host vm-10-155-208-137
>>> 08:52:33,651 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (14/25) (c1192e402ab4a018b67f2c74f4286e13)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,652 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (14/25) (c1192e402ab4a018b67f2c74f4286e13)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,652 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (14/25) (attempt #0) to vm-10-155-208-138
>>> 08:52:33,656 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (8/25) (38f8b7ded3af389ccfc3a92f74761c1b)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,657 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (15/25)
>>> (c8b82db126393db46682e036fd6f8527) switched from DEPLOYING to RUNNING
>>> 08:52:33,657 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (1/25) (bce7a3459fd80414266a6ac891b04551)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,667 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host slave2
>>> 08:52:33,667 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host slave2
>>> 08:52:33,668 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (11/25)
>>> (39e9a6f6c11427bf9024fcf6b88dbd3e) switched from DEPLOYING to RUNNING
>>> 08:52:33,668 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host slave2
>>> 08:52:33,668 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host slave2
>>> 08:52:33,669 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (4/25) (da342b5a369f9da0deaffac409640210)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,669 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (22/25)
>>> (4cbe7fa6850e61d695c3ac2a49995e60) switched from DEPLOYING to RUNNING
>>> 08:52:33,676 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (18/25)
>>> (498d68b92dbb4c8a2a7a1db344fe01a8) switched from DEPLOYING to RUNNING
>>> 08:52:33,690 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host slave3
>>> 08:52:33,690 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host slave3
>>> 08:52:33,690 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host slave3
>>> 08:52:33,691 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host slave3
>>> 08:52:33,707 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (4/25) (6b71db98a7a0f2fa53da59f2f8a21cce)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,707 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (5/25) (011b449fae3ff4b9769c7d75b1cc34ef)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,708 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (6/25) (61dec00d3e2736d3178d22f7c5c9e499)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,722 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (15/25) (4666479516b2ffe3c49ad7c027a80ad3)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,723 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (15/25) (4666479516b2ffe3c49ad7c027a80ad3)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,723 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (15/25) (attempt #0) to vm-10-155-208-138
>>> 08:52:33,753 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (16/25) (ac2ef9c1626ccb5bc92893b7f2302727)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,754 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (16/25) (ac2ef9c1626ccb5bc92893b7f2302727)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,754 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (16/25) (attempt #0) to vm-10-155-208-138
>>> 08:52:33,763 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (17/25) (e41bf8f2e7e1034d4aeb993da711994d)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,770 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (17/25) (e41bf8f2e7e1034d4aeb993da711994d)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,770 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (17/25) (attempt #0) to vm-10-155-208-138
>>> 08:52:33,771 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (23/25)
>>> (1450b59a7088dd0ae6c6720d31839167) switched from DEPLOYING to RUNNING
>>> 08:52:33,772 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (5/25) (0edf387c2b8f7f3e6925192bc7e295ee)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,772 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (12/25)
>>> (f25aa9d229f5df70aafa40fa59d88264) switched from DEPLOYING to RUNNING
>>> 08:52:33,772 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (7/25) (fa434906c2e64e78743049e21494ad08)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,772 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (8/25) (504a1e8018bd16bbc1cdc5474bc5a496)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,773 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (9/25) (ca432acdde8ac296ee1b7fa169d80260)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,803 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (20/25) (f8243db3aa35c8cc6e725cff575e5dd1)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,800 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (18/25) (100bf52dde6379d11399c02e4b2c0a49)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,798 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (19/25)
>>> (f7fcba6ec0608360f363665afd99fb5d) switched from DEPLOYING to RUNNING
>>> 08:52:33,838 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (18/25) (100bf52dde6379d11399c02e4b2c0a49)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,838 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (18/25) (attempt #0) to slave3
>>> 08:52:33,847 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (22/25) (4c3bce50b83ec0e56d3e5940879f8efc)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,848 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (22/25) (4c3bce50b83ec0e56d3e5940879f8efc)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,848 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (22/25) (attempt #0) to slave3
>>> 08:52:33,848 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (22/25) (4c3bce50b83ec0e56d3e5940879f8efc)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,848 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (22/25) (attempt #0) to slave3
>>> 08:52:33,775 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (19/25) (b74e7a60547032d269422f8ccfe5b40b)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,792 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (21/25) (c4666c247ddad9d713670f7a84ab3aca)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,786 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (10/25) (4ec7d078930d8b31e89bc5bf63964029)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,866 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (19/25) (b74e7a60547032d269422f8ccfe5b40b)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,834 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (20/25) (f8243db3aa35c8cc6e725cff575e5dd1)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,866 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (19/25) (attempt #0) to slave3
>>> 08:52:33,867 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (20/25) (attempt #0) to slave3
>>> 08:52:33,869 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (23/25) (00dbd5965e361f476dfabd09ad803e66)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,869 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (23/25) (00dbd5965e361f476dfabd09ad803e66)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,869 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (23/25) (attempt #0) to vm-10-155-208-137
>>> 08:52:33,870 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (24/25) (b0cce659eb47fa75d876f280f53eaa68)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,871 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (24/25) (b0cce659eb47fa75d876f280f53eaa68)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,871 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (24/25) (attempt #0) to vm-10-155-208-137
>>> 08:52:33,878 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (25/25) (580c73a4bff8720e996cfdf248b98163)
>>> switched from CREATED to SCHEDULED
>>> 08:52:33,878 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (25/25) (580c73a4bff8720e996cfdf248b98163)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,878 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (25/25) (attempt #0) to vm-10-155-208-137
>>> 08:52:33,880 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (21/25) (c4666c247ddad9d713670f7a84ab3aca)
>>> switched from SCHEDULED to DEPLOYING
>>> 08:52:33,880 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying
>>> Keyed Aggregation -> Sink: Unnamed (21/25) (attempt #0) to vm-10-155-208-137
>>> 08:52:33,935 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (15/25) (4666479516b2ffe3c49ad7c027a80ad3)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:33,938 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (14/25) (c1192e402ab4a018b67f2c74f4286e13)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:34,013 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (17/25) (e41bf8f2e7e1034d4aeb993da711994d)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:34,173 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (16/25) (ac2ef9c1626ccb5bc92893b7f2302727)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:34,311 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (18/25) (100bf52dde6379d11399c02e4b2c0a49)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:34,311 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (22/25) (4c3bce50b83ec0e56d3e5940879f8efc)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:34,312 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (20/25) (f8243db3aa35c8cc6e725cff575e5dd1)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:34,450 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (24/25) (b0cce659eb47fa75d876f280f53eaa68)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:34,450 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (21/25) (c4666c247ddad9d713670f7a84ab3aca)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:34,455 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (23/25) (00dbd5965e361f476dfabd09ad803e66)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:34,590 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (19/25) (b74e7a60547032d269422f8ccfe5b40b)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:34,726 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (3/25) (07a2adef7d57cdd5081dd88533bb6631)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:34,726 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (17/25)
>>> (da4388c09a9874e7b7afc979a7af79b8) switched from DEPLOYING to RUNNING
>>> 08:52:34,726 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (10/25)
>>> (5f1e48cb807bfc4f837d0d2587d071e7) switched from DEPLOYING to RUNNING
>>> 08:52:34,727 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host vm-10-155-208-158
>>> 08:52:34,727 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host vm-10-155-208-158
>>> 08:52:34,729 INFO
>>>  org.apache.flink.api.common.io.LocatableInputSplitAssigner    - Assigning
>>> remote split to host vm-10-155-208-158
>>> 08:52:34,742 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (13/25) (3d2f2e956e1f5c3518c222f68b37698a)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:34,742 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (12/25) (78fc3cd38db931784d45e8b6a0ecd719)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:34,743 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (25/25) (580c73a4bff8720e996cfdf248b98163)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:34,747 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (11/25) (b6d53055b7787e63cda0ccfe6f1cf67d)
>>> switched from DEPLOYING to RUNNING
>>> 08:52:46,084 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (9/25) (88259e334ed2c8ea47b64f4f8083ae83)
>>> switched from RUNNING to FINISHED
>>> 08:52:46,750 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (16/25)
>>> (7339669af58d2f65f8ea69cb7b318773) switched from RUNNING to FINISHED
>>> 08:52:47,138 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (2/25) (b67073687787b804d628eb6482fa2a30)
>>> switched from RUNNING to FINISHED
>>> 08:52:48,700 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (8/25) (38f8b7ded3af389ccfc3a92f74761c1b)
>>> switched from RUNNING to FINISHED
>>> 08:52:48,891 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (15/25)
>>> (c8b82db126393db46682e036fd6f8527) switched from RUNNING to FINISHED
>>> 08:52:49,133 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (10/25)
>>> (5f1e48cb807bfc4f837d0d2587d071e7) switched from RUNNING to FINISHED
>>> 08:52:49,231 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (17/25)
>>> (da4388c09a9874e7b7afc979a7af79b8) switched from RUNNING to FINISHED
>>> 08:52:49,239 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (1/25) (bce7a3459fd80414266a6ac891b04551)
>>> switched from RUNNING to FINISHED
>>> 08:52:49,404 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (3/25) (07a2adef7d57cdd5081dd88533bb6631)
>>> switched from RUNNING to FINISHED
>>> 08:59:56,690 WARN  akka.remote.RemoteWatcher
>>>         - Detected unreachable: [akka.tcp://flink@10.155.208.138:42222]
>>> 08:59:56,696 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Task manager akka.tcp://
>>> flink@10.155.208.138:42222/user/taskmanager terminated.
>>> 08:59:56,697 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (16/25) (ac2ef9c1626ccb5bc92893b7f2302727)
>>> switched from RUNNING to FAILED
>>> 08:59:56,705 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (4/25) (da342b5a369f9da0deaffac409640210)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,708 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (5/25) (0edf387c2b8f7f3e6925192bc7e295ee)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,708 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (6/25) (de554cfcbc7f7ecf924fd3bfd1f951ce)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,708 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (7/25) (386211d919b02137f42e4df3fefb9619)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,709 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (11/25)
>>> (39e9a6f6c11427bf9024fcf6b88dbd3e) switched from RUNNING to CANCELING
>>> 08:59:56,709 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (12/25)
>>> (f25aa9d229f5df70aafa40fa59d88264) switched from RUNNING to CANCELING
>>> 08:59:56,709 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (13/25)
>>> (1e572ad1eab41b5f219d8ddd3d544699) switched from RUNNING to CANCELING
>>> 08:59:56,709 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (14/25)
>>> (10a5223d3d716bca1a50853f0efcab02) switched from RUNNING to CANCELING
>>> 08:59:56,709 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (18/25)
>>> (498d68b92dbb4c8a2a7a1db344fe01a8) switched from RUNNING to CANCELING
>>> 08:59:56,709 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (19/25)
>>> (f7fcba6ec0608360f363665afd99fb5d) switched from RUNNING to CANCELING
>>> 08:59:56,710 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (20/25)
>>> (fd5b27a36b4f09c2191fbf4c23ab61d1) switched from RUNNING to CANCELING
>>> 08:59:56,710 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (21/25)
>>> (3827b667c8e997c9e7e16ad0018cc767) switched from RUNNING to CANCELING
>>> 08:59:56,720 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (22/25)
>>> (4cbe7fa6850e61d695c3ac2a49995e60) switched from RUNNING to CANCELING
>>> 08:59:56,720 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (23/25)
>>> (1450b59a7088dd0ae6c6720d31839167) switched from RUNNING to CANCELING
>>> 08:59:56,747 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (24/25)
>>> (6766fa50924793cec3a4d1234bb62c06) switched from RUNNING to CANCELING
>>> 08:59:56,748 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (25/25)
>>> (2ce8063004c80a4d16104e873f82132e) switched from RUNNING to CANCELING
>>> 08:59:56,748 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (1/25) (9dee3c4d890f01233d5c95f15a9b3c96)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,748 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (2/25) (fbdd9e6f0a6ab2f5441afa4b171a8db3)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,749 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (3/25) (bbd7f74a8a57288a2e8a5c459418d6b3)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,749 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (4/25) (6b71db98a7a0f2fa53da59f2f8a21cce)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,749 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (5/25) (011b449fae3ff4b9769c7d75b1cc34ef)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,750 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (6/25) (61dec00d3e2736d3178d22f7c5c9e499)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,750 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (7/25) (fa434906c2e64e78743049e21494ad08)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,750 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (8/25) (504a1e8018bd16bbc1cdc5474bc5a496)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,751 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (9/25) (ca432acdde8ac296ee1b7fa169d80260)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,751 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (10/25) (4ec7d078930d8b31e89bc5bf63964029)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,752 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (11/25) (b6d53055b7787e63cda0ccfe6f1cf67d)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,752 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (12/25) (78fc3cd38db931784d45e8b6a0ecd719)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,754 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (13/25) (3d2f2e956e1f5c3518c222f68b37698a)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,755 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (14/25) (c1192e402ab4a018b67f2c74f4286e13)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,755 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (15/25) (4666479516b2ffe3c49ad7c027a80ad3)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,755 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (17/25) (e41bf8f2e7e1034d4aeb993da711994d)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,756 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (18/25) (100bf52dde6379d11399c02e4b2c0a49)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,763 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (19/25) (b74e7a60547032d269422f8ccfe5b40b)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,763 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (20/25) (f8243db3aa35c8cc6e725cff575e5dd1)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,764 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (21/25) (c4666c247ddad9d713670f7a84ab3aca)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,764 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (22/25) (4c3bce50b83ec0e56d3e5940879f8efc)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,764 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (23/25) (00dbd5965e361f476dfabd09ad803e66)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,764 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (24/25) (b0cce659eb47fa75d876f280f53eaa68)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,764 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (25/25) (580c73a4bff8720e996cfdf248b98163)
>>> switched from RUNNING to CANCELING
>>> 08:59:56,765 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (14/25)
>>> (10a5223d3d716bca1a50853f0efcab02) switched from CANCELING to FAILED
>>> 08:59:56,766 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (15/25) (4666479516b2ffe3c49ad7c027a80ad3)
>>> switched from CANCELING to FAILED
>>> 08:59:56,767 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (21/25)
>>> (3827b667c8e997c9e7e16ad0018cc767) switched from CANCELING to FAILED
>>> 08:59:56,768 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (7/25) (386211d919b02137f42e4df3fefb9619)
>>> switched from CANCELING to FAILED
>>> 08:59:56,769 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (17/25) (e41bf8f2e7e1034d4aeb993da711994d)
>>> switched from CANCELING to FAILED
>>> 08:59:56,771 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (14/25) (c1192e402ab4a018b67f2c74f4286e13)
>>> switched from CANCELING to FAILED
>>> 08:59:56,772 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (25/25)
>>> (2ce8063004c80a4d16104e873f82132e) switched from CANCELING to FAILED
>>> 08:59:56,773 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>         - Unregistered task manager akka.tcp://
>>> flink@10.155.208.138:42222/user/taskmanager. Number of registered task
>>> managers 6. Number of available slots 21.
>>> 08:59:56,774 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Status of job aeb1eca3a4feefa6fd00b599c0c783d4 (WordCount from
>>> SocketTextStream Example) changed to FAILING.
>>> java.lang.Exception: The slot in which the task was executed has been
>>> released. Probably loss of TaskManager 831a72dad6fbb533b193820f45bdc5bc @
>>> vm-10-155-208-138 - 4 slots - URL: akka.tcp://
>>> flink@10.155.208.138:42222/user/taskmanager
>>>         at
>>> org.apache.flink.runtime.instance.SimpleSlot.releaseSlot(SimpleSlot.java:153)
>>>         at
>>> org.apache.flink.runtime.instance.SlotSharingGroupAssignment.releaseSharedSlot(SlotSharingGroupAssignment.java:547)
>>>         at
>>> org.apache.flink.runtime.instance.SharedSlot.releaseSlot(SharedSlot.java:119)
>>>         at
>>> org.apache.flink.runtime.instance.Instance.markDead(Instance.java:156)
>>>         at
>>> org.apache.flink.runtime.instance.InstanceManager.unregisterTaskManager(InstanceManager.java:215)
>>>         at
>>> org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1.applyOrElse(JobManager.scala:696)
>>>         at
>>> scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)
>>>         at
>>> org.apache.flink.runtime.LeaderSessionMessageFilter$$anonfun$receive$1.applyOrElse(LeaderSessionMessageFilter.scala:44)
>>>         at
>>> scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)
>>>         at
>>> org.apache.flink.runtime.LogMessages$$anon$1.apply(LogMessages.scala:33)
>>>         at
>>> org.apache.flink.runtime.LogMessages$$anon$1.apply(LogMessages.scala:28)
>>>         at
>>> scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
>>>         at
>>> org.apache.flink.runtime.LogMessages$$anon$1.applyOrElse(LogMessages.scala:28)
>>>         at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
>>>         at
>>> org.apache.flink.runtime.jobmanager.JobManager.aroundReceive(JobManager.scala:100)
>>>         at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
>>>         at
>>> akka.actor.dungeon.DeathWatch$class.receivedTerminated(DeathWatch.scala:46)
>>>         at akka.actor.ActorCell.receivedTerminated(ActorCell.scala:369)
>>>         at akka.actor.ActorCell.autoReceiveMessage(ActorCell.scala:501)
>>>         at akka.actor.ActorCell.invoke(ActorCell.scala:486)
>>>         at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254)
>>>         at akka.dispatch.Mailbox.run(Mailbox.scala:221)
>>>         at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
>>>         at
>>> scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
>>>         at
>>> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
>>>         at
>>> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
>>>         at
>>> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
>>> 08:59:56,791 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (3/25) (bbd7f74a8a57288a2e8a5c459418d6b3)
>>> switched from CANCELING to CANCELED
>>> 08:59:56,791 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (1/25) (9dee3c4d890f01233d5c95f15a9b3c96)
>>> switched from CANCELING to CANCELED
>>> 08:59:56,791 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (2/25) (fbdd9e6f0a6ab2f5441afa4b171a8db3)
>>> switched from CANCELING to CANCELED
>>> 08:59:56,969 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (11/25) (b6d53055b7787e63cda0ccfe6f1cf67d)
>>> switched from CANCELING to CANCELED
>>> 08:59:56,973 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (12/25) (78fc3cd38db931784d45e8b6a0ecd719)
>>> switched from CANCELING to CANCELED
>>> 08:59:56,974 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (13/25) (3d2f2e956e1f5c3518c222f68b37698a)
>>> switched from CANCELING to CANCELED
>>> 08:59:57,117 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (4/25) (6b71db98a7a0f2fa53da59f2f8a21cce)
>>> switched from CANCELING to CANCELED
>>> 08:59:57,117 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (5/25) (011b449fae3ff4b9769c7d75b1cc34ef)
>>> switched from CANCELING to CANCELED
>>> 08:59:57,117 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (6/25) (61dec00d3e2736d3178d22f7c5c9e499)
>>> switched from CANCELING to CANCELED
>>> 09:00:21,690 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Task manager akka.tcp://
>>> flink@10.155.208.136:43058/user/taskmanager terminated.
>>> 09:00:21,692 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (8/25) (504a1e8018bd16bbc1cdc5474bc5a496)
>>> switched from CANCELING to FAILED
>>> 09:00:21,690 WARN  akka.remote.RemoteWatcher
>>>         - Detected unreachable: [akka.tcp://flink@10.155.208.136:43058]
>>> 09:00:21,693 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (19/25)
>>> (f7fcba6ec0608360f363665afd99fb5d) switched from CANCELING to FAILED
>>> 09:00:21,694 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (9/25) (ca432acdde8ac296ee1b7fa169d80260)
>>> switched from CANCELING to FAILED
>>> 09:00:21,694 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (12/25)
>>> (f25aa9d229f5df70aafa40fa59d88264) switched from CANCELING to FAILED
>>> 09:00:21,695 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (7/25) (fa434906c2e64e78743049e21494ad08)
>>> switched from CANCELING to FAILED
>>> 09:00:21,695 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (23/25)
>>> (1450b59a7088dd0ae6c6720d31839167) switched from CANCELING to FAILED
>>> 09:00:21,696 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (10/25) (4ec7d078930d8b31e89bc5bf63964029)
>>> switched from CANCELING to FAILED
>>> 09:00:21,696 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (5/25) (0edf387c2b8f7f3e6925192bc7e295ee)
>>> switched from CANCELING to FAILED
>>> 09:00:21,696 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>         - Unregistered task manager akka.tcp://
>>> flink@10.155.208.136:43058/user/taskmanager. Number of registered task
>>> managers 5. Number of available slots 17.
>>> 09:00:52,689 WARN  akka.remote.RemoteWatcher
>>>         - Detected unreachable: [akka.tcp://flink@10.155.208.135:54516]
>>> 09:00:52,689 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Task manager akka.tcp://
>>> flink@10.155.208.135:54516/user/taskmanager terminated.
>>> 09:00:52,691 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (18/25) (100bf52dde6379d11399c02e4b2c0a49)
>>> switched from CANCELING to FAILED
>>> 09:00:52,692 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (18/25)
>>> (498d68b92dbb4c8a2a7a1db344fe01a8) switched from CANCELING to FAILED
>>> 09:00:52,693 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (22/25)
>>> (4cbe7fa6850e61d695c3ac2a49995e60) switched from CANCELING to FAILED
>>> 09:00:52,697 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (20/25) (f8243db3aa35c8cc6e725cff575e5dd1)
>>> switched from CANCELING to FAILED
>>> 09:00:52,698 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (4/25) (da342b5a369f9da0deaffac409640210)
>>> switched from CANCELING to FAILED
>>> 09:00:52,699 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (19/25) (b74e7a60547032d269422f8ccfe5b40b)
>>> switched from CANCELING to FAILED
>>> 09:00:52,700 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (11/25)
>>> (39e9a6f6c11427bf9024fcf6b88dbd3e) switched from CANCELING to FAILED
>>> 09:00:52,700 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (22/25) (4c3bce50b83ec0e56d3e5940879f8efc)
>>> switched from CANCELING to FAILED
>>> 09:00:52,700 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>         - Unregistered task manager akka.tcp://
>>> flink@10.155.208.135:54516/user/taskmanager. Number of registered task
>>> managers 4. Number of available slots 13.
>>> 09:02:44,698 WARN  akka.remote.RemoteWatcher
>>>         - Detected unreachable: [akka.tcp://flink@10.155.208.137:41096]
>>> 09:02:44,699 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Task manager akka.tcp://
>>> flink@10.155.208.137:41096/user/taskmanager terminated.
>>> 09:02:44,699 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (13/25)
>>> (1e572ad1eab41b5f219d8ddd3d544699) switched from CANCELING to FAILED
>>> 09:02:44,700 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (25/25) (580c73a4bff8720e996cfdf248b98163)
>>> switched from CANCELING to FAILED
>>> 09:02:44,701 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (21/25) (c4666c247ddad9d713670f7a84ab3aca)
>>> switched from CANCELING to FAILED
>>> 09:02:44,701 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (6/25) (de554cfcbc7f7ecf924fd3bfd1f951ce)
>>> switched from CANCELING to FAILED
>>> 09:02:44,711 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (24/25)
>>> (6766fa50924793cec3a4d1234bb62c06) switched from CANCELING to FAILED
>>> 09:02:44,711 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (23/25) (00dbd5965e361f476dfabd09ad803e66)
>>> switched from CANCELING to FAILED
>>> 09:02:44,712 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Keyed
>>> Aggregation -> Sink: Unnamed (24/25) (b0cce659eb47fa75d876f280f53eaa68)
>>> switched from CANCELING to FAILED
>>> 09:02:44,712 INFO
>>>  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
>>> Read Text File Source -> Flat Map (20/25)
>>> (fd5b27a36b4f09c2191fbf4c23ab61d1) switched from CANCELING to FAILED
>>> 09:02:44,713 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>         - Unregistered task manager akka.tcp://
>>> flink@10.155.208.137:41096/user/taskmanager. Number of registered task
>>> managers 3. Number of available slots 9.
>>> 09:02:44,713 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>          - Status of job aeb1eca3a4feefa6fd00b599c0c783d4 (WordCount from
>>> SocketTextStream Example) changed to FAILED.
>>> java.lang.Exception: The slot in which the task was executed has been
>>> released. Probably loss of TaskManager 831a72da
>>>
>>> Kind Regards,
>>> Ravinder
>>>
>>> On Tue, Mar 15, 2016 at 11:34 AM, Till Rohrmann <trohrmann@apache.org>
>>> wrote:
>>>
>>>> Hi Ravinder,
>>>>
>>>> the log of the TM you've sent is the log of the only TM which has not
>>>> been disassociated from the JM. Can it be that you simply stopped the
>>>> cluster which results in the disassociation events?
>>>>
>>>> Normally, Flink should kill all processes. If you have some processes
>>>> lingering around, then you should kill them first.
>>>>
>>>> The more memory you provide the more data can be kept in memory.
>>>> Whenever the managed memory is full, then it will be spilled to disk.
>>>> That's how you can also process data which does not fit completely into
>>>> memory. However, all elements which are given to a user function will be
>>>> kept on the heap space. If it now happens that your elements become too big
>>>> or you keep too many elements on the heap, you'll see an OOM exception.
>>>> Then it helps if you increase the assigned memory or lower the memory
>>>> fraction.
>>>>
>>>> Cheers,
>>>> Till
>>>>
>>>> On Tue, Mar 15, 2016 at 11:17 AM, Ravinder Kaur <neetu0404@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Till,
>>>>>
>>>>> Log of JobManager
>>>>>
>>>>> 09:55:31,574 WARN  org.apache.hadoop.util.NativeCodeLoader
>>>>>           - Unable to load native-hadoop library for your platform... using
>>>>> builtin-java classes where applicable
>>>>> 09:55:31,742 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -
>>>>> --------------------------------------------------------------------------------
>>>>> 09:55:31,742 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -  Starting JobManager (Version: 0.10.1, Rev:2e9b231,
>>>>> Date:22.11.2015 @ 12:41:12 CET)
>>>>> 09:55:31,742 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -  Current user: flink
>>>>> 09:55:31,742 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -  JVM: OpenJDK 64-Bit Server VM - Oracle Corporation -
>>>>> 1.7/24.95-b01
>>>>> 09:55:31,743 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -  Maximum heap size: 246 MiBytes
>>>>> 09:55:31,743 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -  JAVA_HOME: /usr/lib/jvm/java-1.7.0-openjdk-amd64
>>>>> 09:55:31,745 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -  Hadoop version: 2.7.0
>>>>> 09:55:31,746 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -  JVM Options:
>>>>> 09:55:31,746 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -     -Xms256m
>>>>> 09:55:31,746 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -     -Xmx256m
>>>>> 09:55:31,746 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -     -XX:MaxPermSize=256m
>>>>> 09:55:31,746 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -
>>>>> -Dlog.file=/home/flink/flink-0.10.1/log/flink-flink-jobmanager-0-vm-10-155-208-156.cloud.mwn.de.log
>>>>> 09:55:31,746 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -
>>>>> -Dlog4j.configuration=file:/home/flink/flink-0.10.1/conf/log4j.properties
>>>>> 09:55:31,746 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -
>>>>> -Dlogback.configurationFile=file:/home/flink/flink-0.10.1/conf/logback.xml
>>>>> 09:55:31,746 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -  Program Arguments:
>>>>> 09:55:31,746 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -     --configDir
>>>>> 09:55:31,746 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -     /home/flink/flink-0.10.1/conf
>>>>> 09:55:31,746 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -     --executionMode
>>>>> 09:55:31,746 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -     cluster
>>>>> 09:55:31,746 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -     --streamingMode
>>>>> 09:55:31,746 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -     streaming
>>>>> 09:55:31,746 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -  Classpath:
>>>>> /home/flink/flink-0.10.1/lib/flink-dist_2.11-0.10.1.jar:/home/flink/flink-0.10.1/lib/flink-python_2.11-0.10.1.jar:/home/flink/flink-0.10.1/lib/log4j-1.2.17.jar:/home/flink/flink-0.10.1/lib/slf4j-log4j12-1.7.7.jar:::
>>>>> 09:55:31,746 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            -
>>>>> --------------------------------------------------------------------------------
>>>>> 09:55:31,924 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            - Loading configuration from /home/flink/flink-0.10.1/conf
>>>>> 09:55:31,941 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            - Staring JobManager without high-availability
>>>>> 09:55:31,950 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            - Starting JobManager on 10.155.208.156:6123 with
>>>>> execution mode CLUSTER and streaming mode STREAMING
>>>>> 09:55:32,039 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            - Security is not enabled. Starting non-authenticated JobManager.
>>>>> 09:55:32,039 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            - Starting JobManager
>>>>> 09:55:32,040 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            - Starting JobManager actor system at 10.155.208.156:6123
>>>>> 09:55:32,483 INFO  akka.event.slf4j.Slf4jLogger
>>>>>            - Slf4jLogger started
>>>>> 09:55:32,564 INFO  Remoting
>>>>>            - Starting remoting
>>>>> 09:55:32,730 INFO  Remoting
>>>>>            - Remoting started; listening on addresses :[akka.tcp://
>>>>> flink@10.155.208.156:6123]
>>>>> 09:55:32,731 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            - Starting JobManger web frontend
>>>>> 09:55:32,761 INFO
>>>>>  org.apache.flink.runtime.webmonitor.WebRuntimeMonitor         - Using
>>>>> directory /tmp/flink-web-6cd96e7e-62be-4301-9376-c98528bd58b8 for the web
>>>>> interface files
>>>>> 09:55:32,762 INFO
>>>>>  org.apache.flink.runtime.webmonitor.WebRuntimeMonitor         - Serving
>>>>> job manager log from
>>>>> /home/flink/flink-0.10.1/log/flink-flink-jobmanager-0-vm-10-155-208-156.cloud.mwn.de.log
>>>>> 09:55:32,762 INFO
>>>>>  org.apache.flink.runtime.webmonitor.WebRuntimeMonitor         - Serving
>>>>> job manager stdout from
>>>>> /home/flink/flink-0.10.1/log/flink-flink-jobmanager-0-vm-10-155-208-156.cloud.mwn.de.out
>>>>> 09:55:33,040 INFO
>>>>>  org.apache.flink.runtime.webmonitor.WebRuntimeMonitor         - Web
>>>>> frontend listening at 0:0:0:0:0:0:0:0:8081
>>>>> 09:55:33,041 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            - Starting JobManager actor
>>>>> 09:55:33,046 INFO  org.apache.flink.runtime.blob.BlobServer
>>>>>            - Created BLOB server storage directory
>>>>> /tmp/blobStore-28cbb318-efc7-4a8b-85a0-1ea6539f1ba8
>>>>> 09:55:33,047 INFO  org.apache.flink.runtime.blob.BlobServer
>>>>>            - Started BLOB server at 0.0.0.0:59504 - max concurrent
>>>>> requests: 50 - max backlog: 1000
>>>>> 09:55:33,057 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            - Starting JobManager at akka.tcp://
>>>>> flink@10.155.208.156:6123/user/jobmanager.
>>>>> 09:55:33,060 INFO  org.apache.flink.runtime.jobmanager.JobManager
>>>>>            - JobManager akka.tcp://
>>>>> flink@10.155.208.156:6123/user/jobmanager was granted leadership with
>>>>> leader session ID None.
>>>>> 09:55:33,063 INFO  org.apache.flink.runtime.jobmanager.MemoryArchivist
>>>>>           - Started memory archivist akka://flink/user/archive
>>>>> 09:55:33,064 INFO
>>>>>  org.apache.flink.runtime.webmonitor.WebRuntimeMonitor         - Starting
>>>>> with JobManager akka.tcp://flink@10.155.208.156:6123/user/jobmanager
>>>>> on port 8081
>>>>> 09:55:33,064 INFO
>>>>>  org.apache.flink.runtime.webmonitor.JobManagerRetriever       - New leader
>>>>> reachable under akka.tcp://
>>>>> flink@10.155.208.156:6123/user/jobmanager:null.
>>>>> 09:55:34,013 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>>>           - Registered TaskManager at vm-10-155-208-156 (akka.tcp://
>>>>> flink@10.155.208.156:33728/user/taskmanager) as
>>>>> 9b665ef16d88314bf37f816b5afdfe79. Current number of registered hosts is 1.
>>>>> Current number of alive task slots is 3.
>>>>> 09:55:34,735 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>>>           - Registered TaskManager at vm-10-155-208-157 (akka.tcp://
>>>>> flink@10.155.208.157:33728/user/taskmanager) as
>>>>> fc8b661bb0ad5568855c8c2d6a0029f2. Current number of registered hosts is 2.
>>>>> Current number of alive task slots is 6.
>>>>> 09:55:35,394 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>>>           - Registered TaskManager at vm-10-155-208-158 (akka.tcp://
>>>>> flink@10.155.208.158:33728/user/taskmanager) as
>>>>> c12e0ca481aaa4bdd6fa6be4cfc8335e. Current number of registered hosts is 3.
>>>>> Current number of alive task slots is 9.
>>>>> 09:55:36,105 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>>>           - Registered TaskManager at slave3 (akka.tcp://
>>>>> flink@10.155.208.135:37219/user/taskmanager) as
>>>>> 6e023a6417743d1e5d67410dc76824b8. Current number of registered hosts is 4.
>>>>> Current number of alive task slots is 13.
>>>>> 09:55:36,542 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>>>           - Registered TaskManager at vm-10-155-208-137 (akka.tcp://
>>>>> flink@10.155.208.137:35052/user/taskmanager) as
>>>>> c0098ff87ca76c958ca18a4e7d30e24e. Current number of registered hosts is 5.
>>>>> Current number of alive task slots is 17.
>>>>> 09:55:37,422 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>>>           - Registered TaskManager at slave2 (akka.tcp://
>>>>> flink@10.155.208.136:50432/user/taskmanager) as
>>>>> d62208a25ce877757aeb72fe4b6530fc. Current number of registered hosts is 6.
>>>>> Current number of alive task slots is 21.
>>>>> 09:55:38,351 INFO  org.apache.flink.runtime.instance.InstanceManager
>>>>>           - Registered TaskManager at vm-10-155-208-138 (akka.tcp://
>>>>> flink@10.155.208.138:49653/user/taskmanager) as
>>>>> b0c2566fcc0e2baf7e2605e96e5a6b9c. Current number of registered hosts is 7.
>>>>> Current number of alive task slots is 25.
>>>>> 09:56:45,448 WARN  akka.remote.ReliableDeliverySupervisor
>>>>>            - Association with remote system [akka.tcp://
>>>>> flink@10.155.208.156:33728] has failed, address is now gated for
>>>>> [5000] ms. Reason is: [Disassociated].
>>>>> 09:56:46,013 WARN  akka.remote.ReliableDeliverySupervisor
>>>>>            - Association with remote system [akka.tcp://
>>>>> flink@10.155.208.157:33728] has failed, address is now gated for
>>>>> [5000] ms. Reason is: [Disassociated].
>>>>> 09:56:46,630 WARN  akka.remote.ReliableDeliverySupervisor
>>>>>            - Association with remote system [akka.tcp://
>>>>> flink@10.155.208.158:33728] has failed, address is now gated for
>>>>> [5000] ms. Reason is: [Disassociated].
>>>>> 09:56:47,259 WARN  akka.remote.ReliableDeliverySupervisor
>>>>>            - Association with remote system [akka.tcp://
>>>>> flink@10.155.208.135:37219] has failed, address is now gated for
>>>>> [5000] ms. Reason is: [Disassociated].
>>>>> 09:56:47,788 WARN  akka.remote.ReliableDeliverySupervisor
>>>>>            - Association with remote system [akka.tcp://
>>>>> flink@10.155.208.137:35052] has failed, address is now gated for
>>>>> [5000] ms. Reason is: [Disassociated].
>>>>> 09:56:48,333 WARN  akka.remote.ReliableDeliverySupervisor
>>>>>            - Association with remote system [akka.tcp://
>>>>> flink@10.155.208.136:50432] has failed, address is now gated for
>>>>> [5000] ms. Reason is: [Disassociated].
>>>>> 09:56:48,629 INFO
>>>>>  org.apache.flink.runtime.webmonitor.WebRuntimeMonitor         - Removing
>>>>> web root dir /tmp/flink-web-6cd96e7e-62be-4301-9376-c98528bd58b8
>>>>> 09:56:48,635 INFO  org.apache.flink.runtime.blob.BlobServer
>>>>>            - Stopped BLOB server at 0.0.0.0:59504
>>>>>
>>>>> and One of the TaskManagers
>>>>>
>>>>> 09:55:36,694 WARN  org.apache.hadoop.util.NativeCodeLoader
>>>>>           - Unable to load native-hadoop library for your platform... using
>>>>> builtin-java classes where applicable
>>>>> 09:55:36,918 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -
>>>>> --------------------------------------------------------------------------------
>>>>> 09:55:36,918 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -  Starting TaskManager (Version: 0.10.1, Rev:2e9b231,
>>>>> Date:22.11.2015 @ 12:41:12 CET)
>>>>> 09:55:36,919 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -  Current user: flink
>>>>> 09:55:36,919 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -  JVM: OpenJDK 64-Bit Server VM - Oracle Corporation -
>>>>> 1.7/24.91-b01
>>>>> 09:55:36,919 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -  Maximum heap size: 990 MiBytes
>>>>> 09:55:36,919 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -  JAVA_HOME: /usr/lib/jvm/java-1.7.0-openjdk-amd64
>>>>> 09:55:36,922 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -  Hadoop version: 2.7.0
>>>>> 09:55:36,922 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -  JVM Options:
>>>>> 09:55:36,922 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -     -XX:+UseConcMarkSweepGC
>>>>> 09:55:36,922 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -     -XX:+CMSClassUnloadingEnabled
>>>>> 09:55:36,922 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -     -Xms1024M
>>>>> 09:55:36,922 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -     -Xmx1024M
>>>>> 09:55:36,922 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -     -XX:MaxDirectMemorySize=8388607T
>>>>> 09:55:36,922 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -     -XX:MaxPermSize=256m
>>>>> 09:55:36,922 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -
>>>>> -Dlog.file=/home/flink/flink-0.10.1/log/flink-flink-taskmanager-0-vm-10-155-208-138.cloud.mwn.de.log
>>>>> 09:55:36,923 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -
>>>>> -Dlog4j.configuration=file:/home/flink/flink-0.10.1/conf/log4j.properties
>>>>> 09:55:36,923 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -
>>>>> -Dlogback.configurationFile=file:/home/flink/flink-0.10.1/conf/logback.xml
>>>>> 09:55:36,923 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -  Program Arguments:
>>>>> 09:55:36,923 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -     --configDir
>>>>> 09:55:36,923 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -     /home/flink/flink-0.10.1/conf
>>>>> 09:55:36,923 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -     --streamingMode
>>>>> 09:55:36,923 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -     streaming
>>>>> 09:55:36,923 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -  Classpath:
>>>>> /home/flink/flink-0.10.1/lib/flink-dist_2.11-0.10.1.jar:/home/flink/flink-0.10.1/lib/flink-python_2.11-0.10.1.jar:/home/flink/flink-0.10.1/lib/log4j-1.2.17.jar:/home/flink/flink-0.10.1/lib/slf4j-log4j12-1.7.7.jar:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar::
>>>>> 09:55:36,923 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            -
>>>>> --------------------------------------------------------------------------------
>>>>> 09:55:36,928 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - Maximum number of open file descriptors is 4096
>>>>> 09:55:36,955 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - Loading configuration from /home/flink/flink-0.10.1/conf
>>>>> 09:55:37,040 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - Security is not enabled. Starting non-authenticated
>>>>> TaskManager.
>>>>> 09:55:37,071 INFO  org.apache.flink.runtime.util.LeaderRetrievalUtils
>>>>>            - Trying to select the network interface and address to use by
>>>>> connecting to the leading JobManager.
>>>>> 09:55:37,072 INFO  org.apache.flink.runtime.util.LeaderRetrievalUtils
>>>>>            - TaskManager will try to connect for 10000 milliseconds before
>>>>> falling back to heuristics
>>>>> 09:55:37,075 INFO  org.apache.flink.runtime.net.ConnectionUtils
>>>>>            - Retrieved new target address /10.155.208.156:6123.
>>>>> 09:55:37,084 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - TaskManager will use hostname/address '
>>>>> vm-10-155-208-138.cloud.mwn.de' (10.155.208.138) for communication.
>>>>> 09:55:37,085 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - Starting TaskManager in streaming mode STREAMING
>>>>> 09:55:37,085 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - Starting TaskManager actor system at 10.155.208.138:0
>>>>> 09:55:37,531 INFO  akka.event.slf4j.Slf4jLogger
>>>>>            - Slf4jLogger started
>>>>> 09:55:37,587 INFO  Remoting
>>>>>            - Starting remoting
>>>>> 09:55:37,774 INFO  Remoting
>>>>>            - Remoting started; listening on addresses :[akka.tcp://
>>>>> flink@10.155.208.138:49653]
>>>>> 09:55:37,782 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - Starting TaskManager actor
>>>>> 09:55:37,798 INFO
>>>>>  org.apache.flink.runtime.io.network.netty.NettyConfig         -
>>>>> NettyConfig [server address:
>>>>> vm-10-155-208-138.cloud.mwn.de/10.155.208.138, server port: 32798,
>>>>> memory segment size (bytes): 32768, transport type: NIO, number of server
>>>>> threads: 0 (use Netty's default), number of client threads: 0 (use Netty's
>>>>> default), server connect backlog: 0 (use Netty's default), client connect
>>>>> timeout (sec): 120, send/receive buffer size (bytes): 0 (use Netty's
>>>>> default)]
>>>>> 09:55:37,803 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - Messages between TaskManager and JobManager have a max timeout
>>>>> of 100000 milliseconds
>>>>> 09:55:37,811 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - Temporary file directory '/tmp': total 4 GB, usable 0 GB
>>>>> (0.00% usable)
>>>>> 09:55:37,848 INFO
>>>>>  org.apache.flink.runtime.io.network.buffer.NetworkBufferPool  - Allocated
>>>>> 64 MB for network buffer pool (number of memory segments: 2048, bytes per
>>>>> segment: 32768).
>>>>> 09:55:37,955 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - Using 0.5 of the currently free heap space for Flink managed
>>>>> heap memory (455 MB).
>>>>> 09:55:37,978 INFO
>>>>>  org.apache.flink.runtime.io.disk.iomanager.IOManager          - I/O
>>>>> manager uses directory /tmp/flink-io-3b11098e-a3ea-4a8a-8ea4-c3f1c5b13d6f
>>>>> for spill files.
>>>>> 09:55:37,986 INFO  org.apache.flink.runtime.filecache.FileCache
>>>>>            - User file cache uses directory
>>>>> /tmp/flink-dist-cache-516dd09a-1dfe-46eb-b50b-b6e24b6e9fad
>>>>> 09:55:38,146 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - Starting TaskManager actor at
>>>>> akka://flink/user/taskmanager#56985599.
>>>>> 09:55:38,146 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - TaskManager data connection information:
>>>>> vm-10-155-208-138.cloud.mwn.de (dataPort=32798)
>>>>> 09:55:38,147 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - TaskManager has 4 task slot(s).
>>>>> 09:55:38,148 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - Memory usage stats: [HEAP: 100/990/990 MB, NON HEAP: 24/37/304
>>>>> MB (used/committed/max)]
>>>>> 09:55:38,151 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - Trying to register at JobManager akka.tcp://
>>>>> flink@10.155.208.156:6123/user/jobmanager (attempt 1, timeout: 500
>>>>> milliseconds)
>>>>> 09:55:38,301 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - Successful registration at JobManager (akka.tcp://
>>>>> flink@10.155.208.156:6123/user/jobmanager), starting network stack
>>>>> and library cache.
>>>>> 09:55:38,479 INFO
>>>>>  org.apache.flink.runtime.io.network.netty.NettyClient         - Successful
>>>>> initialization (took 55 ms).
>>>>> 09:55:38,533 INFO
>>>>>  org.apache.flink.runtime.io.network.netty.NettyServer         - Successful
>>>>> initialization (took 54 ms). Listening on SocketAddress /
>>>>> 10.155.208.138:32798.
>>>>> 09:55:38,534 INFO  org.apache.flink.runtime.taskmanager.TaskManager
>>>>>            - Determined BLOB server address to be /
>>>>> 10.155.208.156:59504. Starting BLOB cache.
>>>>> 09:55:38,536 INFO  org.apache.flink.runtime.blob.BlobCache
>>>>>           - Created BLOB cache storage directory
>>>>> /tmp/blobStore-8e88302d-3303-4c80-8613-f0be13911fb2
>>>>> 09:56:48,371 INFO
>>>>>  org.apache.flink.runtime.io.disk.iomanager.IOManager          - I/O
>>>>> manager removed spill file directory
>>>>> /tmp/flink-io-3b11098e-a3ea-4a8a-8ea4-c3f1c5b13d6f
>>>>>
>>>>> I also found that 4 of the deamons were not stopped after the cluster
>>>>> was stopped, though the JM claimed it had stopped these TM deamons.
>>>>>
>>>>> I have another question from you previous comment as to why is it
>>>>> necessary to allocate 50 GB of memory to taskmanager.heap.mb? Is the value
>>>>> I provided not sufficient for the job? If this is so, how was I able to run
>>>>> the previous examples with input datasets as large as 50GB error-free? It
>>>>> would help if you could provide some explanation here?
>>>>>
>>>>> Thank you.
>>>>>
>>>>> Kind Regards,
>>>>> Ravinder Kaur
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Mar 15, 2016 at 11:01 AM, Till Rohrmann <trohrmann@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Hi Ravinder,
>>>>>>
>>>>>> this should not be the relevant log extract. The log says that the TM
>>>>>> is started on port 49653 and the JM log says that the TM on port 42222 is
>>>>>> lost. Would you mind to share the complete JM and TM logs with us?
>>>>>>
>>>>>> Cheers,
>>>>>> Till
>>>>>>
>>>>>> On Tue, Mar 15, 2016 at 10:54 AM, Ravinder Kaur <neetu0404@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hello Ufuk,
>>>>>>>
>>>>>>> Yes, the same WordCount program is being run.
>>>>>>>
>>>>>>> Kind Regards,
>>>>>>> Ravinder Kaur
>>>>>>>
>>>>>>> On Tue, Mar 15, 2016 at 10:45 AM, Ufuk Celebi <uce@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> What do you mean with iteration in this context? Are you repeatedly
>>>>>>>> running the same WordCount program for streaming and batch
>>>>>>>> respectively?
>>>>>>>>
>>>>>>>> – Ufuk
>>>>>>>>
>>>>>>>> On Tue, Mar 15, 2016 at 10:22 AM, Till Rohrmann <
>>>>>>>> trohrmann@apache.org> wrote:
>>>>>>>> > Hi Ravinder,
>>>>>>>> >
>>>>>>>> > could you tell us what's written in the taskmanager log of the
>>>>>>>> failing
>>>>>>>> > taskmanager? There should be some kind of failure why the
>>>>>>>> taskmanager
>>>>>>>> > stopped working.
>>>>>>>> >
>>>>>>>> > Moreover, given that you have 64 GB of main memory, you could
>>>>>>>> easily give
>>>>>>>> > 50GB as heap memory to each taskmanager.
>>>>>>>> >
>>>>>>>> > Cheers,
>>>>>>>> > Till
>>>>>>>> >
>>>>>>>> > On Tue, Mar 15, 2016 at 9:48 AM, Ravinder Kaur <
>>>>>>>> neetu0404@gmail.com> wrote:
>>>>>>>> >>
>>>>>>>> >> Hello All,
>>>>>>>> >>
>>>>>>>> >> I'm running a simple word count example using the quickstart
>>>>>>>> package from
>>>>>>>> >> the Flink(0.10.1), on an input dataset of 500MB. This dataset is
>>>>>>>> a set of
>>>>>>>> >> randomly generated words of length 8.
>>>>>>>> >>
>>>>>>>> >> Cluster Configuration:
>>>>>>>> >>
>>>>>>>> >> Number of machines: 7
>>>>>>>> >> Total cores : 25
>>>>>>>> >> Memory on each: 64GB
>>>>>>>> >>
>>>>>>>> >> I'm interested in the performance measure between Batch and
>>>>>>>> Stream modes
>>>>>>>> >> and so I'm running WordCount example with number of iteration
>>>>>>>> (max 10) on
>>>>>>>> >> datasets of sizes ranging between 100MB and 50GB consisting of
>>>>>>>> random words
>>>>>>>> >> of length 4 and 8.
>>>>>>>> >>
>>>>>>>> >> While I ran the experiments in Batch mode all iterations ran
>>>>>>>> fine, but now
>>>>>>>> >> I'm stuck in the Streaming mode at this
>>>>>>>> >>
>>>>>>>> >> Caused by: java.lang.OutOfMemoryError: Java heap space
>>>>>>>> >>         at java.util.HashMap.resize(HashMap.java:580)
>>>>>>>> >>         at java.util.HashMap.addEntry(HashMap.java:879)
>>>>>>>> >>         at java.util.HashMap.put(HashMap.java:505)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> org.apache.flink.runtime.state.AbstractHeapKvState.update(AbstractHeapKvState.java:98)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> org.apache.flink.streaming.api.operators.StreamGroupedReduce.processElement(StreamGroupedReduce.java:59)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:166)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:63)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:218)
>>>>>>>> >>         at
>>>>>>>> org.apache.flink.runtime.taskmanager.Task.run(Task.java:584)
>>>>>>>> >>         at java.lang.Thread.run(Thread.java:745)
>>>>>>>> >>
>>>>>>>> >> I investigated found 2 solutions. (1) Increasing the
>>>>>>>> taskmanager.heap.mb
>>>>>>>> >> and (2) Reducing the taskmanager.memory.fraction
>>>>>>>> >>
>>>>>>>> >> Therefore I set taskmanager.heap.mb: 1024 and
>>>>>>>> taskmanager.memory.fraction:
>>>>>>>> >> 0.5 (default 0.7)
>>>>>>>> >>
>>>>>>>> >> When I ran the example with this setting I loose taskmanagers
>>>>>>>> one by one
>>>>>>>> >> during the job execution with the following cause
>>>>>>>> >>
>>>>>>>> >> Caused by: java.lang.Exception: The slot in which the task was
>>>>>>>> executed
>>>>>>>> >> has been released. Probably loss of TaskManager
>>>>>>>> >> 831a72dad6fbb533b193820f45bdc5bc @ vm-10-155-208-138 - 4 slots -
>>>>>>>> URL:
>>>>>>>> >> akka.tcp://flink@10.155.208.138:42222/user/taskmanager
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> org.apache.flink.runtime.instance.SimpleSlot.releaseSlot(SimpleSlot.java:153)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> org.apache.flink.runtime.instance.SlotSharingGroupAssignment.releaseSharedSlot(SlotSharingGroupAssignment.java:547)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> org.apache.flink.runtime.instance.SharedSlot.releaseSlot(SharedSlot.java:119)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> org.apache.flink.runtime.instance.Instance.markDead(Instance.java:156)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> org.apache.flink.runtime.instance.InstanceManager.unregisterTaskManager(InstanceManager.java:215)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1.applyOrElse(JobManager.scala:696)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> org.apache.flink.runtime.LeaderSessionMessageFilter$$anonfun$receive$1.applyOrElse(LeaderSessionMessageFilter.scala:44)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> org.apache.flink.runtime.LogMessages$$anon$1.apply(LogMessages.scala:33)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> org.apache.flink.runtime.LogMessages$$anon$1.apply(LogMessages.scala:28)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> org.apache.flink.runtime.LogMessages$$anon$1.applyOrElse(LogMessages.scala:28)
>>>>>>>> >>         at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> org.apache.flink.runtime.jobmanager.JobManager.aroundReceive(JobManager.scala:100)
>>>>>>>> >>         at
>>>>>>>> akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> akka.actor.dungeon.DeathWatch$class.receivedTerminated(DeathWatch.scala:46)
>>>>>>>> >>         at
>>>>>>>> akka.actor.ActorCell.receivedTerminated(ActorCell.scala:369)
>>>>>>>> >>         at
>>>>>>>> akka.actor.ActorCell.autoReceiveMessage(ActorCell.scala:501)
>>>>>>>> >>         at akka.actor.ActorCell.invoke(ActorCell.scala:486)
>>>>>>>> >>         at
>>>>>>>> akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254)
>>>>>>>> >>         at akka.dispatch.Mailbox.run(Mailbox.scala:221)
>>>>>>>> >>         at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
>>>>>>>> >>         at
>>>>>>>> >>
>>>>>>>> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
>>>>>>>> >>         ... 2 more
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> >> While I look at the results generated at each taskmanager, they
>>>>>>>> are fine.
>>>>>>>> >> The logs also don't show any causes for the the job to get
>>>>>>>> cancelled.
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> >> Could anyone kindly guide me here?
>>>>>>>> >>
>>>>>>>> >> Kind Regards,
>>>>>>>> >> Ravinder Kaur.
>>>>>>>> >
>>>>>>>> >
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message