flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Will Walters <wwalt...@yahoo-inc.com>
Subject Cluster Memory Error
Date Fri, 02 Jun 2017 22:26:43 GMT
I am attempting to use the Flink runner to execute a data pipeline on a Hadoop cluster. The
pipeline is created successfully in Flink, but when it attempts to execute the tasks, I receive
a 'Size of total memory must be positive' error, despite having plenty of memory.
Here's the log around the area where the error occurs:
17/06/02 22:06:06 INFO taskmanager.TaskManager: Messages between TaskManager and JobManager
have a max timeout of 10000 milliseconds17/06/02 22:06:06 INFO taskmanager.TaskManager: Temporary
file directory '/tmp': total 416 GB, usable 310 GB (74.52% usable)17/06/02 22:06:06 INFO jobmanager.JobManager:
JobManager akka://flink/user/jobmanager_1 was granted leadership with leader session ID None.17/06/02
22:06:06 INFO buffer.NetworkBufferPool: Allocated 64 MB for network buffer pool (number of
memory segments: 2048, bytes per segment: 32768).17/06/02 22:06:06 INFO standalone.StandaloneResourceManager:
Resource Manager associating with leading JobManager Actor[akka://flink/user/jobmanager_1#2084519917]
- leader session null17/06/02 22:06:06 INFO network.NetworkEnvironment: Starting the network
environment and its components.17/06/02 22:06:06 INFO taskmanager.TaskManager: Limiting managed
memory to 17592186044406 MB, memory will be allocated lazily.17/06/02 22:06:06 ERROR flink.FlinkRunner:
Pipeline execution failedjava.lang.IllegalArgumentException: Size of total memory must be
positive. at org.apache.flink.runtime.memory.MemoryManager.<init>(MemoryManager.java:131)
Any help would be much appreciated!
Thanks,Will Walters.
View raw message