flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1952) Cannot run ConnectedComponents example: Could not allocate a slot on instance
Date Tue, 19 May 2015 10:28:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550201#comment-14550201
] 

Stephan Ewen commented on FLINK-1952:
-------------------------------------

I cannot reproduce this error any more with these steps:
  - ConnectedComponents (successful)
  - KMeans (runs out of buffers)
  - ConnectedComponents

The third program now succeeds as expected.

I think the reason is that the task lifecycle is more robust now (with my changes from two
weeks ago). 

Still, it does not mean that the slot sharing groups are bug free. It only means that this
situation does not trigger that bug any more...

> Cannot run ConnectedComponents example: Could not allocate a slot on instance
> -----------------------------------------------------------------------------
>
>                 Key: FLINK-1952
>                 URL: https://issues.apache.org/jira/browse/FLINK-1952
>             Project: Flink
>          Issue Type: Bug
>          Components: Scheduler
>    Affects Versions: 0.9
>            Reporter: Robert Metzger
>            Priority: Blocker
>
> Steps to reproduce
> {code}
> ./bin/yarn-session.sh -n 350 
> {code}
> ... wait until they are connected ...
> {code}
> Number of connected TaskManagers changed to 266. Slots available: 266
> Number of connected TaskManagers changed to 323. Slots available: 323
> Number of connected TaskManagers changed to 334. Slots available: 334
> Number of connected TaskManagers changed to 343. Slots available: 343
> Number of connected TaskManagers changed to 350. Slots available: 350
> {code}
> Start CC
> {code}
> ./bin/flink run -p 350 ./examples/flink-java-examples-0.9-SNAPSHOT-ConnectedComponents.jar
> {code}
> ---> it runs
> Run KMeans, let it fail with 
> {code}
> Failed to deploy the task Map (Map at main(KMeans.java:100)) (1/350) - execution #0 to
slot SimpleSlot (2)(2)(0) - 182b7661ca9547a84591de940c47a200 - ALLOCATED/ALIVE: java.io.IOException:
Insufficient number of network buffers: required 350, but only 254 available. The total number
of network buffers is currently set to 2048. You can increase this number by setting the configuration
key 'taskmanager.network.numberOfBuffers'.
> {code}
> ... as expected.
> (I've waited for 10 minutes between the two submissions)
> Starting CC now will fail:
> {code}
> ./bin/flink run -p 350 ./examples/flink-java-examples-0.9-SNAPSHOT-ConnectedComponents.jar

> {code}
> Error message(s):
> {code}
> Caused by: java.lang.IllegalStateException: Could not schedule consumer vertex IterationHead(WorksetIteration
(Unnamed Delta Iteration)) (19/350)
> 	at org.apache.flink.runtime.executiongraph.Execution$3.call(Execution.java:479)
> 	at org.apache.flink.runtime.executiongraph.Execution$3.call(Execution.java:469)
> 	at akka.dispatch.Futures$$anonfun$future$1.apply(Future.scala:94)
> 	at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
> 	at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
> 	at scala.concurrent.impl.ExecutionContextImpl$$anon$3.exec(ExecutionContextImpl.scala:107)
> 	... 4 more
> Caused by: org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException:
Could not allocate a slot on instance 4a6d761cb084c32310ece1f849556faf @ cloud-19 - 1 slots
- URL: akka.tcp://flink@130.149.21.23:51400/user/taskmanager, as required by the co-location
constraint.
> 	at org.apache.flink.runtime.jobmanager.scheduler.Scheduler.scheduleTask(Scheduler.java:247)
> 	at org.apache.flink.runtime.jobmanager.scheduler.Scheduler.scheduleImmediately(Scheduler.java:110)
> 	at org.apache.flink.runtime.executiongraph.Execution.scheduleForExecution(Execution.java:262)
> 	at org.apache.flink.runtime.executiongraph.ExecutionVertex.scheduleForExecution(ExecutionVertex.java:436)
> 	at org.apache.flink.runtime.executiongraph.Execution$3.call(Execution.java:475)
> 	... 9 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message