flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: TM failure when deploying a large number of sources
Date Wed, 07 Oct 2015 13:44:34 GMT
I think the error message could have been better, though...

This actually warrants a JIRA issue...

On Wed, Oct 7, 2015 at 2:44 PM, Gyula Fóra <gyula.fora@gmail.com> wrote:

> Thanks!
>
> Yes, it was indeed a memory issue:
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:714)
> at org.apache.flink.runtime.taskmanager.Task.startTaskThread(Task.java:415)
> at
>
> org.apache.flink.runtime.taskmanager.TaskManager.submitTask(TaskManager.scala:904)
>
> I will just decrease the parallelism locally :)
>
> Cheers,
> Gyula
>
> Stephan Ewen <sewen@apache.org> ezt írta (időpont: 2015. okt. 7., Sze,
> 14:16):
>
> > Any further information from teh log?
> >
> > If you create so many tasks (8 x 80) on one machine, the JVM often has
> not
> > enough memory reserved for the stack space to create enough threads (1-2
> > threads per task)...
> >
> > On Wed, Oct 7, 2015 at 2:13 PM, Gyula Fóra <gyfora@apache.org> wrote:
> >
> > > Hey guys,
> > >
> > > I am writing a job which involves creating many different sources to
> read
> > > data from (in this case 80 sources wiht the parallelism of 8 each,
> > running
> > > locally on my mac). I cannot create less unfortunately.
> > >
> > > The problem is that the job fails while deploying the tasks with the
> > > following exception:
> > >
> > > java.lang.Exception: Failed to deploy the task to slot SimpleSlot
> > (1)(63) -
> > > eea7250ab5b368693e3c4f14fb94f86d @ localhost - 8 slots - URL:
> > > akka://flink/user/taskmanager_1 - ALLOCATED/ALIVE: Response was not of
> > type
> > > Acknowledge
> > > at
> > >
> > >
> >
> org.apache.flink.runtime.executiongraph.Execution$2.onComplete(Execution.java:392)
> > >
> > > at akka.dispatch.OnComplete.internal(Future.scala:247)
> > > at akka.dispatch.OnComplete.internal(Future.scala:244)
> > > at akka.dispatch.japi$CallbackBridge.apply(Future.scala:174)
> > > at akka.dispatch.japi$CallbackBridge.apply(Future.scala:171)
> > > at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
> > > at
> > >
> > >
> >
> scala.concurrent.impl.ExecutionContextImpl$anon$3.exec(ExecutionContextImpl.scala:107)
> > >
> > > at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> > > at
> > >
> > >
> >
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> > >
> > > at
> > scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> > > at
> > >
> > >
> >
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> > >
> > > Any idea what might cause this?
> > >
> > > Cheers,
> > > Gyula
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message