spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "nigel (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-1764) EOF reached before Python server acknowledged
Date Mon, 25 Aug 2014 19:16:59 GMT

    [ https://issues.apache.org/jira/browse/SPARK-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109583#comment-14109583
] 

nigel commented on SPARK-1764:
------------------------------

Hi;
Sadly I moved jobs and I don't have a working Spark environment at the moment (I will be doing
some Spark work soon :-). I'll pass this on to the guys that are still there and get them
to confirm. 
Cheers

> EOF reached before Python server acknowledged
> ---------------------------------------------
>
>                 Key: SPARK-1764
>                 URL: https://issues.apache.org/jira/browse/SPARK-1764
>             Project: Spark
>          Issue Type: Bug
>          Components: Mesos, PySpark
>    Affects Versions: 1.0.0
>            Reporter: Bouke van der Bijl
>            Assignee: Davies Liu
>            Priority: Blocker
>              Labels: mesos, pyspark
>
> I'm getting "EOF reached before Python server acknowledged" while using PySpark on Mesos.
The error manifests itself in multiple ways. One is:
> 14/05/08 18:10:40 ERROR DAGSchedulerActorSupervisor: eventProcesserActor failed due to
the error EOF reached before Python server acknowledged; shutting down SparkContext
> And the other has a full stacktrace:
> 14/05/08 18:03:06 ERROR OneForOneStrategy: EOF reached before Python server acknowledged
> org.apache.spark.SparkException: EOF reached before Python server acknowledged
> 	at org.apache.spark.api.python.PythonAccumulatorParam.addInPlace(PythonRDD.scala:416)
> 	at org.apache.spark.api.python.PythonAccumulatorParam.addInPlace(PythonRDD.scala:387)
> 	at org.apache.spark.Accumulable.$plus$plus$eq(Accumulators.scala:71)
> 	at org.apache.spark.Accumulators$$anonfun$add$2.apply(Accumulators.scala:279)
> 	at org.apache.spark.Accumulators$$anonfun$add$2.apply(Accumulators.scala:277)
> 	at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772)
> 	at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
> 	at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
> 	at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226)
> 	at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
> 	at scala.collection.mutable.HashMap.foreach(HashMap.scala:98)
> 	at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771)
> 	at org.apache.spark.Accumulators$.add(Accumulators.scala:277)
> 	at org.apache.spark.scheduler.DAGScheduler.handleTaskCompletion(DAGScheduler.scala:818)
> 	at org.apache.spark.scheduler.DAGSchedulerEventProcessActor$$anonfun$receive$2.applyOrElse(DAGScheduler.scala:1204)
> 	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
> 	at akka.actor.ActorCell.invoke(ActorCell.scala:456)
> 	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
> 	at akka.dispatch.Mailbox.run(Mailbox.scala:219)
> 	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
> 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> This error causes the SparkContext to shutdown. I have not been able to reliably reproduce
this bug, it seems to happen randomly, but if you run enough tasks on a SparkContext it'll
hapen eventually



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message