spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ankur Srivastava <ankur.srivast...@gmail.com>
Subject Re: how to debug this kind of error, e.g. "lost executor"?
Date Thu, 05 Feb 2015 16:56:52 GMT
Li, I cannot tell you the reason for this exception but have seen these
kind of errors when using HASH based shuffle manager (which is default)
until v 1.2. Try the SORT shuffle manager.

Hopefully that will help

Thanks
Ankur

Anyone has idea on where I can find the detailed log of that lost
executor(why it was lost)?

Thanks in advance!





On 05 Feb 2015, at 16:14, Yifan LI <iamyifanli@gmail.com> wrote:

Hi,

I am running a heavy memory/cpu overhead graphx application, I think the
memory is sufficient and set RDDs’ StorageLevel using MEMORY_AND_DISK.

But I found there were some tasks failed due to following errors:

java.io.FileNotFoundException:
/data/spark/local/spark-local-20150205151711-9700/09/rdd_3_275 (No files or
folders of this type)

ExecutorLostFailure (executor 11 lost)


So, finally that stage failed:

org.apache.spark.shuffle.FetchFailedException:
java.io.FileNotFoundException:
/data/spark/local/spark-local-20150205151711-587a/16/shuffle_11_219_0.index


Anyone has points? Where I can get more details for this issue?


Best,
Yifan LI

Mime
View raw message