spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fengyu Cao (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-26389) temp checkpoint folder at executor should be deleted on graceful shutdown
Date Tue, 18 Dec 2018 03:33:01 GMT

     [ https://issues.apache.org/jira/browse/SPARK-26389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Fengyu Cao updated SPARK-26389:
-------------------------------
    Description: 
`spark-submit --master mesos://<mesos>- -conf spark.streaming.stopGracefullyOnShutdown=true
<structured streaming framework>`

CTRL-C, framework shutdown

18/12/18 10:27:36 ERROR MicroBatchExecution: Query [id = f512e17a-df88-4414-a5cd-a23550cf1e7f,
runId = 24d99723-8d61-48c0-beab-af432f7a19d3] terminated with error
 org.apache.spark.SparkException: Writing job aborted.

/tmp/temporary-<uuid> on executor not deleted due to `org.apache.spark.SparkException:
Writing job aborted.`, and this temp checkpoint can't used to recovery.

 

  was:
spark-submit --master mesos://<mesos> --conf spark.streaming.stopGracefullyOnShutdown=true
<structured streaming framework>

CTRL-C, framework shutdown

18/12/18 10:27:36 ERROR MicroBatchExecution: Query [id = f512e17a-df88-4414-a5cd-a23550cf1e7f,
runId = 24d99723-8d61-48c0-beab-af432f7a19d3] terminated with error
org.apache.spark.SparkException: Writing job aborted.

/tmp/temporary-<uuid> on executor not deleted due to `org.apache.spark.SparkException:
Writing job aborted.`, and this temp checkpoint can't used to recovery.

 


> temp checkpoint folder at executor should be deleted on graceful shutdown
> -------------------------------------------------------------------------
>
>                 Key: SPARK-26389
>                 URL: https://issues.apache.org/jira/browse/SPARK-26389
>             Project: Spark
>          Issue Type: Bug
>          Components: Structured Streaming
>    Affects Versions: 2.4.0
>            Reporter: Fengyu Cao
>            Priority: Major
>
> `spark-submit --master mesos://<mesos>- -conf spark.streaming.stopGracefullyOnShutdown=true
<structured streaming framework>`
> CTRL-C, framework shutdown
> 18/12/18 10:27:36 ERROR MicroBatchExecution: Query [id = f512e17a-df88-4414-a5cd-a23550cf1e7f,
runId = 24d99723-8d61-48c0-beab-af432f7a19d3] terminated with error
>  org.apache.spark.SparkException: Writing job aborted.
> /tmp/temporary-<uuid> on executor not deleted due to `org.apache.spark.SparkException:
Writing job aborted.`, and this temp checkpoint can't used to recovery.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message