spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Gummelt (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-4899) Support Mesos features: roles and checkpoints
Date Wed, 24 May 2017 18:19:04 GMT

    [ https://issues.apache.org/jira/browse/SPARK-4899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16023383#comment-16023383
] 

Michael Gummelt commented on SPARK-4899:
----------------------------------------

Thanks Kamal.  I responded to the thread, which I'll copy here:

bq. Restarting the agent without checkpointing enabled will kill the executor, but that still
shouldn't cause the Spark job to fail, since Spark jobs should tolerate executor failures.

So I'm fine with adding checkpointing support, but I'm not sure it actually solves any problem.

> Support Mesos features: roles and checkpoints
> ---------------------------------------------
>
>                 Key: SPARK-4899
>                 URL: https://issues.apache.org/jira/browse/SPARK-4899
>             Project: Spark
>          Issue Type: New Feature
>          Components: Mesos
>    Affects Versions: 1.2.0
>            Reporter: Andrew Ash
>
> Inspired by https://github.com/apache/spark/pull/60
> Mesos has two features that would be nice for Spark to take advantage of:
> 1. Roles -- a way to specify ACLs and priorities for users
> 2. Checkpoints -- a way to restart a failed Mesos slave without losing all the work that
was happening on the box
> Some of these may require a Mesos upgrade past our current 0.18.1



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message