spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcelo Vanzin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-11074) Attempt to automatically create event log directory if it doesn't already exist
Date Tue, 13 Oct 2015 03:30:05 GMT

    [ https://issues.apache.org/jira/browse/SPARK-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14954304#comment-14954304
] 

Marcelo Vanzin commented on SPARK-11074:
----------------------------------------

The shuffle files are created under an application-specific root directory that is created
by the cluster manager. That directory belongs to the application, and Spark is free to do
whatever it wants with it. No other application depends on it, unlike the event log directory.

> Attempt to automatically create event log directory if it doesn't already exist
> -------------------------------------------------------------------------------
>
>                 Key: SPARK-11074
>                 URL: https://issues.apache.org/jira/browse/SPARK-11074
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 1.5.1
>            Reporter: Kay Ousterhout
>            Assignee: Kay Ousterhout
>            Priority: Minor
>
> Right now, if the log where event logs are to be placed (spark.eventLog.dir) doesn't
already exist, the SparkContext fails to start.  We should try to automatically create this
directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message