spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wenchen Fan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-23253) Only write shuffle temporary index file when there is not an existing one
Date Sat, 01 Sep 2018 06:53:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-23253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599557#comment-16599557
] 

Wenchen Fan commented on SPARK-23253:
-------------------------------------

cc [~joshrosen] [~zsxwing] [~rxin@databricks.com]

> Only write shuffle temporary index file when there is not an existing one
> -------------------------------------------------------------------------
>
>                 Key: SPARK-23253
>                 URL: https://issues.apache.org/jira/browse/SPARK-23253
>             Project: Spark
>          Issue Type: Improvement
>          Components: Shuffle, Spark Core
>    Affects Versions: 2.2.1
>            Reporter: Kent Yao
>            Assignee: Kent Yao
>            Priority: Major
>             Fix For: 2.4.0
>
>
> Shuffle Index temporay file is used for atomic creating shuffle index file, it is not
needed when the index file already exists after another attempts of same task had it done.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message