spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Wendell (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-1155) Clean up and document use of SparkEnv
Date Sun, 30 Mar 2014 04:13:15 GMT

     [ https://issues.apache.org/jira/browse/SPARK-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Patrick Wendell updated SPARK-1155:
-----------------------------------

    Reporter: Patrick Wendell  (was: Patrick Cogan)

> Clean up and document use of SparkEnv
> -------------------------------------
>
>                 Key: SPARK-1155
>                 URL: https://issues.apache.org/jira/browse/SPARK-1155
>             Project: Apache Spark
>          Issue Type: New Feature
>          Components: Spark Core
>            Reporter: Patrick Wendell
>            Assignee: Andrew Or
>             Fix For: 1.0.0
>
>
> We should provide better documentation explaining what SparkEnv is and why it needs to
be thread local (basically, to allow it to be accessed inside of closures on executors). Also,
in cases where SparkEnv is being accessed on the driver we should access it through the associated
SparkContext rather than through the thread local. Finally, we should see if it's possible
to just remove this as a thread local and instead make it a static singleton that the exeucutor
sets once. This last thing might not be possible if, under certain code paths, this is used
on the driver.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message