Mailing-List: contact issues-help@spark.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 17 Jul 2015 22:59:04 +0000 (UTC)
From: "Josh Rosen (JIRA)" <jira@apache.org>
To: issues@spark.apache.org
Message-ID: <JIRA.12846004.1437170982000.223387.1437173944589@Atlassian.JIRA>
In-Reply-To: <JIRA.12846004.1437170982000@Atlassian.JIRA>
References: <JIRA.12846004.1437170982000@Atlassian.JIRA>
 <JIRA.12846004.1437170982798@arcas>
Subject: [jira] [Updated] (SPARK-9144) Remove
 DAGScheduler.runLocallyWithinThread and spark.localExecution.enabled
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/SPARK-9144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Josh Rosen updated SPARK-9144:
------------------------------
    Issue Type: Improvement  (was: New Feature)

> Remove DAGScheduler.runLocallyWithinThread and spark.localExecution.enabled
> ---------------------------------------------------------------------------
>
>                 Key: SPARK-9144
>                 URL: https://issues.apache.org/jira/browse/SPARK-9144
>             Project: Spark
>          Issue Type: Improvement
>          Components: Scheduler, Spark Core
>            Reporter: Josh Rosen
>            Assignee: Josh Rosen
>
> Spark has an option called {{spark.localExecution.enabled}}; according to the docs:
> {quote}
> Enables Spark to run certain jobs, such as first() or take() on the driver, without sending tasks to the cluster. This can make certain jobs execute very quickly, but may require shipping a whole partition of data to the driver.
> {quote}
> This feature ends up adding quite a bit of complexity to DAGScheduler, especially in the {{runLocallyWithinThread}} method, but as far as I know nobody uses this feature (I searched the mailing list and haven't seen any recent mentions of the configuration nor stacktraces including the runLocally method).  As a step towards scheduler complexity reduction, I propose that we remove this feature and all code related to it for Spark 1.5. 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org