spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-4585) Spark dynamic executor allocation shouldn't use maxExecutors as initial number
Date Wed, 14 Jan 2015 21:30:35 GMT

    [ https://issues.apache.org/jira/browse/SPARK-4585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14277716#comment-14277716
] 

Apache Spark commented on SPARK-4585:
-------------------------------------

User 'sryza' has created a pull request for this issue:
https://github.com/apache/spark/pull/4051

> Spark dynamic executor allocation shouldn't use maxExecutors as initial number
> ------------------------------------------------------------------------------
>
>                 Key: SPARK-4585
>                 URL: https://issues.apache.org/jira/browse/SPARK-4585
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core, YARN
>    Affects Versions: 1.1.0
>            Reporter: Chengxiang Li
>
> With SPARK-3174, one can configure a minimum and maximum number of executors for a Spark
application on Yarn. However, the application always starts with the maximum. It seems more
reasonable, at least for Hive on Spark, to start from the minimum and scale up as needed up
to the maximum.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message