spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Imran Rashid (JIRA)" <j...@apache.org>
Subject [jira] [Reopened] (SPARK-10582) using dynamic-executor-allocation, if AM failed. the new AM will be started. But the new AM does not allocate executors to dirver
Date Thu, 19 Nov 2015 19:41:11 GMT

     [ https://issues.apache.org/jira/browse/SPARK-10582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Imran Rashid reopened SPARK-10582:
----------------------------------

> using dynamic-executor-allocation, if AM failed. the new AM will be started. But the
new AM does not allocate executors to dirver
> ---------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-10582
>                 URL: https://issues.apache.org/jira/browse/SPARK-10582
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 1.4.1, 1.5.1
>            Reporter: KaiXinXIaoLei
>
> During running tasks, when the total number of executors is the value of spark.dynamicAllocation.maxExecutors
and the AM is failed. Then a new AM restarts. Because in ExecutorAllocationManager, the total
number of executors does not changed, driver does not send RequestExecutors to AM to ask executors.
Then the total number of executors is the value of spark.dynamicAllocation.initialExecutors
. So the total number of executors in driver and AM is different.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message