spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Saisai Shao (JIRA)" <>
Subject [jira] [Commented] (SPARK-20898) spark.blacklist.killBlacklistedExecutors doesn't work in YARN
Date Wed, 31 May 2017 06:44:04 GMT


Saisai Shao commented on SPARK-20898:

[~tgraves], I addressed this issue in 

> spark.blacklist.killBlacklistedExecutors doesn't work in YARN
> -------------------------------------------------------------
>                 Key: SPARK-20898
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 2.2.0
>            Reporter: Thomas Graves
> I was trying out the new spark.blacklist.killBlacklistedExecutors on YARN but it doesn't
appear to work.  Everytime I get:
> 17/05/26 16:28:12 WARN BlacklistTracker: Not attempting to kill blacklisted executor
id 4 since allocation client is not defined
> Even though dynamic allocation is on.  Taking a quick look, I think the way it creates
the blacklisttracker and passes the allocation client is wrong. The scheduler backend is 
>  not set yet so it never passes the allocation client to the blacklisttracker correctly.
 Thus it will never kill.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message