spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GraceH <...@git.apache.org>
Subject [GitHub] spark pull request: Return "false" is nothing to kill in killExecu...
Date Wed, 18 Nov 2015 07:08:16 GMT
GitHub user GraceH opened a pull request:

    https://github.com/apache/spark/pull/9796

    Return "false" is nothing to kill in killExecutors

    In discussion (SPARK-9552), we proposed a force kill in `killExecutors`. But if there
is nothing to kill, it will return back with true (acknowledgement). And then, it causes the
certain executor(s) (which is not eligible to kill) adding to pendingToRemove list for further
actions. 
    
    In this patch, we'd like to change the return semantics. If there is nothing to kill,
we will return "false". and therefore  all those non-eligible executors won't be added to
the pendingToRemove list. 
    
    @vanzin @andrewor14 As the follow up of PR#7888, please let me know your comments.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/GraceH/spark emptyPendingToRemove

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/9796.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #9796
    
----
commit c23f887b62a75415bab74036e78d03b92b1a5541
Author: Grace <jie.huang@intel.com>
Date:   2015-10-28T15:04:37Z

    rebase to master branch

commit dc660f63c416c300bd3da48c6a0b9442633313fc
Author: Grace <jie.huang@intel.com>
Date:   2015-11-10T07:10:09Z

    change the task number to count; change the WithLoads to WithTaskCount

commit 27faa6b70e8332f6f70359739cb417be5be7e31e
Author: Grace <jie.huang@intel.com>
Date:   2015-11-10T07:14:39Z

    keep sparkcontext public API un-changed

commit 8774124e07666d337501d0e79bc27bbb35d78b74
Author: Grace <jie.huang@intel.com>
Date:   2015-11-10T12:50:46Z

    fix compile issue

commit 946ed7e0966b10a706372200551345304d2c8089
Author: Grace <jie.huang@intel.com>
Date:   2015-11-10T13:04:08Z

    fix compile issue

commit feefbfef79e396df0187ae947e1b39c80e810082
Author: Grace <jie.huang@intel.com>
Date:   2015-11-11T07:10:41Z

    keep public API

commit fa3c88ea1033b569b853e7a758587bf84323bd6c
Author: Grace <jie.huang@intel.com>
Date:   2015-11-11T07:12:32Z

    keep public API

commit cb78e5605679bad667e5a93ed24a1d280827d121
Author: Grace <jie.huang@intel.com>
Date:   2015-11-11T07:14:05Z

    remove unnecessary comments

commit 5bcfd8148af20c43a993dfaa5a90597070a7c343
Author: Grace <jie.huang@intel.com>
Date:   2015-11-11T07:15:35Z

    clean code

commit 01c236ad3cb435c8b63f8be59c3f5d099b797cf3
Author: Grace <jie.huang@intel.com>
Date:   2015-11-11T07:16:24Z

    clean code

commit 2108dbfdfa962b40ffabf8874a364e8bf1009f93
Author: Grace <jie.huang@intel.com>
Date:   2015-11-12T13:53:54Z

    refine code

commit c0a1d549e84b110e13e06810a9b64da1f5b3230d
Author: Grace <jie.huang@intel.com>
Date:   2015-11-12T14:00:12Z

    remove unnecessary imports

commit 4b1959f6c7d393f34e145aed5eca3ae28e7b7a83
Author: Grace <jie.huang@intel.com>
Date:   2015-11-12T14:05:24Z

    set sc.killExecutor as force = true

commit 0293d8241df79b9c60ebefb6e18c89456be30547
Author: Grace <jie.huang@intel.com>
Date:   2015-11-12T14:20:44Z

    use force = false to do the unittest

commit 342a59d34e76c55f2fc306b45d8510c74cf73af8
Author: Grace <jie.huang@intel.com>
Date:   2015-11-13T03:32:06Z

    refine the unit test & change semantics for force == true only

commit 806a64d8e360e4589f4ad1569cc8c6f379e5987b
Author: Grace <jie.huang@intel.com>
Date:   2015-11-13T03:43:49Z

    refine the unit test & change semantics for force == true only

commit 4ce0ec06d79d2d6a5c68d0415c25c41d19f34736
Author: Grace <jie.huang@intel.com>
Date:   2015-11-13T03:51:48Z

    remove unnecessary imports

commit 0000551cd2860a85a6895b8c0a3b1815a9a7f108
Author: Grace <jie.huang@intel.com>
Date:   2015-11-13T04:55:42Z

    fix checkstyle issue

commit c44ef8714c6cde70404043e065e81330746f7881
Author: Andrew Or <andrew@databricks.com>
Date:   2015-11-13T18:51:48Z

    Suggestions

commit d3f51dbfa4d0b9e416c56a69383d19da2699d478
Author: Andrew Or <andrew@databricks.com>
Date:   2015-11-13T23:02:51Z

    Clean up state in ExecutorAllocationManager

commit 0daeb5a70cea3aa3d0a34189fcaee6fef59d578e
Author: Jie Huang <jie.huang@intel.com>
Date:   2015-11-17T00:55:20Z

    Merge pull request #1 from andrewor14/pr-7888-suggestions
    
    Suggestions for PR 7888

commit 1938e6158838bb1f21884c3d3eba9f0b8ffabe05
Author: Grace <jie.huang@intel.com>
Date:   2015-11-17T08:29:05Z

    fix unittest issue

commit 589083b8d9208cb1044556d0571f4dfd71284263
Author: Grace <jie.huang@intel.com>
Date:   2015-11-18T07:05:00Z

    return false is nothing to kill in killExecutors

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message