From reviews-return-1014402-archive-asf-public=cust-asf.ponee.io@spark.apache.org Wed Jan 15 23:48:14 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id A71B218065E for ; Thu, 16 Jan 2020 00:48:14 +0100 (CET) Received: (qmail 51875 invoked by uid 500); 15 Jan 2020 23:48:14 -0000 Mailing-List: contact reviews-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list reviews@spark.apache.org Received: (qmail 51863 invoked by uid 99); 15 Jan 2020 23:48:14 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Jan 2020 23:48:14 +0000 From: GitBox To: reviews@spark.apache.org Subject: [GitHub] [spark] ifilonenko commented on a change in pull request #26586: [SPARK-29950][k8s] Blacklist deleted executors in K8S with dynamic allocation. Message-ID: <157913209396.13252.7239837825862824868.gitbox@gitbox.apache.org> References: In-Reply-To: Date: Wed, 15 Jan 2020 23:48:13 -0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit ifilonenko commented on a change in pull request #26586: [SPARK-29950][k8s] Blacklist deleted executors in K8S with dynamic allocation. URL: https://github.com/apache/spark/pull/26586#discussion_r367164947 ########## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ########## @@ -207,15 +207,14 @@ class CoarseGrainedSchedulerBackend(scheduler: TaskSchedulerImpl, val rpcEnv: Rp case RegisterExecutor(executorId, executorRef, hostname, cores, logUrls, attributes, resources) => if (executorDataMap.contains(executorId)) { - executorRef.send(RegisterExecutorFailed("Duplicate executor ID: " + executorId)) - context.reply(true) - } else if (scheduler.nodeBlacklist.contains(hostname)) { + context.sendFailure(new IllegalStateException(s"Duplicate executor ID: $executorId")) + } else if (scheduler.nodeBlacklist.contains(hostname) || + isBlacklisted(executorId, hostname)) { // If the cluster manager gives us an executor on a blacklisted node (because it // already started allocating those resources before we informed it of our blacklist, // or if it ignored our blacklist), then we reject that executor immediately. logInfo(s"Rejecting $executorId as it has been blacklisted.") - executorRef.send(RegisterExecutorFailed(s"Executor is blacklisted: $executorId")) - context.reply(true) + context.sendFailure(new IllegalStateException(s"Executor is blacklisted: $executorId")) Review comment: Is there a reason we would rather `sendFailure(_)` instead of the exiting the executor with a `RegisterExecutorFailed` message? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org For additional commands, e-mail: reviews-help@spark.apache.org