Return-Path: X-Original-To: apmail-myriad-commits-archive@minotaur.apache.org Delivered-To: apmail-myriad-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6736F18E1A for ; Mon, 19 Oct 2015 22:22:14 +0000 (UTC) Received: (qmail 72641 invoked by uid 500); 19 Oct 2015 22:22:14 -0000 Delivered-To: apmail-myriad-commits-archive@myriad.apache.org Received: (qmail 72611 invoked by uid 500); 19 Oct 2015 22:22:14 -0000 Mailing-List: contact commits-help@myriad.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@myriad.incubator.apache.org Delivered-To: mailing list commits@myriad.incubator.apache.org Received: (qmail 72602 invoked by uid 99); 19 Oct 2015 22:22:14 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 19 Oct 2015 22:22:14 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id CB115C04B9 for ; Mon, 19 Oct 2015 22:22:13 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.771 X-Spam-Level: X-Spam-Status: No, score=0.771 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id EL0leHrd-95X for ; Mon, 19 Oct 2015 22:22:01 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with SMTP id 729FE43A17 for ; Mon, 19 Oct 2015 22:22:01 +0000 (UTC) Received: (qmail 72482 invoked by uid 99); 19 Oct 2015 22:22:01 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 19 Oct 2015 22:22:01 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id B732EE0522; Mon, 19 Oct 2015 22:22:00 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: smarella@apache.org To: commits@myriad.incubator.apache.org Message-Id: <900a9b42e08543b995fd45fcad93e252@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: =?utf-8?q?incubator-myriad_git_commit=3A_This_commit_address_Myria?= =?utf-8?q?d-135=2C_it_is_possible_an_offer_may_not_conta=E2=80=A6_=E2=80=A6?= =?utf-8?q?in_a_particular_resource_type=2C_in_which_case_an_NPE_occured=2E?= Date: Mon, 19 Oct 2015 22:22:00 +0000 (UTC) Repository: incubator-myriad Updated Branches: refs/heads/master 981c4d24d -> fceaf7301 This commit address Myriad-135, it is possible an offer may not conta… …in a particular resource type, in which case an NPE occured. In reviewing the code it was also discovered Myriad was only taking the first resource type offered instead of all resources of a type offered. This meant if Myriad was running with role "Hadoop" and got an offer with cpu(Hadoop): 2 and cpu(*): 1, it would ignore the 1 cpu it could've taken from the * role. This closes: #3 Review: https://github.com/apache/incubator-myriad/pull/3 Project: http://git-wip-us.apache.org/repos/asf/incubator-myriad/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-myriad/commit/fceaf730 Tree: http://git-wip-us.apache.org/repos/asf/incubator-myriad/tree/fceaf730 Diff: http://git-wip-us.apache.org/repos/asf/incubator-myriad/diff/fceaf730 Branch: refs/heads/master Commit: fceaf73013a38ba9e43e7f4ef1a258fbb96f09f1 Parents: 981c4d2 Author: DarinJ Authored: Wed Sep 30 22:02:51 2015 -0400 Committer: Santosh Marella Committed: Mon Oct 19 15:20:38 2015 -0700 ---------------------------------------------------------------------- .../handlers/ResourceOffersEventHandler.java | 49 +++++++++++++------- 1 file changed, 31 insertions(+), 18 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-myriad/blob/fceaf730/myriad-scheduler/src/main/java/com/ebay/myriad/scheduler/event/handlers/ResourceOffersEventHandler.java ---------------------------------------------------------------------- diff --git a/myriad-scheduler/src/main/java/com/ebay/myriad/scheduler/event/handlers/ResourceOffersEventHandler.java b/myriad-scheduler/src/main/java/com/ebay/myriad/scheduler/event/handlers/ResourceOffersEventHandler.java index de5211d..1ca9ec0 100644 --- a/myriad-scheduler/src/main/java/com/ebay/myriad/scheduler/event/handlers/ResourceOffersEventHandler.java +++ b/myriad-scheduler/src/main/java/com/ebay/myriad/scheduler/event/handlers/ResourceOffersEventHandler.java @@ -64,6 +64,12 @@ public class ResourceOffersEventHandler implements EventHandler results = new HashMap(5); + //Assign default values to avoid NPE + results.put(RESOURCES_CPU_KEY, Double.valueOf(0.0)); + results.put(RESOURCES_MEM_KEY, Double.valueOf(0.0)); + results.put(RESOURCES_DISK_KEY, Double.valueOf(0.0)); + results.put(RESOURCES_PORTS_KEY, Integer.valueOf(0)); for (Resource resource : offer.getResourcesList()) { if (resourceEvaluators.containsKey(resource.getName())) { resourceEvaluators.get(resource.getName()).eval(resource, results); } else { LOGGER.warn("Ignoring unknown resource type: {}", - resource.getName()); + resource.getName()); } } - double cpus = (Double) results.get("cpus"); - double mem = (Double) results.get("mem"); - int ports = (Integer) results.get("ports"); + double cpus = (Double) results.get(RESOURCES_CPU_KEY); + double mem = (Double) results.get(RESOURCES_MEM_KEY); + int ports = (Integer) results.get(RESOURCES_PORTS_KEY); - checkResource(cpus < 0, "cpus"); - checkResource(mem < 0, "mem"); - checkResource(ports < 0, "port"); + checkResource(cpus <= 0, RESOURCES_CPU_KEY); + checkResource(mem <= 0, RESOURCES_MEM_KEY); + checkResource(ports <= 0, RESOURCES_PORTS_KEY); return checkAggregates(offer, taskToLaunch, ports, cpus, mem); } @@ -243,7 +254,7 @@ public class ResourceOffersEventHandler implements EventHandler(4); - resourceEvaluators.put("cpus", new EvalResources() { + resourceEvaluators.put(RESOURCES_CPU_KEY, new EvalResources() { public void eval(Resource resource, Map results) { - results.put("cpus", scalarToDouble(resource, "cpus")); + results.put(RESOURCES_CPU_KEY, (Double) results.get(RESOURCES_CPU_KEY) + + scalarToDouble(resource, RESOURCES_CPU_KEY)); } }); - resourceEvaluators.put("mem", new EvalResources() { + resourceEvaluators.put(RESOURCES_MEM_KEY, new EvalResources() { public void eval(Resource resource, Map results) { - results.put("mem", scalarToDouble(resource, "mem")); + results.put(RESOURCES_MEM_KEY, (Double) results.get(RESOURCES_MEM_KEY) + + scalarToDouble(resource, RESOURCES_MEM_KEY)); } }); - resourceEvaluators.put("disk", new EvalResources() { + resourceEvaluators.put(RESOURCES_DISK_KEY, new EvalResources() { public void eval(Resource resource, Map results) { } }); - resourceEvaluators.put("ports", new EvalResources() { + resourceEvaluators.put(RESOURCES_PORTS_KEY, new EvalResources() { public void eval(Resource resource, Map results) { int ports = 0; if (resource.getType().equals(Value.Type.RANGES)) { @@ -280,13 +293,13 @@ public class ResourceOffersEventHandler implements EventHandler