Return-Path: X-Original-To: apmail-ignite-commits-archive@minotaur.apache.org Delivered-To: apmail-ignite-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 15CC6187D1 for ; Tue, 11 Aug 2015 08:47:11 +0000 (UTC) Received: (qmail 59653 invoked by uid 500); 11 Aug 2015 08:46:58 -0000 Delivered-To: apmail-ignite-commits-archive@ignite.apache.org Received: (qmail 59622 invoked by uid 500); 11 Aug 2015 08:46:58 -0000 Mailing-List: contact commits-help@ignite.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.incubator.apache.org Delivered-To: mailing list commits@ignite.incubator.apache.org Received: (qmail 59612 invoked by uid 99); 11 Aug 2015 08:46:58 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Aug 2015 08:46:58 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id D09D21A9D03 for ; Tue, 11 Aug 2015 08:46:57 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.791 X-Spam-Level: X-Spam-Status: No, score=0.791 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id VdNWy58-0UuX for ; Tue, 11 Aug 2015 08:46:43 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with SMTP id F3E3B42BF3 for ; Tue, 11 Aug 2015 08:46:33 +0000 (UTC) Received: (qmail 57057 invoked by uid 99); 11 Aug 2015 08:46:32 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Aug 2015 08:46:32 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 45B63E03CE; Tue, 11 Aug 2015 08:46:32 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: ntikhonov@apache.org To: commits@ignite.incubator.apache.org Date: Tue, 11 Aug 2015 08:47:01 -0000 Message-Id: <761acfd7859649a1b37611804de99501@git.apache.org> In-Reply-To: <2f17b8f613b944ed9c530db35bf76e1d@git.apache.org> References: <2f17b8f613b944ed9c530db35bf76e1d@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [31/50] incubator-ignite git commit: # ignite-1198 # ignite-1198 Project: http://git-wip-us.apache.org/repos/asf/incubator-ignite/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-ignite/commit/2173b0e1 Tree: http://git-wip-us.apache.org/repos/asf/incubator-ignite/tree/2173b0e1 Diff: http://git-wip-us.apache.org/repos/asf/incubator-ignite/diff/2173b0e1 Branch: refs/heads/ignite-1108 Commit: 2173b0e1500d3dbc198a610042fca9b92d4f7de7 Parents: 023ffe0 Author: Alexey Goncharuk Authored: Wed Aug 5 14:05:07 2015 -0700 Committer: Alexey Goncharuk Committed: Wed Aug 5 14:25:13 2015 -0700 ---------------------------------------------------------------------- .../org/apache/ignite/spark/IgniteContext.scala | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/2173b0e1/modules/spark/src/main/scala/org/apache/ignite/spark/IgniteContext.scala ---------------------------------------------------------------------- diff --git a/modules/spark/src/main/scala/org/apache/ignite/spark/IgniteContext.scala b/modules/spark/src/main/scala/org/apache/ignite/spark/IgniteContext.scala index 5267244a..6e48017 100644 --- a/modules/spark/src/main/scala/org/apache/ignite/spark/IgniteContext.scala +++ b/modules/spark/src/main/scala/org/apache/ignite/spark/IgniteContext.scala @@ -19,7 +19,8 @@ package org.apache.ignite.spark import org.apache.ignite.internal.IgnitionEx -import org.apache.ignite.{Ignition, Ignite} +import org.apache.ignite.internal.util.IgniteUtils +import org.apache.ignite.{IgniteSystemProperties, Ignition, Ignite} import org.apache.ignite.configuration.{CacheConfiguration, IgniteConfiguration} import org.apache.spark.{Logging, SparkContext} import org.apache.spark.sql.SQLContext @@ -41,8 +42,12 @@ class IgniteContext[K, V]( private val cfgClo = new Once(cfgF) + private val igniteHome = IgniteUtils.getIgniteHome + if (!client) { - val workers = sparkContext.getExecutorStorageStatus.length - 1 + // Get required number of executors with default equals to number of available executors. + val workers = sparkContext.getConf.getInt("spark.executor.instances", + sparkContext.getExecutorStorageStatus.length) if (workers <= 0) throw new IllegalStateException("No Spark executors found to start Ignite nodes.") @@ -125,6 +130,16 @@ class IgniteContext[K, V]( * @return Ignite instance. */ def ignite(): Ignite = { + val home = IgniteUtils.getIgniteHome + + if (home == null && igniteHome != null) { + logInfo("Setting IGNITE_HOME from driver not as it is not available on this worker: " + igniteHome) + + IgniteUtils.nullifyHomeDirectory() + + System.setProperty(IgniteSystemProperties.IGNITE_HOME, igniteHome) + } + val igniteCfg = cfgClo() try {