Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 4E7B4200D8A for ; Wed, 27 Dec 2017 00:13:12 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 4CC72160C13; Tue, 26 Dec 2017 23:13:12 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 9B9BE160C32 for ; Wed, 27 Dec 2017 00:13:11 +0100 (CET) Received: (qmail 51714 invoked by uid 500); 26 Dec 2017 23:13:10 -0000 Mailing-List: contact issues-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list issues@ignite.apache.org Received: (qmail 51701 invoked by uid 99); 26 Dec 2017 23:13:10 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Dec 2017 23:13:10 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id E7CFF180624 for ; Tue, 26 Dec 2017 23:13:09 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id sCeiKzA80qw3 for ; Tue, 26 Dec 2017 23:13:09 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id E8F455F474 for ; Tue, 26 Dec 2017 23:13:08 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id E3843E0F61 for ; Tue, 26 Dec 2017 23:13:06 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 22AB4240F2 for ; Tue, 26 Dec 2017 23:13:03 +0000 (UTC) Date: Tue, 26 Dec 2017 23:13:03 +0000 (UTC) From: "Valentin Kulichenko (JIRA)" To: issues@ignite.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Resolved] (IGNITE-7092) Deprecate embedded mode in IgniteRDD MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 26 Dec 2017 23:13:12 -0000 [ https://issues.apache.org/jira/browse/IGNITE-7092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentin Kulichenko resolved IGNITE-7092. ----------------------------------------- Resolution: Fixed > Deprecate embedded mode in IgniteRDD > ------------------------------------ > > Key: IGNITE-7092 > URL: https://issues.apache.org/jira/browse/IGNITE-7092 > Project: Ignite > Issue Type: Improvement > Components: spark > Affects Versions: 2.3 > Reporter: Valentin Kulichenko > Assignee: Valentin Kulichenko > Fix For: 2.4 > > > Currently we claim to support IgniteRDD in two modes: standalone and embedded. Standalone means there is a separately running Ignite cluster, and Spark start client node(s) to interact with it. In embedded node everything runs within Spark, including Ignite server nodes that are started embedded into Spark executors. > The latter case doesn't really work, mainly because the lifecycle of Spark executors is not very predictable - Spark can start and stop them while application is running. In case Ignite cluster is used to store data (which is usually the case), this causes unnecessary rebalancing or even unexpected data loss. > I propose to deprecate and eventually discontinue the embedded mode. Luckily, standalone mode is the default one, so we can simply print out a clear warning if one switches to embedded mode, and also mention this in the docs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)