ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Valentin Kulichenko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-7092) Deprecate embedded mode in IgniteRDD
Date Tue, 26 Dec 2017 18:01:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-7092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16303948#comment-16303948

Valentin Kulichenko commented on IGNITE-7092:

[~dmagda], standalone mode assumes that there is a standalone Ignite cluster. Embedded clients
are started to connect to that cluster.

> Deprecate embedded mode in IgniteRDD
> ------------------------------------
>                 Key: IGNITE-7092
>                 URL: https://issues.apache.org/jira/browse/IGNITE-7092
>             Project: Ignite
>          Issue Type: Improvement
>          Components: spark
>    Affects Versions: 2.3
>            Reporter: Valentin Kulichenko
>            Assignee: Valentin Kulichenko
>             Fix For: 2.4
> Currently we claim to support IgniteRDD in two modes: standalone and embedded. Standalone
means there is a separately running Ignite cluster, and Spark start client node(s) to interact
with it. In embedded node everything runs within Spark, including Ignite server nodes that
are started embedded into Spark executors.
> The latter case doesn't really work, mainly because the lifecycle of Spark executors
is not very predictable - Spark can start and stop them while application is running. In case
Ignite cluster is used to store data (which is usually the case), this causes unnecessary
rebalancing or even unexpected data loss.
> I propose to deprecate and eventually discontinue the embedded mode. Luckily, standalone
mode is the default one, so we can simply print out a clear warning if one switches to embedded
mode, and also mention this in the docs.

This message was sent by Atlassian JIRA

View raw message