ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stuart Macdonald (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (IGNITE-9180) IgniteSparkSession Should Copy State on cloneSession()
Date Sun, 19 Aug 2018 13:49:00 GMT

     [ https://issues.apache.org/jira/browse/IGNITE-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Stuart Macdonald reassigned IGNITE-9180:
----------------------------------------

    Assignee: Stuart Macdonald

> IgniteSparkSession Should Copy State on cloneSession()
> ------------------------------------------------------
>
>                 Key: IGNITE-9180
>                 URL: https://issues.apache.org/jira/browse/IGNITE-9180
>             Project: Ignite
>          Issue Type: Bug
>          Components: spark
>    Affects Versions: 2.6
>            Reporter: Stuart Macdonald
>            Assignee: Stuart Macdonald
>            Priority: Major
>             Fix For: 2.7
>
>
> The IgniteSparkSession class extends SparkSession and overrides the cloneSession() method.
The contract for cloneSession() explicitly states that it should clone all state (ie. the
sharedState and sessionState fields), however the IgniteSparkSession implementation doesn't clone its
state fields.
> This has the side-effect of breaking stateful operations for anything which uses cloneSession(),
for example a Spark streaming job will not be able to use cached data across streaming
microbatches, which is a significant issue for such applications.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message