spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <>
Subject [jira] [Commented] (SPARK-12260) Graceful Shutdown with In-Memory State
Date Sun, 13 Dec 2015 11:49:46 GMT


Sean Owen commented on SPARK-12260:

I'm asking what you can't do with updateStateByKey that you need to do?
This sounds pretty app-specific, and something you can maintain in your own app or repo to

> Graceful Shutdown with In-Memory State
> --------------------------------------
>                 Key: SPARK-12260
>                 URL:
>             Project: Spark
>          Issue Type: New Feature
>          Components: Streaming
>            Reporter: Mao, Wei
>              Labels: streaming
> Users often stop and restart their streaming jobs for tasks such as maintenance, software
upgrades or even application logic updates. When a job re-starts it should pick up where it
left off i.e. any state information that existed when the job stopped should be used as the
initial state when the job restarts.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message