apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Yan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (APEXMALHAR-2130) implement scalable windowed storage
Date Thu, 30 Jun 2016 18:40:10 GMT

     [ https://issues.apache.org/jira/browse/APEXMALHAR-2130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

David Yan updated APEXMALHAR-2130:
----------------------------------
    Description: 
This feature is used for supporting windowing.

The storage needs to have the following features:
1. Spillable key value storage (integrate with APEXMALHAR-2026)
2. Upon checkpoint, it saves a snapshot for the entire data set with the checkpointing window
id
3. When recovering, it takes the recovery window id and restores to that snapshot
4. It should implement the WindowedStorage and WindowedKeyedStorage interfaces, and because
of 2 and 3, we may want to add methods to the WindowedStorage interface so that the implementation
of WindowedOperator can notify the storage of checkpointing and recovering.
5. Item 2 should be done incrementally (ManagedState


  was:
This feature is used for supporting windowing.

The storage needs to have the following features:
1. Spillable key value storage (integrate with APEXMALHAR-2026)
2. Upon checkpoint, it saves a snapshot for the entire data set with the checkpointing window
id
3. When recovering, it takes the recovery window id and restores to that snapshot
4. It should implement the WindowedStorage and WindowedKeyedStorage interfaces, and because
of 2 and 3, we may want to add methods to the WindowedStorage interface so that the implementation
of WindowedOperator can notify the storage of checkpointing and recovering.



> implement scalable windowed storage
> -----------------------------------
>
>                 Key: APEXMALHAR-2130
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2130
>             Project: Apache Apex Malhar
>          Issue Type: Task
>            Reporter: bright chen
>            Assignee: bright chen
>
> This feature is used for supporting windowing.
> The storage needs to have the following features:
> 1. Spillable key value storage (integrate with APEXMALHAR-2026)
> 2. Upon checkpoint, it saves a snapshot for the entire data set with the checkpointing
window id
> 3. When recovering, it takes the recovery window id and restores to that snapshot
> 4. It should implement the WindowedStorage and WindowedKeyedStorage interfaces, and because
of 2 and 3, we may want to add methods to the WindowedStorage interface so that the implementation
of WindowedOperator can notify the storage of checkpointing and recovering.
> 5. Item 2 should be done incrementally (ManagedState



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message