apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Yan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXMALHAR-2361) Optimise SpillableWindowedKeyedStorage remove(Window) to improve the performance
Date Thu, 01 Dec 2016 00:23:59 GMT

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15710326#comment-15710326
] 

David Yan commented on APEXMALHAR-2361:
---------------------------------------

I think removing it is better since you don't want to trigger again next time with the default
value.

> Optimise SpillableWindowedKeyedStorage remove(Window) to improve the performance
> --------------------------------------------------------------------------------
>
>                 Key: APEXMALHAR-2361
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2361
>             Project: Apache Apex Malhar
>          Issue Type: Improvement
>            Reporter: bright chen
>            Assignee: bright chen
>   Original Estimate: 120h
>  Remaining Estimate: 120h
>
> Currently, SpillableWindowedKeyedStorage remove(Window) will go through each key and
mark all of them as deleted. It would be expensive when there are lots of keys and especially
these entry already spill out of memory (this the common case when remove() was called).
> Suggest to mark whole window as deleted. When the window was marked as deleted, it will
not allowed to add/update any entry of this window ( this should match the requirement as
remove(Window) only be called after allowed lateness



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message