apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siyuan Hua (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXMALHAR-2301) Implement another TimeBucketAssigner to work with any time
Date Mon, 21 Nov 2016 06:14:58 GMT

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15682657#comment-15682657

Siyuan Hua commented on APEXMALHAR-2301:

Ok, there are couple things here, the current implementation(Assigner) is not easy to use
to resolve the unbounded time buckets. You have to set the reference to infinite time and
the expireduration to Long.MAX - 0 which is not very obvious. 
What we do is just assign time to a rounded time buckets(just by bucket duration)
For example if a bucket span is 1000, time 1001,1002,1003 will all be in bucket 1000

Also the current "moving/sliding" mechanism is something we don't need in event time handling

Another issue we need to fix is the bucket metadata current is stored in a array which is
not good enough for a unbounded time buckets. I also need to change it to a map to make it
suitable  for monotonically unlimited bucket ids. 

> Implement another TimeBucketAssigner to work with any time
> ----------------------------------------------------------
>                 Key: APEXMALHAR-2301
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2301
>             Project: Apache Apex Malhar
>          Issue Type: Sub-task
>            Reporter: Siyuan Hua
>            Assignee: Siyuan Hua
>             Fix For: 3.6.0
> The current TimeBucketAssigner only works for time that is close to current system timestamp.
But to support arbitrary event time, we need another TimeBucketAssigner 

This message was sent by Atlassian JIRA

View raw message