beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jingsong Lee (JIRA)" <>
Subject [jira] [Commented] (BEAM-912) Trigger can be more flexible
Date Sun, 06 Nov 2016 14:19:58 GMT


Jingsong Lee commented on BEAM-912:

Thanks for the reply!
The detail of my example is explained in the picture of attachments.
We try to implement this case by SessionWindow. First, we use sessionWindow + defualtTrigger(pastEndOfWindow),
but The window will be fired too late when there are many overlap between B elements. Second,
we use earlyTrigger, but results will be outputted repeated and windows will be bigger and
bigger. We can clear the buffer state if we can access the state in trigger.
So we implement this case in GroupByKey+ParDoWithStateAndTimer by modify our runner to support
a new pTransform just like Combine.perKey.  I want to know this solution is a good way? Is
there a better way to support these between join?
My English is not good, please forgive me! 

> Trigger can be more flexible
> ------------------------------
>                 Key: BEAM-912
>                 URL:
>             Project: Beam
>          Issue Type: New Feature
>          Components: beam-model
>            Reporter: Jingsong Lee
>            Assignee: Kenneth Knowles
>         Attachments: betweenJoin.png
> 1.We can support some data-driven trigger, so we need expose data in OnElementContext
of onElement method. 
> 2.We can support more flexible join, so we need expose buffer tag in TriggerContext,
now this buffer tag is in SystemReduceFn.
> for example: SELECT STREAM * FROM Orders AS o JOIN Shipments AS s
> ON o.orderId = s.orderId AND s.rowtime BETWEEN o.rowtime AND o.rowtime + INTERVAL '1'
> link:

This message was sent by Atlassian JIRA

View raw message