beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anton Kedin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (BEAM-3360) [SQL] Do not assign triggers for HOP/TUMBLE
Date Fri, 15 Dec 2017 18:01:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Anton Kedin updated BEAM-3360:
------------------------------
    Description: 
Currently when parsing HOP/TUMBLE/SESSION expressions we create a repeating trigger for the
defined windows, see:

{code:java|title=BeamAggregationRule.java}

  private Trigger createTriggerWithDelay(GregorianCalendar delayTime) {
    return Repeatedly.forever(AfterWatermark.pastEndOfWindow().withLateFirings(AfterProcessingTime
        .pastFirstElementInPane().plusDelayOf(Duration.millis(delayTime.getTimeInMillis()))));
  }
{code}

This will not work correctly with joins, as joins with multiple trigger firings are currently
broken: https://issues.apache.org/jira/browse/BEAM-3190 .

Even if joins with multiple firings worked correctly, SQL parsing stage is still probably
an incorrect place to infer them.

Better alternatives:
 - inherit the user-defined triggers for the input pcollection without modification;
 - triggering at sinks ( https://s.apache.org/beam-sink-triggers ) might define a way to backpropagate
triggers with correct semantics;

  was:
Currently when parsing HOP/TUMBLE expressions we create a repeating trigger for the defined
windows, see:

{code:java|title=BeamAggregationRule.java}

  private Trigger createTriggerWithDelay(GregorianCalendar delayTime) {
    return Repeatedly.forever(AfterWatermark.pastEndOfWindow().withLateFirings(AfterProcessingTime
        .pastFirstElementInPane().plusDelayOf(Duration.millis(delayTime.getTimeInMillis()))));
  }
{code}

This will not work correctly with joins, as joins with multiple trigger firings are currently
broken: https://issues.apache.org/jira/browse/BEAM-3190 .

Even if joins with multiple firings worked correctly, SQL parsing stage is still probably
an incorrect place to infer them.

Better alternatives:
 - inherit the user-defined triggers for the input pcollection without modification;
 - triggering at sinks ( https://s.apache.org/beam-sink-triggers ) might define a way to backpropagate
triggers with correct semantics;


> [SQL] Do not assign triggers for HOP/TUMBLE
> -------------------------------------------
>
>                 Key: BEAM-3360
>                 URL: https://issues.apache.org/jira/browse/BEAM-3360
>             Project: Beam
>          Issue Type: Task
>          Components: dsl-sql
>            Reporter: Anton Kedin
>
> Currently when parsing HOP/TUMBLE/SESSION expressions we create a repeating trigger for
the defined windows, see:
> {code:java|title=BeamAggregationRule.java}
>   private Trigger createTriggerWithDelay(GregorianCalendar delayTime) {
>     return Repeatedly.forever(AfterWatermark.pastEndOfWindow().withLateFirings(AfterProcessingTime
>         .pastFirstElementInPane().plusDelayOf(Duration.millis(delayTime.getTimeInMillis()))));
>   }
> {code}
> This will not work correctly with joins, as joins with multiple trigger firings are currently
broken: https://issues.apache.org/jira/browse/BEAM-3190 .
> Even if joins with multiple firings worked correctly, SQL parsing stage is still probably
an incorrect place to infer them.
> Better alternatives:
>  - inherit the user-defined triggers for the input pcollection without modification;
>  - triggering at sinks ( https://s.apache.org/beam-sink-triggers ) might define a way
to backpropagate triggers with correct semantics;



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message