beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anton Kedin (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-3360) [SQL] Do not assign triggers for HOP/TUMBLE
Date Fri, 15 Dec 2017 18:01:00 GMT
Anton Kedin created BEAM-3360:
---------------------------------

             Summary: [SQL] Do not assign triggers for HOP/TUMBLE
                 Key: BEAM-3360
                 URL: https://issues.apache.org/jira/browse/BEAM-3360
             Project: Beam
          Issue Type: Task
          Components: dsl-sql
            Reporter: Anton Kedin


Currently when parsing HOP/TUMBLE expressions we create a repeating trigger for the defined
windows, see:

{code:java|title=BeamAggregationRule.java}

  private Trigger createTriggerWithDelay(GregorianCalendar delayTime) {
    return Repeatedly.forever(AfterWatermark.pastEndOfWindow().withLateFirings(AfterProcessingTime
        .pastFirstElementInPane().plusDelayOf(Duration.millis(delayTime.getTimeInMillis()))));
  }
{code}

This will not work correctly with joins, as joins with multiple trigger firings are currently
broken: https://issues.apache.org/jira/browse/BEAM-3190 .

Even if joins with multiple firings worked correctly, SQL parsing stage is still probably
an incorrect place to infer them.

Better alternatives:
 - inherit the user-defined triggers for the input pcollection without modification;
 - triggering at sinks ( https://s.apache.org/beam-sink-triggers ) might define a way to backpropagate
triggers with correct semantics;



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message