flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rong Rong <walter...@gmail.com>
Subject Re: Over Window Not Processing Messages
Date Wed, 27 Jun 2018 03:38:25 GMT
Hi Greg.

Based on a quick test I cannot reproduce the issue, it is emitting messages
correctly in the ITCase environment.
can you share more information? Does the same problem happen if you use
proctime?
I am guessing this could be highly correlated with how you set your
watermark strategy of your input streams of "user_things" and "user_stuff".

--
Rong

On Tue, Jun 26, 2018 at 6:37 PM Gregory Fee <gfee@lyft.com> wrote:

> Hello User Community!
>
> I am running some streaming SQL that involves a union all into an over
> window similar to the below:
>
> SELECT user_id, count(action) OVER (PARTITION BY user_id ORDER BY rowtime
> RANGE INTERVAL '14' DAY PRECEDING) action_count, rowtime
> FROM
>     (SELECT rowtime, user_id, thing as action FROM user_things
>      UNION ALL SELECT rowtime, user_id, stuff as action FROM user_stuff)
>
> The SQL generates three operators. There are two operators that process
> the 'from' part of the clause that feed into an 'over' operator. I notice
> that messages flow into the 'over' operator and just buffer there for a
> long time (hours in some cases). Eventually something happens and the data
> starts to flush through to the downstream operators. Can anyone help me
> understand what is causing that behavior? I want the data to flow through
> more consistently.
>
> Thanks!
>
>
>
> --
> *Gregory Fee*
> Engineer
> 425.830.4734 <+14258304734>
> [image: Lyft] <http://www.lyft.com>
>

Mime
View raw message