flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Jerman <st...@kloudspot.com>
Subject Question about Global Windows.
Date Mon, 14 Aug 2017 04:32:01 GMT
Hi Folks,

I have a question regarding Global Windows.

I have a stream with a large number of records. The records have a key which has a very high
cardinality. They also have a state ( start, status, finish).

I need to do some processing where I look at the records separated into windows using the
‘state’ property.

From the documentation, I believe I should be using a Global Window with a custom trigger
to identify the windows….

I have this implemented.. the Trigger returns ‘CONTINUE” for ‘start', and FIRE_AND_PURGE
for ‘finish'.

I also need to avoid running out of memory  since sometimes I don’t get ‘finish’ records…
so I added a timer to the Trigger which PURGE’s if it fires..

Is this the correct approach?

I say this since I do in fact see a memory leak …  is there anything else I need to be aware
of?

Thanks

Steve
Mime
View raw message