Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
Date: Mon, 23 Jan 2017 13:46:01 +0000 (UTC)
From: Joe Olson <technology@nododos.com>
To: user@flink.apache.org
Message-ID: <944779580.51400.1485179161639.JavaMail.zimbra@nododos.com>
Subject: Queryable State and Windows
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_51399_1392513303.1485179161635"
Thread-Topic: Queryable State and Windows
Thread-Index: kqIaKXgpuMWprDB3FGX25TnkHTUIdQ==
archived-at: Mon, 23 Jan 2017 13:54:33 -0000

------=_Part_51399_1392513303.1485179161635
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

From what I've read in the documentation, and from the examples I've seen, in order to make state queryable externally to Flink, the state descriptor variables need access to the Flink runtime context. 

This means the stream processor has to have access to the 'Rich' level objects - 'RichFlatMap' for example. All the SNAPSHOT1.2 queryable state examples I have seen revolve around RichFlatMap. 

Is there a way to get the runtime context exposed so that you can have state descriptor variables queryable from within a Flink window, while the window is loading? 

My processor is built around the following: 

.addSource(new FlinkKafkaConsumer010()) 
.assignTimestampsAndWatermarks(new MyTimestampsAndWatermarks()) 
.keyBy() 
.window(GlobalWindows.create()) 
.trigger(new myTrigger()) 
.apply(new myWindowFunction()) 
.addSink(new mySink()) 

The only rich object in this chain are available in the apply (RichWindowFunction). But that is too late - I want to be able to query out whats in the window while it is filling. I know I have access to onElement in the trigger, and I can set up the state descriptor variables there, but the variables don't seem to have exposure to the runtime environment within the trigger. 

Is there a way to get queryable state within a Flink window while it is filling? 


------=_Part_51399_1392513303.1485179161635
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><body><div style=3D"font-family: arial, helvetica, sans-serif; font-s=
ize: 12pt; color: #000000"><div>From what I've read in the documentation, a=
nd from the examples I've seen, in order to make state queryable externally=
 to Flink, the state descriptor variables need access to the Flink runtime =
context.<br data-mce-bogus=3D"1"></div><div><br data-mce-bogus=3D"1"></div>=
<div>This means the stream processor has to have access to the 'Rich' level=
 objects - 'RichFlatMap' for example. All the SNAPSHOT1.2 queryable state e=
xamples I have seen revolve around RichFlatMap.<br data-mce-bogus=3D"1"></d=
iv><div><br data-mce-bogus=3D"1"></div><div>Is there a way to get the runti=
me context exposed so that you can have state descriptor variables queryabl=
e from within a Flink window, while the window is loading?<br data-mce-bogu=
s=3D"1"></div><div><br data-mce-bogus=3D"1"></div><div>My processor is buil=
t around the following:<br data-mce-bogus=3D"1"></div><div><br data-mce-bog=
us=3D"1"></div><div>.addSource(new FlinkKafkaConsumer010())<br data-mce-bog=
us=3D"1"></div><div>.assignTimestampsAndWatermarks(new MyTimestampsAndWater=
marks())<br>.keyBy()<br>.window(GlobalWindows.create())<br>.trigger(new myT=
rigger())<br>.apply(new myWindowFunction())<br>.addSink(new mySink())</div>=
<div><br data-mce-bogus=3D"1"></div><div>The only rich object in this chain=
 are available in the apply (RichWindowFunction). But that is too late - I =
want to be able to query out whats in the window while it is filling. I kno=
w I have access to onElement in the trigger, and I can set up the state des=
criptor variables there, but the variables don't seem to have exposure to t=
he runtime environment within the trigger.<br data-mce-bogus=3D"1"></div><d=
iv><br data-mce-bogus=3D"1"></div><div>Is there a way to get queryable stat=
e within a Flink window while it is filling?<br data-mce-bogus=3D"1"></div>=
<div><br data-mce-bogus=3D"1"></div></div></body></html>
------=_Part_51399_1392513303.1485179161635--