flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 徐涛 <happydexu...@gmail.com>
Subject How does flink know which data is modified in dynamic table?
Date Mon, 20 Aug 2018 10:43:53 GMT
<html><head></head><body dir="auto" style="word-wrap: break-word; -webkit-nbsp-mode:
space; line-break: after-white-space;"><meta http-equiv="Content-Type" content="text/html;
charset=gb2312"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break:
after-white-space;" class="">Hi All,<div class=""><span class="Apple-tab-span"
style="white-space:pre">	</span>Like the following code,If I use retract stream,
I think Flink is able to know which item is modified( if praise has 10000 items now, when
one item comes to the stream, only very small amount of data is write to sink)&nbsp;</div><div
class=""><pre style="background-color: rgb(255, 255, 255);" class=""><span style="font-family:
Menlo; font-size: 9pt; color: rgb(0, 0, 128); font-weight: bold;" class=""><span class="Apple-tab-span"
style="white-space:pre">	</span>var </span><font face="Menlo" class=""><span
style="font-size: 9pt;" class="">praiseAggr = tableEnv.sqlQuery(</span></font><font
color="#008000" face="Menlo" class=""><span style="font-size: 9pt;" class=""><b
class="">s"SELECT article_id,hll(uid) as PU FROM praise group by article_id</b></span><b
class="">”</b><span style="font-size: 9pt;" class=""><b class=""> </b></span></font><font
face="Menlo" class=""><span style="font-size: 9pt;" class="">)</span></font></pre><pre
style="background-color: rgb(255, 255, 255); font-family: Menlo; font-size: 9pt;" class=""><pre
style="font-family: Menlo; font-size: 9pt;" class="">        tableEnv.registerTable(<span
style="color:#008000;font-weight:bold;" class="">"finalTable"</span>, praiseAggr)</pre></pre></div><div
class=""><pre style="background-color: rgb(255, 255, 255); font-family: Menlo; font-size:
9pt;" class=""><span class="Apple-tab-span" style="white-space:pre">	</span>tableEnv.sqlUpdate(<span
style="color:#008000;font-weight:bold;" class="">s"insert into sinkTableName SELECT * from
finalTable</span><span style="color:#008000;font-weight:bold;" class="">"</span>)</pre><div
class=""><br class=""></div></div><div class="">&nbsp; &nbsp;
&nbsp; &nbsp; But if I use the following sql, by adding a dynamic timestamp field:</div><div
class=""><span class="Apple-tab-span" style="white-space:pre">		</span><span
style="font-family: Menlo; font-size: 9pt; background-color: rgb(255, 255, 255); color: rgb(0,
0, 128); font-weight: bold;" class="">var </span><span style="font-family: Menlo;
font-size: 9pt; background-color: rgb(255, 255, 255);" class="">praiseAggr = tableEnv.sqlQuery(</span><span
style="font-family: Menlo; font-size: 9pt; background-color: rgb(255, 255, 255); color: rgb(0,
128, 0); font-weight: bold;" class="">s"SELECT article_id,hll(uid)&nbsp;</span><span
style="color: rgb(0, 128, 0); font-family: Menlo; font-weight: bold; background-color: rgb(255,
255, 255);" class="">as PU</span><span style="font-family: Menlo; font-size: 9pt;
background-color: rgb(255, 255, 255); color: rgb(0, 128, 0); font-weight: bold;" class="">,</span><span
style="color: rgb(0, 128, 0); font-weight: bold; font-family: Menlo; font-size: 9pt; background-color:
rgb(255, 255, 255);" class="">LOCALTIMESTAMP as update_timestamp</span><span style="background-color:
rgb(255, 255, 255);" class=""><font color="#008000" face="Menlo" class=""><span
style="font-size: 9pt;" class=""><b class="">&nbsp;FROM praise group by article_id</b></span><b
class="">”</b><span style="font-size: 9pt;" class=""><b class="">&nbsp;</b></span></font></span><span
style="font-family: Menlo; font-size: 9pt; background-color: rgb(255, 255, 255);" class="">)</span></div><div
class=""><span style="background-color: rgb(255, 255, 255);" class=""><font face="Menlo"
class="">&nbsp; </font>&nbsp; &nbsp; Is the whole table flush to the
sink? Or only the incremental value will flush to the sink? Why?</span></div><div
class=""><span style="background-color: rgb(255, 255, 255);" class=""><br class=""></span></div><div
class=""><span style="background-color: rgb(255, 255, 255);" class="">Thanks,</span></div><div
class=""><span style="background-color: rgb(255, 255, 255);" class="">Henry</span></div><div
class=""><br class=""></div></div></body></html>
Mime
View raw message