flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From clay4444 <clay4me...@gmail.com>
Subject About the retract of the calculation result of flink sql
Date Sat, 29 Sep 2018 07:42:33 GMT
Hi everyone,

I am having some problems in the process of using flink sql, my sql is as
follows:

SELECT COUNT(DISTINCT mm.student_id),sum(price)
FROM (
SELECT a.student_id,a.confirm_date_time,sum(a.real_total_price)as price
FROM (
SELECT DISTINCT po.id as order_id
,po.student_id
,po.create_id
,po.paid_date_time
,po.confirm_date_time
,po.real_total_price
from `pack_order` po
left join user1 u on u.id=po.student_id
WHERE po.status='PAY_CONFIRMED'
AND po.real_total_price>500
AND po.confirm_date_time >= '2018-09-27' and po.confirm_date_time <
'2018-09-28'
AND po.type IN (1,2)
AND u.id is null
)a
join pack_order1 por on a.student_id=por.student_id
GROUP BY a.student_id,a.confirm_date_time
HAVING MIN(por.confirm_date_time)=a.confirm_date_time
)mm


I registered 3 tables by receiving data from 3 kafka topics (each topic
represents a table, each topic has only one partition), and then calculated
by flink sql, the result is sent again to a kafka topic, The same partition
I used, my final calculation is a count value and a sum value. According to
my understanding, the final result of kafka should always be incremented,
but the final result is as follows:

<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t1671/%E5%9B%BE%E7%89%87.png>


The result marked in the figure is our final result, and the last result is
not, that is, the final calculation result is wrong.

By observing the final result, we find that the data will increase first,
then decrease, and then increase. This situation is unexplained. We need the
value of count and sum to be incremented in kafka. Because our goal is to
achieve real-time display,

Can you please explain why this happens and how to avoid it?
thank you very much


best wishes!



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Mime
View raw message