flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Radu Tudoran <radu.tudo...@huawei.com>
Subject RE: concurrency?
Date Fri, 31 Mar 2017 09:36:58 GMT
Hi,

Thanks Fabian. But is there also a fixed order that is imposed in their execution?

I am asking this because it is not enough just to have them executed atomically. If once you
have the processElement() being called and then onTimer(), and in the next called you have
them vice versa, it would mean that you need additional mechanism to synchronize your logic.
Right?
For example if in the
process element you do state.update (newValue)
and in the ontimer you do out.collect(state.getValue())

than if you have ev1,ev2 and eve3 coming at consecutive times and once the function are executed
processelement and than timer and then in reverse order your output would be:

time1: (processElement)                                    ev1 –arrives state=ev1
time2: (processElement – executed first)       ev2-arrives state=ev2              onTime(executed
second):   out = ev2
time3: (processElement – executed second) ev3-arrives state=ev3              onTime(executed
first):          out = ev2

Best regards,

Dr. Radu Tudoran
Senior Research Engineer - Big Data Expert
IT R&D Division

[cid:image007.jpg@01CD52EB.AD060EE0]
HUAWEI TECHNOLOGIES Duesseldorf GmbH
German Research Center
Munich Office
Riesstrasse 25, 80992 München

E-mail: radu.tudoran@huawei.com<mailto:radu.tudoran@huawei.com>
Mobile: +49 15209084330
Telephone: +49 891588344173

HUAWEI TECHNOLOGIES Duesseldorf GmbH
Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com<http://www.huawei.com/>
Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
Managing Director: Bo PENG, Qiuen Peng, Shengli Wang
Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
Geschäftsführer: Bo PENG, Qiuen Peng, Shengli Wang
This e-mail and its attachments contain confidential information from HUAWEI, which is intended
only for the person or entity whose address is listed above. Any use of the information contained
herein in any way (including, but not limited to, total or partial disclosure, reproduction,
or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive
this e-mail in error, please notify the sender by phone or email immediately and delete it!

From: Fabian Hueske [mailto:fhueske@gmail.com]
Sent: Friday, March 31, 2017 11:05 AM
To: Radu Tudoran
Cc: user@flink.apache.org
Subject: Re: concurrency?

Hi Radu,
the processElement() and onTimer() calls are synchronized by a lock, i.e., they won't be called
at the same time.
Best, Fabian

2017-03-31 9:34 GMT+02:00 Radu Tudoran <radu.tudoran@huawei.com<mailto:radu.tudoran@huawei.com>>:
Hi,

I would like to use a processFunction to accumulate elements. Therefore in the processElement
function I will accumulate this element into a state. However, I would like to emit the output
only 1ms later. Therefore I would register a timer to trigger one second later and read the
state and emit it.
However, I am curious of what happens if in the next ms another event arrives. In principle
both the processElement function and the onTimer function should be triggered in the same
time.  My question is: is there a fix order to execute them?  Because if any of them work
just like normal threads, than concurrency related issues could happen when accessing the
state.


Mime
View raw message