flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Ge Wu <andrew.ge...@eniro.com>
Subject Parallel execution on AllWindows
Date Wed, 03 Aug 2016 12:09:58 GMT

I have such task that I want to count window on a stream and execute them batch by batch.
Execute a count window may take some time, so I want it to be executed in parallel.
I read this part in the documentation when I found it automatically reduced parallelization
to 1

* Note: This operation can be inherently non-parallel since all elements have to pass through
* the same operator instance. (Only for special cases, such as aligned time windows is
* it possible to perform this operation in parallel).
(It looks like the java doc is copied from timeWindowAll)

If I force all window function to run in parallel, what will happen?
Will a time/count window broadcast to all instances of the function? or will it be send to
one of the instance so I can parallelize my work?


Confidentiality Notice: This e-mail transmission may contain confidential 
or legally privileged information that is intended only for the individual 
or entity named in the e-mail address. If you are not the intended 
recipient, you are hereby notified that any disclosure, copying, 
distribution, or reliance upon the contents of this e-mail is strictly 
prohibited and may be unlawful. If you have received this e-mail in error, 
please notify the sender immediately by return e-mail and delete all copies 
of this message.

View raw message