flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Using Custom Partitioner in Streaming with parallelism 1 adding latency
Date Fri, 16 Jun 2017 10:07:18 GMT
Hi,

These two documentation pages might be interesting:
 - https://ci.apache.org/projects/flink/flink-docs-release-1.3/concepts/runtime.html <https://ci.apache.org/projects/flink/flink-docs-release-1.3/concepts/runtime.html>
 - https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/datastream_api.html#task-chaining-and-resource-groups
<https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/datastream_api.html#task-chaining-and-resource-groups>

Flink will chain operations together whenever this is possible and the basic prerequisites
for this are: the parallelism is the same and the connection pattern is “forwarding”,
i.e. there is no broadcast, shuffle, or custom partitioning scheme.

Best,
Aljoscha 
> On 16. Jun 2017, at 06:42, sohimankotia <sohimankotia@gmail.com> wrote:
> 
> You are right Aljoscha . Jog graph is splitted after introducing partitioner
> .
> 
> I was under impression that If parallelism is set everything will be chained
> together .
> Can you explain how data will flow for  map -> partitioner -> flatmap  if
> parallelism or It would be great if point me to right documentation ?
> 
> Can is this possible it can add more than 100 seconds delay if partitioner
> is added ?
> 
> Sorry for dumb question but I did not find any detailed documentation on
> flink website . Most of links
> https://cwiki.apache.org/confluence/display/FLINK/Parallelism+and+Scheduling
> are also empty as of now .
> 
> 
> 
> 
> 
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Using-Custom-Partitioner-in-Streaming-with-parallelism-1-adding-latency-tp13766p13774.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.


Mime
View raw message