flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ray Ruvinskiy <ray.ruvins...@arcticwolf.com>
Subject Question regarding configuring number of network buffers
Date Wed, 07 Jun 2017 15:58:57 GMT
The documentation provides the formula #slots-per-TM^2 * #TMs * 4 to determine the number of
network buffers we should configure. The documentation also says, “A logical network connection
exists for each point-to-point exchange of data over the network, which typically happens
at repartitioning- or broadcasting steps (shuffle phase). In those, each parallel task inside
the TaskManager has to be able to talk to all other parallel tasks.” Does that say that
each parallel task inside the TaskManager talk to all parallel tasks inside the same TaskManager
or to all parallel tasks across all task managers? Intuitively, I would assume the latter,
but then wouldn’t the formula for determining the number of network buffers be more along
the lines of (#slots-per-TM * #TMs)^2?

Thanks,

Ray

Mime
View raw message