flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ufuk Celebi <...@apache.org>
Subject Re: How do network transmissions in Flink work?
Date Thu, 08 Oct 2015 11:06:57 GMT
Hey Niklas,

I’ve missed your answer. Sorry for the delay!

> On 20 Jul 2015, at 19:00, Niklas Semmler <nsemmler@inet.tu-berlin.de> wrote:
> 
> Hello Ufuk,
> 
> thank you very much for the answer. You helped me to bring a great deal of context into
the problem :).
> 
> I have one final question: What is a good indicator that the transfer of data contained
in a single ResultPartition is finished? Is there any? Or can the amount of retrieved data
be retrieved from the consumer?

The result life-cycle is independent of the task life-cycle.

On the output side you can look into the “notifySubpartitionConsumed()” notification (in
ResultSubpartition). This signals that every buffer has been handed over to the network or
consuming task (but buffers might still be in flight).

On the input side you can look into the SingleInputGate. The FINISHED event signals that the
result partition has been fully received.

Does this help? What are you trying to do with the partitions?

Again, sorry for the delay.

– Ufuk


Mime
View raw message