flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] pnowojski commented on a change in pull request #6417: [FLINK-9913][runtime] Improve output serialization only once in RecordWriter
Date Fri, 14 Sep 2018 09:35:08 GMT
pnowojski commented on a change in pull request #6417: [FLINK-9913][runtime] Improve output
serialization only once in RecordWriter
URL: https://github.com/apache/flink/pull/6417#discussion_r217654112
 
 

 ##########
 File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/writer/RecordWriter.java
 ##########
 @@ -113,11 +113,7 @@ public void broadcastEmit(T record) throws IOException, InterruptedException
{
 	 * This is used to send LatencyMarks to a random target channel.
 	 */
 	public void randomEmit(T record) throws IOException, InterruptedException {
-		serializer.serializeRecord(record);
-
-		if (copyFromSerializerToTargetChannel(rng.nextInt(numChannels))) {
-			serializer.prune();
-		}
+		emit(record, new int[] { rng.nextInt(numChannels) });
 
 Review comment:
   Wait a minute here. 
   
   ```
   emit(record, new int[] { rng.nextInt(numChannels) });
   ```
   
   @NicoK is an actual performance regression. Creating a single element int array (once per
every record!) actually reduces the throughput by about 10%-20% (tested during one of the
hackathons). Please revert to using `copyFromSerializerToTargetChannel(rng.nextInt(numChannels))`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message