nemo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] wonook commented on a change in pull request #191: [NEMO-332] Refactor RunTimePass
Date Fri, 01 Feb 2019 03:26:50 GMT
wonook commented on a change in pull request #191: [NEMO-332] Refactor RunTimePass
URL: https://github.com/apache/incubator-nemo/pull/191#discussion_r252923351
 
 

 ##########
 File path: compiler/optimizer/src/main/java/org/apache/nemo/compiler/optimizer/pass/compiletime/reshaping/SkewReshapingPass.java
 ##########
 @@ -116,6 +125,13 @@ public IRDAG apply(final IRDAG dag) {
         final MessageBarrierVertex mbv = new MessageBarrierVertex<>(dynOptDataCollector);
         final MessageAggregatorVertex mav = new MessageAggregatorVertex(new HashMap(), dynOptDataAggregator);
         dag.insert(mbv, mav, encoderProperty, decoderProperty, shuffleEdgeGroup);
+
+        // Set the partitioner property
+        final int dstParallelism = representativeEdge.getDst().getPropertyValue(MinParallelismProperty.class).get();
+        shuffleEdgeGroup.forEach(e -> {
+          e.setPropertyPermanently(PartitionerProperty.of(
+            PartitionerProperty.Type.Hash, dstParallelism * HASH_RANGE_MULTIPLIER));
 
 Review comment:
   Should this be included in the `reshaping pass`?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message