nemo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-nemo] wynot12 commented on a change in pull request #220: [NEMO-395] Address SonarCloud issues for the scheduler package
Date Fri, 14 Jun 2019 07:28:07 GMT
wynot12 commented on a change in pull request #220: [NEMO-395] Address SonarCloud issues for
the scheduler package
URL: https://github.com/apache/incubator-nemo/pull/220#discussion_r293688023
 
 

 ##########
 File path: runtime/master/src/main/java/org/apache/nemo/runtime/master/scheduler/BatchScheduler.java
 ##########
 @@ -447,6 +424,60 @@ private void doSchedule() {
     return tasks;
   }
 
+  ////////////////////////////////////////////////////////////////////// Task cloning methods.
+
+  /**
+   * @return true if the number of clones is modified.
+   *         false otherwise.
+   */
+  private boolean modifyStageNumClone(final Stage stage, final ClonedSchedulingProperty.CloneConf
cloneConf) {
+    final double fractionToWaitFor = cloneConf.getFractionToWaitFor();
+    final Object[] completedTaskTimes = planStateManager.getCompletedTaskTimeListMs(stage.getId()).toArray();
+
+    // Only after the fraction of the tasks are done...
+    // Delayed cloning (aggressive)
+    if (completedTaskTimes.length > 0
+      && completedTaskTimes.length >= Math.round(stage.getTaskIndices().size()
* fractionToWaitFor)) {
+      // Only if the running task is considered a 'straggler'....
+      Arrays.sort(completedTaskTimes);
+      final long medianTime = (long) completedTaskTimes[completedTaskTimes.length / 2];
+      final double medianTimeMultiplier = cloneConf.getMedianTimeMultiplier();
+      final Map<String, Long> executingTaskToTime = planStateManager.getExecutingTaskToRunningTimeMs(stage.getId());
+
+      return modifyStageNumCloneUsingMedianTime(
+        stage.getId(), completedTaskTimes.length, medianTime, medianTimeMultiplier, executingTaskToTime);
+    } else {
+      return false;
+    }
+  }
+
+  /**
+   * @return true if the number of clones is modified.
+   *         false otherwise.
+   */
+  private boolean modifyStageNumCloneUsingMedianTime(final String stageId,
 
 Review comment:
   Do we need to separate this part from a `modifyStageNumClone`?
   
   The both function has too similar function name and exactly same function description..
   
   IMHO, it's better to make them as separate functions when we can clearly define them..
in other words, when we can give them clear names and descriptions.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message