nemo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] sanha commented on a change in pull request #193: [NEMO-338] SkewSamplingPass
Date Thu, 21 Feb 2019 04:36:47 GMT
sanha commented on a change in pull request #193: [NEMO-338] SkewSamplingPass
URL: https://github.com/apache/incubator-nemo/pull/193#discussion_r258776185
 
 

 ##########
 File path: common/src/main/java/org/apache/nemo/common/ir/vertex/utility/SamplingVertex.java
 ##########
 @@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.nemo.common.ir.vertex.utility;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import org.apache.nemo.common.Util;
+import org.apache.nemo.common.ir.edge.IREdge;
+import org.apache.nemo.common.ir.vertex.IRVertex;
+
+/**
+ * Executes the original IRVertex using a subset of input data partitions.
+ */
+public final class SamplingVertex extends IRVertex {
+  private final IRVertex originalVertex;
+  private final IRVertex cloneOfOriginalVertex;
+  private final float desiredSampleRate;
+
+  /**
+   * @param originalVertex to clone.
+   * @param desiredSampleRate percentage of tasks to execute.
+   *                          The actual sample rate may vary depending on neighboring sampling
vertices.
+   */
+  public SamplingVertex(final IRVertex originalVertex, final float desiredSampleRate) {
+    super();
+    if (originalVertex instanceof SamplingVertex) {
+      throw new IllegalArgumentException("Cannot sample again: " + originalVertex.toString());
+    }
+    if (desiredSampleRate > 1 || desiredSampleRate <= 0) {
+      throw new IllegalArgumentException(String.valueOf(desiredSampleRate));
+    }
+    this.originalVertex = originalVertex;
+    this.cloneOfOriginalVertex = originalVertex.getClone();
+    this.desiredSampleRate = desiredSampleRate;
+    originalVertex.copyExecutionPropertiesTo(this);
+  }
+
+  public IRVertex getOriginalVertex() {
+    return originalVertex;
+  }
+
+  public IRVertex getCloneOfOriginalVertex() {
+    this.copyExecutionPropertiesTo(cloneOfOriginalVertex);
 
 Review comment:
   If the purpose of this method is creating a new clone of the original vertex (for every
call), let's create a new clone and copy the EPs instead of returning already cloned `cloneOfOriginalVertex`.
   If not, let's copy the EPs in construction.
   
   Plus, please add some method level comments for these methods.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message