flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From StephanEwen <...@git.apache.org>
Subject [GitHub] flink pull request #3295: [FLINK-5747] [distributed coordination] Eager sche...
Date Fri, 17 Feb 2017 13:21:35 GMT
Github user StephanEwen commented on a diff in the pull request:

    https://github.com/apache/flink/pull/3295#discussion_r101754578
  
    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/concurrent/FutureUtils.java
---
    @@ -88,4 +100,104 @@ public RetryException(Throwable cause) {
     			super(cause);
     		}
     	}
    +
    +	// ------------------------------------------------------------------------
    +	//  composing futures
    +	// ------------------------------------------------------------------------
    +
    +	/**
    +	 * Creates a future that is complete once multiple other futures completed. 
    +	 * The ConjunctFuture fails (completes exceptionally) once one of the Futures in the
    +	 * conjunction fails.
    +	 *
    +	 * <p>The ConjunctFuture gives access to how many Futures in the conjunction have
already
    +	 * completed successfully, via {@link ConjunctFuture#getNumFuturesCompleted()}. 
    +	 * 
    +	 * @param futures The futures that make up the conjunction. No null entries are allowed.
    +	 * @return The ConjunctFuture that completes once all given futures are complete (or
one fails).
    +	 */
    +	public static ConjunctFuture combineAll(Collection<? extends Future<?>>
futures) {
    +		checkNotNull(futures, "futures");
    +		checkArgument(!futures.isEmpty(), "futures is empty");
    +
    +		final ConjunctFutureImpl conjunct = new ConjunctFutureImpl(futures.size());
    +
    +		for (Future<?> future : futures) {
    +			future.handle(conjunct.completionHandler);
    +		}
    +
    +		return conjunct;
    +	}
    +
    +	/**
    +	 * A future that is complete once multiple other futures completed. The futures are
not
    +	 * necessarily of the same type, which is why the type of this Future is {@code Void}.
    +	 * The ConjunctFuture fails (completes exceptionally) once one of the Futures in the
    +	 * conjunction fails.
    +	 * 
    +	 * <p>The advantage of using the ConjunctFuture over chaining all the futures
(such as via
    +	 * {@link Future#thenCombine(Future, BiFunction)}) is that ConjunctFuture also tracks
how
    +	 * many of the Futures are already complete.
    +	 */
    +	public interface ConjunctFuture extends CompletableFuture<Void> {
    +
    +		/**
    +		 * Gets the total number of Futures in the conjunction.
    +		 * @return The total number of Futures in the conjunction.
    +		 */
    +		int getNumFuturesTotal();
    +
    +		/**
    +		 * Gets the number of Futures in the conjunction that are already complete.
    +		 * @return The number of Futures in the conjunction that are already complete
    +		 */
    +		int getNumFuturesCompleted();
    +	}
    +
    +	/**
    +	 * The implementation of the {@link ConjunctFuture}.
    +	 * 
    +	 * <p>Implementation notice: The member fields all have package-private access,
because they are
    +	 * either accessed by an inner subclass or by the enclosing class.
    +	 */
    +	private static class ConjunctFutureImpl extends FlinkCompletableFuture<Void> implements
ConjunctFuture {
    --- End diff --
    
    Interesting idea. I think the linked implementation is not yet thread safe, because the
BiFunction that adds the results to the collection is called concurrently as the different
original futures complete. For this particular use case, we'd need to also preserve the order.
This is easy to change by simply pre-allocating a target array and setting the results to
the positions (the completion function would need to get the target index).
    
    I would actually like to do that as a separate follow-up, unless you object there.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message