From commits-return-34554-archive-asf-public=cust-asf.ponee.io@spark.apache.org Thu Nov 8 15:48:03 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 589BA1807BA for ; Thu, 8 Nov 2018 15:47:57 +0100 (CET) Received: (qmail 62814 invoked by uid 500); 8 Nov 2018 14:47:51 -0000 Mailing-List: contact commits-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list commits@spark.apache.org Received: (qmail 61405 invoked by uid 99); 8 Nov 2018 14:47:50 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 08 Nov 2018 14:47:50 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 8E864E1264; Thu, 8 Nov 2018 14:47:49 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: lixiao@apache.org To: commits@spark.apache.org Date: Thu, 08 Nov 2018 14:48:12 -0000 Message-Id: <194caab0d1cb4877901363fb69222306@git.apache.org> In-Reply-To: <12667b086cfe433d8f04cd695583d652@git.apache.org> References: <12667b086cfe433d8f04cd695583d652@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [24/61] [partial] spark-website git commit: Add docs for Spark 2.4.0 and update the latest link http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/ComplexFutureAction.html ---------------------------------------------------------------------- diff --git a/site/docs/2.4.0/api/java/org/apache/spark/ComplexFutureAction.html b/site/docs/2.4.0/api/java/org/apache/spark/ComplexFutureAction.html new file mode 100644 index 0000000..06aecf7 --- /dev/null +++ b/site/docs/2.4.0/api/java/org/apache/spark/ComplexFutureAction.html @@ -0,0 +1,557 @@ + + + + + +ComplexFutureAction (Spark 2.4.0 JavaDoc) + + + + + + + + +
+ + +
Skip navigation links
+ + + + +
+
+ + + +
+ +
+
+
    +
  • Summary: 
    • +
  • Nested | 
    • +
  • Field | 
    • +
  • Constr | 
    • +
  • Method
    • +
+ +
+ + +
+ + +
+
org.apache.spark
+

Class ComplexFutureAction<T>

+
+
+
    +
  • Object
    • +
  • +
      +
    • org.apache.spark.ComplexFutureAction<T>
      • +
    +
    • +
+
+
    +
  • +
    +
    All Implemented Interfaces:
    +
    FutureAction<T>, scala.concurrent.Awaitable<T>, scala.concurrent.Future<T>
    +
    +
    +
    +
    public class ComplexFutureAction<T>
+extends Object
+implements FutureAction<T>
    +
    A FutureAction for actions that could trigger multiple Spark jobs. Examples include take, + takeSample. Cancellation works by setting the cancelled flag to true and cancelling any pending + jobs.
    +
    • +
+
+
+
    +
  • + +
      +
    • + + +

      Nested Class Summary

      +
        +
      • + + +

        Nested classes/interfaces inherited from interface scala.concurrent.Future

        +scala.concurrent.Future.InternalCallbackExecutor$
        • +
      +
      • +
    + + + +
      +
    • + + +

      Method Summary

      + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      All Methods Instance Methods Concrete Methods 
      Modifier and TypeMethod and Description
      voidcancel() +
      Cancels the execution of this action.
      +
      booleanisCancelled() +
      Returns whether the action has been cancelled.
      +
      booleanisCompleted() +
      Returns whether the action has already been completed with a value or an exception.
      +
      scala.collection.Seq<Object>jobIds() +
      Returns the job IDs run by the underlying async operation.
      +
      <U> voidonComplete(scala.Function1<scala.util.Try<T>,U> func, + scala.concurrent.ExecutionContext executor) +
      When this action is completed, either through an exception, or a value, applies the provided + function.
      +
      ComplexFutureAction<T>ready(scala.concurrent.duration.Duration atMost, + scala.concurrent.CanAwait permit) +
      Blocks until this action completes.
      +
      Tresult(scala.concurrent.duration.Duration atMost, + scala.concurrent.CanAwait permit) +
      Awaits and returns the result (of type T) of this action.
      +
      <S> scala.concurrent.Future<S>transform(scala.Function1<scala.util.Try<T>,scala.util.Try<S>> f, + scala.concurrent.ExecutionContext e) 
      <S> scala.concurrent.Future<S>transformWith(scala.Function1<scala.util.Try<T>,scala.concurrent.Future<S>> f, + scala.concurrent.ExecutionContext e) 
      scala.Option<scala.util.Try<T>>value() +
      The value of this Future.
      +
      +
        +
      • + + +

        Methods inherited from class Object

        +equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
        • +
      + +
        +
      • + + +

        Methods inherited from interface scala.concurrent.Future

        +andThen, collect, failed, fallbackTo, filter, flatMap, foreach, map, mapTo, onFailure, onSuccess, recover, recoverWith, transform, withFilter, zip
        • +
      +
      • +
    +
    • +
+
+
+
    +
  • + +
      +
    • + + +

      Constructor Detail

      + + + +
        +
      • +

        ComplexFutureAction

        +
        public ComplexFutureAction(scala.Function1<JobSubmitter,scala.concurrent.Future<T>> run)
        +
        • +
      +
      • +
    + +
      +
    • + + +

      Method Detail

      + + + +
        +
      • +

        cancel

        +
        public void cancel()
        +
        Description copied from interface: FutureAction
        +
        Cancels the execution of this action.
        +
        +
        Specified by:
        +
        cancel in interface FutureAction<T>
        +
        +
        • +
      + + + +
        +
      • +

        isCancelled

        +
        public boolean isCancelled()
        +
        Description copied from interface: FutureAction
        +
        Returns whether the action has been cancelled.
        +
        +
        Specified by:
        +
        isCancelled in interface FutureAction<T>
        +
        Returns:
        +
        (undocumented)
        +
        +
        • +
      + + + +
        +
      • +

        isCompleted

        +
        public boolean isCompleted()
        +
        Description copied from interface: FutureAction
        +
        Returns whether the action has already been completed with a value or an exception.
        +
        +
        Specified by:
        +
        isCompleted in interface FutureAction<T>
        +
        Specified by:
        +
        isCompleted in interface scala.concurrent.Future<T>
        +
        Returns:
        +
        (undocumented)
        +
        +
        • +
      + + + +
        +
      • +

        jobIds

        +
        public scala.collection.Seq<Object> jobIds()
        +
        Description copied from interface: FutureAction
        +
        Returns the job IDs run by the underlying async operation. +

        + This returns the current snapshot of the job list. Certain operations may run multiple + jobs, so multiple calls to this method may return different lists.

        +
        +
        Specified by:
        +
        jobIds in interface FutureAction<T>
        +
        Returns:
        +
        (undocumented)
        +
        +
        • +
      + + + +
        +
      • +

        onComplete

        +
        public <U> void onComplete(scala.Function1<scala.util.Try<T>,U> func,
+                           scala.concurrent.ExecutionContext executor)
        +
        Description copied from interface: FutureAction
        +
        When this action is completed, either through an exception, or a value, applies the provided + function.
        +
        +
        Specified by:
        +
        onComplete in interface FutureAction<T>
        +
        Specified by:
        +
        onComplete in interface scala.concurrent.Future<T>
        +
        Parameters:
        +
        func - (undocumented)
        +
        executor - (undocumented)
        +
        +
        • +
      + + + +
        +
      • +

        ready

        +
        public ComplexFutureAction<T> ready(scala.concurrent.duration.Duration atMost,
+                                    scala.concurrent.CanAwait permit)
+                             throws InterruptedException,
+                                    java.util.concurrent.TimeoutException
        +
        Description copied from interface: FutureAction
        +
        Blocks until this action completes. +

        +
        +
        Specified by:
        +
        ready in interface FutureAction<T>
        +
        Specified by:
        +
        ready in interface scala.concurrent.Awaitable<T>
        +
        Parameters:
        +
        atMost - maximum wait time, which may be negative (no waiting is done), Duration.Inf + for unbounded waiting, or a finite positive duration
        +
        permit - (undocumented)
        +
        Returns:
        +
        this FutureAction
        +
        Throws:
        +
        InterruptedException
        +
        java.util.concurrent.TimeoutException
        +
        +
        • +
      + + + +
        +
      • +

        result

        +
        public T result(scala.concurrent.duration.Duration atMost,
+                scala.concurrent.CanAwait permit)
+         throws Exception
        +
        Description copied from interface: FutureAction
        +
        Awaits and returns the result (of type T) of this action. +

        +
        +
        Specified by:
        +
        result in interface FutureAction<T>
        +
        Specified by:
        +
        result in interface scala.concurrent.Awaitable<T>
        +
        Parameters:
        +
        atMost - maximum wait time, which may be negative (no waiting is done), Duration.Inf + for unbounded waiting, or a finite positive duration
        +
        permit - (undocumented)
        +
        Returns:
        +
        the result value if the action is completed within the specific maximum wait time
        +
        Throws:
        +
        Exception - exception during action execution
        +
        +
        • +
      + + + +
        +
      • +

        transform

        +
        public <S> scala.concurrent.Future<S> transform(scala.Function1<scala.util.Try<T>,scala.util.Try<S>> f,
+                                                scala.concurrent.ExecutionContext e)
        +
        +
        Specified by:
        +
        transform in interface FutureAction<T>
        +
        +
        • +
      + + + +
        +
      • +

        transformWith

        +
        public <S> scala.concurrent.Future<S> transformWith(scala.Function1<scala.util.Try<T>,scala.concurrent.Future<S>> f,
+                                                    scala.concurrent.ExecutionContext e)
        +
        +
        Specified by:
        +
        transformWith in interface FutureAction<T>
        +
        +
        • +
      + + + +
        +
      • +

        value

        +
        public scala.Option<scala.util.Try<T>> value()
        +
        Description copied from interface: FutureAction
        +
        The value of this Future. +

        + If the future is not completed the returned value will be None. If the future is completed + the value will be Some(Success(t)) if it contains a valid result, or Some(Failure(error)) if + it contains an exception.

        +
        +
        Specified by:
        +
        value in interface FutureAction<T>
        +
        Specified by:
        +
        value in interface scala.concurrent.Future<T>
        +
        Returns:
        +
        (undocumented)
        +
        +
        • +
      +
      • +
    +
    • +
+
+
+ + +
+ + +
Skip navigation links
+ + + + +
+
+ + + +
+ +
+
+
    +
  • Summary: 
    • +
  • Nested | 
    • +
  • Field | 
    • +
  • Constr | 
    • +
  • Method
    • +
+ +
+ + +
+ + + http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/ContextBarrierId.html ---------------------------------------------------------------------- diff --git a/site/docs/2.4.0/api/java/org/apache/spark/ContextBarrierId.html b/site/docs/2.4.0/api/java/org/apache/spark/ContextBarrierId.html new file mode 100644 index 0000000..d2601db --- /dev/null +++ b/site/docs/2.4.0/api/java/org/apache/spark/ContextBarrierId.html @@ -0,0 +1,405 @@ + + + + + +ContextBarrierId (Spark 2.4.0 JavaDoc) + + + + + + + + +
+ + +
Skip navigation links
+ + + + +
+
+ + + +
+ +
+
+
    +
  • Summary: 
    • +
  • Nested | 
    • +
  • Field | 
    • +
  • Constr | 
    • +
  • Method
    • +
+ +
+ + +
+ + +
+
org.apache.spark
+

Class ContextBarrierId

+
+
+
    +
  • Object
    • +
  • +
      +
    • org.apache.spark.ContextBarrierId
      • +
    +
    • +
+
+
    +
  • +
    +
    All Implemented Interfaces:
    +
    java.io.Serializable, scala.Equals, scala.Product
    +
    +
    +
    +
    public class ContextBarrierId
+extends Object
+implements scala.Product, scala.Serializable
    +
    For each barrier stage attempt, only at most one barrier() call can be active at any time, thus + we can use (stageId, stageAttemptId) to identify the stage attempt where the barrier() call is + from.
    +
    +
    See Also:
    +
    Serialized Form
    +
    +
    • +
+
+
+
    +
  • + +
      +
    • + + +

      Constructor Summary

      + + + + + + + + +
      Constructors 
      Constructor and Description
      ContextBarrierId(int stageId, + int stageAttemptId) 
      +
      • +
    + + +
    • +
+
+
+
    +
  • + +
      +
    • + + +

      Constructor Detail

      + + + +
        +
      • +

        ContextBarrierId

        +
        public ContextBarrierId(int stageId,
+                        int stageAttemptId)
        +
        • +
      +
      • +
    + +
      +
    • + + +

      Method Detail

      + + + +
        +
      • +

        canEqual

        +
        public abstract static boolean canEqual(Object that)
        +
        • +
      + + + +
        +
      • +

        equals

        +
        public abstract static boolean equals(Object that)
        +
        • +
      + + + +
        +
      • +

        productElement

        +
        public abstract static Object productElement(int n)
        +
        • +
      + + + +
        +
      • +

        productArity

        +
        public abstract static int productArity()
        +
        • +
      + + + +
        +
      • +

        productIterator

        +
        public static scala.collection.Iterator<Object> productIterator()
        +
        • +
      + + + +
        +
      • +

        productPrefix

        +
        public static String productPrefix()
        +
        • +
      + + + +
        +
      • +

        stageId

        +
        public int stageId()
        +
        • +
      + + + +
        +
      • +

        stageAttemptId

        +
        public int stageAttemptId()
        +
        • +
      + + + +
        +
      • +

        toString

        +
        public String toString()
        +
        +
        Overrides:
        +
        toString in class Object
        +
        +
        • +
      +
      • +
    +
    • +
+
+
+ + +
+ + +
Skip navigation links
+ + + + +
+
+ + + +
+ +
+
+
    +
  • Summary: 
    • +
  • Nested | 
    • +
  • Field | 
    • +
  • Constr | 
    • +
  • Method
    • +
+ +
+ + +
+ + + http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/Dependency.html ---------------------------------------------------------------------- diff --git a/site/docs/2.4.0/api/java/org/apache/spark/Dependency.html b/site/docs/2.4.0/api/java/org/apache/spark/Dependency.html new file mode 100644 index 0000000..d4bd419 --- /dev/null +++ b/site/docs/2.4.0/api/java/org/apache/spark/Dependency.html @@ -0,0 +1,284 @@ + + + + + +Dependency (Spark 2.4.0 JavaDoc) + + + + + + + + +
+ + +
Skip navigation links
+ + + + +
+
+ + + +
+ +
+
+
    +
  • Summary: 
    • +
  • Nested | 
    • +
  • Field | 
    • +
  • Constr | 
    • +
  • Method
    • +
+ +
+ + +
+ + +
+
org.apache.spark
+

Class Dependency<T>

+
+
+
    +
  • Object
    • +
  • +
      +
    • org.apache.spark.Dependency<T>
      • +
    +
    • +
+
+
    +
  • +
    +
    All Implemented Interfaces:
    +
    java.io.Serializable
    +
    +
    +
    Direct Known Subclasses:
    +
    NarrowDependency, ShuffleDependency
    +
    +
    +
    +
    public abstract class Dependency<T>
+extends Object
+implements scala.Serializable
    +
    :: DeveloperApi :: + Base class for dependencies.
    +
    +
    See Also:
    +
    Serialized Form
    +
    +
    • +
+
+
+
    +
  • + +
      +
    • + + +

      Constructor Summary

      + + + + + + + + +
      Constructors 
      Constructor and Description
      Dependency() 
      +
      • +
    + +
      +
    • + + +

      Method Summary

      + + + + + + + + + + +
      All Methods Instance Methods Abstract Methods 
      Modifier and TypeMethod and Description
      abstract RDD<T>rdd() 
      +
        +
      • + + +

        Methods inherited from class Object

        +equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
        • +
      +
      • +
    +
    • +
+
+
+
    +
  • + +
      +
    • + + +

      Constructor Detail

      + + + +
        +
      • +

        Dependency

        +
        public Dependency()
        +
        • +
      +
      • +
    + +
      +
    • + + +

      Method Detail

      + + + +
        +
      • +

        rdd

        +
        public abstract RDD<T> rdd()
        +
        • +
      +
      • +
    +
    • +
+
+
+ + +
+ + +
Skip navigation links
+ + + + +
+
+ + + +
+ +
+
+
    +
  • Summary: 
    • +
  • Nested | 
    • +
  • Field | 
    • +
  • Constr | 
    • +
  • Method
    • +
+ +
+ + +
+ + + http://git-wip-us.apache.org/repos/asf/spark-website/blob/52917ac4/site/docs/2.4.0/api/java/org/apache/spark/ExceptionFailure.html ---------------------------------------------------------------------- diff --git a/site/docs/2.4.0/api/java/org/apache/spark/ExceptionFailure.html b/site/docs/2.4.0/api/java/org/apache/spark/ExceptionFailure.html new file mode 100644 index 0000000..36b5bc6 --- /dev/null +++ b/site/docs/2.4.0/api/java/org/apache/spark/ExceptionFailure.html @@ -0,0 +1,502 @@ + + + + + +ExceptionFailure (Spark 2.4.0 JavaDoc) + + + + + + + + +
+ + +
Skip navigation links
+ + + + +
+
+ + + +
+ +
+
+
    +
  • Summary: 
    • +
  • Nested | 
    • +
  • Field | 
    • +
  • Constr | 
    • +
  • Method
    • +
+ +
+ + +
+ + +
+
org.apache.spark
+

Class ExceptionFailure

+
+
+
    +
  • Object
    • +
  • +
      +
    • org.apache.spark.ExceptionFailure
      • +
    +
    • +
+
+
    +
  • +
    +
    All Implemented Interfaces:
    +
    java.io.Serializable, TaskEndReason, TaskFailedReason, scala.Equals, scala.Product
    +
    +
    +
    +
    public class ExceptionFailure
+extends Object
+implements TaskFailedReason, scala.Product, scala.Serializable
    +
    :: DeveloperApi :: + Task failed due to a runtime exception. This is the most common failure case and also captures + user program exceptions. +

    + stackTrace contains the stack trace of the exception itself. It still exists for backward + compatibility. It's better to use this(e: Throwable, metrics: Option[TaskMetrics]) to + create ExceptionFailure as it will handle the backward compatibility properly. +

    + fullStackTrace is a better representation of the stack trace because it contains the whole + stack trace including the exception and its causes +

    + exception is the actual exception that caused the task to fail. It may be None in + the case that the exception is not in fact serializable. If a task fails more than + once (due to retries), exception is that one that caused the last failure.

    +
    +
    See Also:
    +
    Serialized Form
    +
    +
    • +
+
+
+ +
+
+
    +
  • + +
      +
    • + + +

      Constructor Detail

      + + + +
        +
      • +

        ExceptionFailure

        +
        public ExceptionFailure(String className,
+                        String description,
+                        StackTraceElement[] stackTrace,
+                        String fullStackTrace,
+                        scala.Option<org.apache.spark.ThrowableSerializationWrapper> exceptionWrapper,
+                        scala.collection.Seq<AccumulableInfo> accumUpdates,
+                        scala.collection.Seq<AccumulatorV2<?,?>> accums)
        +
        • +
      +
      • +
    + +
      +
    • + + +

      Method Detail

      + + + +
        +
      • +

        countTowardsTaskFailures

        +
        public static boolean countTowardsTaskFailures()
        +
        • +
      + + + +
        +
      • +

        canEqual

        +
        public abstract static boolean canEqual(Object that)
        +
        • +
      + + + +
        +
      • +

        equals

        +
        public abstract static boolean equals(Object that)
        +
        • +
      + + + +
        +
      • +

        productElement

        +
        public abstract static Object productElement(int n)
        +
        • +
      + + + +
        +
      • +

        productArity

        +
        public abstract static int productArity()
        +
        • +
      + + + +
        +
      • +

        productIterator

        +
        public static scala.collection.Iterator<Object> productIterator()
        +
        • +
      + + + +
        +
      • +

        productPrefix

        +
        public static String productPrefix()
        +
        • +
      + + + +
        +
      • +

        className

        +
        public String className()
        +
        • +
      + + + +
        +
      • +

        description

        +
        public String description()
        +
        • +
      + + + +
        +
      • +

        stackTrace

        +
        public StackTraceElement[] stackTrace()
        +
        • +
      + + + +
        +
      • +

        fullStackTrace

        +
        public String fullStackTrace()
        +
        • +
      + + + +
        +
      • +

        accumUpdates

        +
        public scala.collection.Seq<AccumulableInfo> accumUpdates()
        +
        • +
      + + + +
        +
      • +

        exception

        +
        public scala.Option<Throwable> exception()
        +
        • +
      + + + + +
      • +
    +
    • +
+
+
+ + +
+ + +
Skip navigation links
+ + + + +
+
+ + + +
+ +
+
+
    +
  • Summary: 
    • +
  • Nested | 
    • +
  • Field | 
    • +
  • Constr | 
    • +
  • Method
    • +
+ +
+ + +
+ + + --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org For additional commands, e-mail: commits-help@spark.apache.org