Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 591BD200C39 for ; Wed, 1 Mar 2017 22:04:30 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 57BBE160B56; Wed, 1 Mar 2017 21:04:30 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 1DC10160B78 for ; Wed, 1 Mar 2017 22:04:27 +0100 (CET) Received: (qmail 70925 invoked by uid 500); 1 Mar 2017 21:04:27 -0000 Mailing-List: contact commits-help@predictionio.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@predictionio.incubator.apache.org Delivered-To: mailing list commits@predictionio.incubator.apache.org Received: (qmail 70906 invoked by uid 99); 1 Mar 2017 21:04:27 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Mar 2017 21:04:27 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id B37151A001B for ; Wed, 1 Mar 2017 21:04:26 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -6.567 X-Spam-Level: X-Spam-Status: No, score=-6.567 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-2.999, SPF_NEUTRAL=0.652] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id pSBgDK6czgPX for ; Wed, 1 Mar 2017 21:04:13 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id A3A1D60D5F for ; Wed, 1 Mar 2017 21:04:02 +0000 (UTC) Received: (qmail 61539 invoked by uid 99); 1 Mar 2017 21:02:31 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Mar 2017 21:02:31 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 8F346F1761; Wed, 1 Mar 2017 21:02:31 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: git-site-role@apache.org To: commits@predictionio.incubator.apache.org Date: Wed, 01 Mar 2017 21:02:44 -0000 Message-Id: In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [14/51] [abbrv] [partial] incubator-predictionio-site git commit: Documentation based on apache/incubator-predictionio#877c29292a49fffca5ded3da919c7de464cf5e32 archived-at: Wed, 01 Mar 2017 21:04:30 -0000 http://git-wip-us.apache.org/repos/asf/incubator-predictionio-site/blob/8d79cb29/api/current/org/apache/predictionio/controller/PAlgorithm.html ---------------------------------------------------------------------- diff --git a/api/current/org/apache/predictionio/controller/PAlgorithm.html b/api/current/org/apache/predictionio/controller/PAlgorithm.html new file mode 100644 index 0000000..0294364 --- /dev/null +++ b/api/current/org/apache/predictionio/controller/PAlgorithm.html @@ -0,0 +1,649 @@ + + + + + PAlgorithm - PredictionIO Scala API 0.10.0-incubating - org.apache.predictionio.controller.PAlgorithm + + + + + + + + + + +
+ +

org.apache.predictionio.controller

+

PAlgorithm

+
+ +

+ + abstract + class + + + PAlgorithm[PD, M, Q, P] extends BaseAlgorithm[PD, M, Q, P] + +

+ +

Base class of a parallel algorithm.

A parallel algorithm can be run in parallel on a cluster and produces a +model that can also be distributed across a cluster.

If your input query class requires custom JSON4S serialization, the most +idiomatic way is to implement a trait that extends CustomQuerySerializer, +and mix that into your algorithm class, instead of overriding +querySerializer directly.

To provide evaluation feature, one must override and implement the +batchPredict method. Otherwise, an exception will be thrown when pio eval +is used. +

PD

Prepared data class.

M

Trained model class.

Q

Input query class.

P

Output prediction class.

+ Linear Supertypes +
BaseAlgorithm[PD, M, Q, P], BaseQuerySerializer, AbstractDoer, Serializable, Serializable, AnyRef, Any
+
+ + +
+
+
+ Ordering +
    + +
  1. Alphabetic
    2. +
  2. By inheritance
    3. +
+
+
+ Inherited
+ +
    +
  1. PAlgorithm
  2. BaseAlgorithm
  3. BaseQuerySerializer
  4. AbstractDoer
  5. Serializable
  6. Serializable
  7. AnyRef
  8. Any
    9. +
+
+ +
    +
  1. Hide All
    2. +
  2. Show all
    3. +
+ Learn more about member selection +
+
+ Visibility +
  1. Public
  2. All
+
+
+ +
+
+
+

Instance Constructors

+
  1. + + +

    + + + new + + + PAlgorithm() + +

    +

    +
+
+ + + +
+

Abstract Value Members

+
  1. + + +

    + + abstract + def + + + predict(model: M, query: Q): P + +

    +

    Implement this method to produce a prediction from a query and trained +model.

    Implement this method to produce a prediction from a query and trained +model. +

    model

    Trained model produced by train.

    query

    An input query.

    returns

    A prediction. +

    +
  2. + + +

    + + abstract + def + + + train(sc: SparkContext, pd: PD): M + +

    +

    Implement this method to produce a model from prepared data.

    Implement this method to produce a model from prepared data. +

    pd

    Prepared data for model training.

    returns

    Trained model. +

    +
+
+ +
+

Concrete Value Members

+
  1. + + +

    + + final + def + + + !=(arg0: AnyRef): Boolean + +

    +
    Definition Classes
    AnyRef
    +
  2. + + +

    + + final + def + + + !=(arg0: Any): Boolean + +

    +
    Definition Classes
    Any
    +
  3. + + +

    + + final + def + + + ##(): Int + +

    +
    Definition Classes
    AnyRef → Any
    +
  4. + + +

    + + final + def + + + ==(arg0: AnyRef): Boolean + +

    +
    Definition Classes
    AnyRef
    +
  5. + + +

    + + final + def + + + ==(arg0: Any): Boolean + +

    +
    Definition Classes
    Any
    +
  6. + + +

    + + final + def + + + asInstanceOf[T0]: T0 + +

    +
    Definition Classes
    Any
    +
  7. + + +

    + + + def + + + batchPredict(m: M, qs: RDD[(Long, Q)]): RDD[(Long, P)] + +

    +

    To provide evaluation feature, one must override and implement this method +to generate many predictions in batch.

    To provide evaluation feature, one must override and implement this method +to generate many predictions in batch. Otherwise, an exception will be +thrown when pio eval is used.

    The default implementation throws an exception. +

    m

    Trained model produced by train.

    qs

    An RDD of index-query tuples. The index is used to keep track of + predicted results with corresponding queries. +

    +
  8. + + +

    + + + def + + + batchPredictBase(sc: SparkContext, bm: Any, qs: RDD[(Long, Q)]): RDD[(Long, P)] + +

    +

    :: DeveloperApi :: +Engine developers should not use this directly.

    :: DeveloperApi :: +Engine developers should not use this directly. This is called by +evaluation workflow to perform batch prediction. +

    sc

    Spark context

    bm

    Model

    qs

    Batch of queries

    returns

    Batch of predicted results +

    Definition Classes
    PAlgorithm → BaseAlgorithm
    +
  9. + + +

    + + + def + + + clone(): AnyRef + +

    +
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    + @throws( + + ... + ) + +
    +
  10. + + +

    + + final + def + + + eq(arg0: AnyRef): Boolean + +

    +
    Definition Classes
    AnyRef
    +
  11. + + +

    + + + def + + + equals(arg0: Any): Boolean + +

    +
    Definition Classes
    AnyRef → Any
    +
  12. + + +

    + + + def + + + finalize(): Unit + +

    +
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    + @throws( + + classOf[java.lang.Throwable] + ) + +
    +
  13. + + +

    + + final + def + + + getClass(): Class[_] + +

    +
    Definition Classes
    AnyRef → Any
    +
  14. + + +

    + + + lazy val + + + gsonTypeAdapterFactories: Seq[TypeAdapterFactory] + +

    +

    :: DeveloperApi :: +Serializer for Java query classes using Gson +

    :: DeveloperApi :: +Serializer for Java query classes using Gson +

    Definition Classes
    BaseQuerySerializer
    +
  15. + + +

    + + + def + + + hashCode(): Int + +

    +
    Definition Classes
    AnyRef → Any
    +
  16. + + +

    + + final + def + + + isInstanceOf[T0]: Boolean + +

    +
    Definition Classes
    Any
    +
  17. + + +

    + + + def + + + makePersistentModel(sc: SparkContext, modelId: String, algoParams: Params, bm: Any): Any + +

    +

    :: DeveloperApi :: +Engine developers should not use this directly (read on to see how parallel +algorithm models are persisted).

    :: DeveloperApi :: +Engine developers should not use this directly (read on to see how parallel +algorithm models are persisted).

    In general, parallel models may contain multiple RDDs. It is not easy to +infer and persist them programmatically since these RDDs may be +potentially huge. To persist these models, engine developers need to mix +the PersistentModel trait into the model class and implement +PersistentModel.save. If it returns true, a +org.apache.predictionio.workflow.PersistentModelManifest will be +returned so that during deployment, PredictionIO will use +PersistentModelLoader to retrieve the model. Otherwise, Unit will be +returned and the model will be re-trained on-the-fly. +

    sc

    Spark context

    modelId

    Model ID

    algoParams

    Algorithm parameters that trained this model

    bm

    Model

    returns

    The model itself for automatic persistence, an instance of + org.apache.predictionio.workflow.PersistentModelManifest for manual + persistence, or Unit for re-training on deployment +

    Definition Classes
    PAlgorithm → BaseAlgorithm
    Annotations
    + @DeveloperApi() + +
    +
  18. + + +

    + + final + def + + + ne(arg0: AnyRef): Boolean + +

    +
    Definition Classes
    AnyRef
    +
  19. + + +

    + + final + def + + + notify(): Unit + +

    +
    Definition Classes
    AnyRef
    +
  20. + + +

    + + final + def + + + notifyAll(): Unit + +

    +
    Definition Classes
    AnyRef
    +
  21. + + +

    + + + def + + + predictBase(baseModel: Any, query: Q): P + +

    +

    :: DeveloperApi :: +Engine developers should not use this directly.

    :: DeveloperApi :: +Engine developers should not use this directly. Called by serving to +perform a single prediction. +

    returns

    Predicted result +

    Definition Classes
    PAlgorithm → BaseAlgorithm
    +
  22. + + +

    + + + def + + + queryClass: Class[Q] + +

    +

    :: DeveloperApi :: +Obtains the type signature of query for this algorithm +

    :: DeveloperApi :: +Obtains the type signature of query for this algorithm +

    returns

    Type signature of query +

    Definition Classes
    BaseAlgorithm
    +
  23. + + +

    + + + lazy val + + + querySerializer: Formats + +

    +

    :: DeveloperApi :: +Serializer for Scala query classes using +org.apache.predictionio.controller.Utils.json4sDefaultFormats +

    :: DeveloperApi :: +Serializer for Scala query classes using +org.apache.predictionio.controller.Utils.json4sDefaultFormats +

    Definition Classes
    BaseQuerySerializer
    +
  24. + + +

    + + final + def + + + synchronized[T0](arg0: ⇒ T0): T0 + +

    +
    Definition Classes
    AnyRef
    +
  25. + + +

    + + + def + + + toString(): String + +

    +
    Definition Classes
    AnyRef → Any
    +
  26. + + +

    + + + def + + + trainBase(sc: SparkContext, pd: PD): M + +

    +

    :: DeveloperApi :: +Engine developers should not use this directly.

    :: DeveloperApi :: +Engine developers should not use this directly. This is called by workflow +to train a model. +

    sc

    Spark context

    pd

    Prepared data

    returns

    Trained model +

    Definition Classes
    PAlgorithm → BaseAlgorithm
    +
  27. + + +

    + + final + def + + + wait(): Unit + +

    +
    Definition Classes
    AnyRef
    Annotations
    + @throws( + + ... + ) + +
    +
  28. + + +

    + + final + def + + + wait(arg0: Long, arg1: Int): Unit + +

    +
    Definition Classes
    AnyRef
    Annotations
    + @throws( + + ... + ) + +
    +
  29. + + +

    + + final + def + + + wait(arg0: Long): Unit + +

    +
    Definition Classes
    AnyRef
    Annotations
    + @throws( + + ... + ) + +
    +
+
+ + + + +
+ +
+
+

Inherited from BaseAlgorithm[PD, M, Q, P]

+
+

Inherited from BaseQuerySerializer

+
+

Inherited from AbstractDoer

+
+

Inherited from Serializable

+
+

Inherited from Serializable

+
+

Inherited from AnyRef

+
+

Inherited from Any

+
+ +
+ +
+
+

Ungrouped

+ +
+
+ +
+ +
+ + + + + \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-predictionio-site/blob/8d79cb29/api/current/org/apache/predictionio/controller/PDataSource.html ---------------------------------------------------------------------- diff --git a/api/current/org/apache/predictionio/controller/PDataSource.html b/api/current/org/apache/predictionio/controller/PDataSource.html new file mode 100644 index 0000000..df447fb --- /dev/null +++ b/api/current/org/apache/predictionio/controller/PDataSource.html @@ -0,0 +1,539 @@ + + + + + PDataSource - PredictionIO Scala API 0.10.0-incubating - org.apache.predictionio.controller.PDataSource + + + + + + + + + + +
+ +

org.apache.predictionio.controller

+

PDataSource

+
+ +

+ + abstract + class + + + PDataSource[TD, EI, Q, A] extends BaseDataSource[TD, EI, Q, A] + +

+ +

Base class of a parallel data source.

A parallel data source runs locally within a single machine, or in parallel +on a cluster, to return data that is distributed across a cluster. +

TD

Training data class.

EI

Evaluation Info class.

Q

Input query class.

A

Actual value class.

+ Linear Supertypes +
BaseDataSource[TD, EI, Q, A], AbstractDoer, Serializable, Serializable, AnyRef, Any
+
+ + +
+
+
+ Ordering +
    + +
  1. Alphabetic
    2. +
  2. By inheritance
    3. +
+
+
+ Inherited
+ +
    +
  1. PDataSource
  2. BaseDataSource
  3. AbstractDoer
  4. Serializable
  5. Serializable
  6. AnyRef
  7. Any
    8. +
+
+ +
    +
  1. Hide All
    2. +
  2. Show all
    3. +
+ Learn more about member selection +
+
+ Visibility +
  1. Public
  2. All
+
+
+ +
+
+
+

Instance Constructors

+
  1. + + +

    + + + new + + + PDataSource() + +

    +

    +
+
+ + + +
+

Abstract Value Members

+
  1. + + +

    + + abstract + def + + + readTraining(sc: SparkContext): TD + +

    +

    Implement this method to only return training data from a data source

    +
+
+ +
+

Concrete Value Members

+
  1. + + +

    + + final + def + + + !=(arg0: AnyRef): Boolean + +

    +
    Definition Classes
    AnyRef
    +
  2. + + +

    + + final + def + + + !=(arg0: Any): Boolean + +

    +
    Definition Classes
    Any
    +
  3. + + +

    + + final + def + + + ##(): Int + +

    +
    Definition Classes
    AnyRef → Any
    +
  4. + + +

    + + final + def + + + ==(arg0: AnyRef): Boolean + +

    +
    Definition Classes
    AnyRef
    +
  5. + + +

    + + final + def + + + ==(arg0: Any): Boolean + +

    +
    Definition Classes
    Any
    +
  6. + + +

    + + final + def + + + asInstanceOf[T0]: T0 + +

    +
    Definition Classes
    Any
    +
  7. + + +

    + + + def + + + clone(): AnyRef + +

    +
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    + @throws( + + ... + ) + +
    +
  8. + + +

    + + final + def + + + eq(arg0: AnyRef): Boolean + +

    +
    Definition Classes
    AnyRef
    +
  9. + + +

    + + + def + + + equals(arg0: Any): Boolean + +

    +
    Definition Classes
    AnyRef → Any
    +
  10. + + +

    + + + def + + + finalize(): Unit + +

    +
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    + @throws( + + classOf[java.lang.Throwable] + ) + +
    +
  11. + + +

    + + final + def + + + getClass(): Class[_] + +

    +
    Definition Classes
    AnyRef → Any
    +
  12. + + +

    + + + def + + + hashCode(): Int + +

    +
    Definition Classes
    AnyRef → Any
    +
  13. + + +

    + + final + def + + + isInstanceOf[T0]: Boolean + +

    +
    Definition Classes
    Any
    +
  14. + + +

    + + final + def + + + ne(arg0: AnyRef): Boolean + +

    +
    Definition Classes
    AnyRef
    +
  15. + + +

    + + final + def + + + notify(): Unit + +

    +
    Definition Classes
    AnyRef
    +
  16. + + +

    + + final + def + + + notifyAll(): Unit + +

    +
    Definition Classes
    AnyRef
    +
  17. + + +

    + + + def + + + readEval(sc: SparkContext): Seq[(TD, EI, RDD[(Q, A)])] + +

    +

    To provide evaluation feature for your engine, your must override this +method to return data for evaluation from a data source.

    To provide evaluation feature for your engine, your must override this +method to return data for evaluation from a data source. Returned data can +optionally include a sequence of query and actual value pairs for +evaluation purpose.

    The default implementation returns an empty sequence as a stub, so that +an engine can be compiled without implementing evaluation. +

    +
  18. + + +

    + + + def + + + readEvalBase(sc: SparkContext): Seq[(TD, EI, RDD[(Q, A)])] + +

    +

    :: DeveloperApi :: +Engine developer should not use this directly.

    :: DeveloperApi :: +Engine developer should not use this directly. This is called by +evaluation workflow to read training and validation data. +

    sc

    Spark context

    returns

    Sets of training data, evaluation information, queries, and actual + results +

    Definition Classes
    PDataSource → BaseDataSource
    +
  19. + + +

    + + + def + + + readTrainingBase(sc: SparkContext): TD + +

    +

    :: DeveloperApi :: +Engine developer should not use this directly.

    :: DeveloperApi :: +Engine developer should not use this directly. This is called by workflow +to read training data. +

    sc

    Spark context

    returns

    Training data +

    Definition Classes
    PDataSource → BaseDataSource
    +
  20. + + +

    + + final + def + + + synchronized[T0](arg0: ⇒ T0): T0 + +

    +
    Definition Classes
    AnyRef
    +
  21. + + +

    + + + def + + + toString(): String + +

    +
    Definition Classes
    AnyRef → Any
    +
  22. + + +

    + + final + def + + + wait(): Unit + +

    +
    Definition Classes
    AnyRef
    Annotations
    + @throws( + + ... + ) + +
    +
  23. + + +

    + + final + def + + + wait(arg0: Long, arg1: Int): Unit + +

    +
    Definition Classes
    AnyRef
    Annotations
    + @throws( + + ... + ) + +
    +
  24. + + +

    + + final + def + + + wait(arg0: Long): Unit + +

    +
    Definition Classes
    AnyRef
    Annotations
    + @throws( + + ... + ) + +
    +
+
+ + + +
+

Deprecated Value Members

+
  1. + + +

    + + + def + + + read(sc: SparkContext): Seq[(TD, EI, RDD[(Q, A)])] + +

    +
    Annotations
    + @deprecated + +
    Deprecated

    (Since version 0.9.0) Use readEval() instead.

    +
+
+
+ +
+
+

Inherited from BaseDataSource[TD, EI, Q, A]

+
+

Inherited from AbstractDoer

+
+

Inherited from Serializable

+
+

Inherited from Serializable

+
+

Inherited from AnyRef

+
+

Inherited from Any

+
+ +
+ +
+
+

Ungrouped

+ +
+
+ +
+ +
+ + + + + \ No newline at end of file