Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 3D218200D14 for ; Tue, 29 Aug 2017 07:15:49 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 3B50D16610D; Tue, 29 Aug 2017 05:15:49 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id EA8B616610F for ; Tue, 29 Aug 2017 07:15:46 +0200 (CEST) Received: (qmail 31787 invoked by uid 500); 29 Aug 2017 05:15:46 -0000 Mailing-List: contact commits-help@predictionio.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@predictionio.incubator.apache.org Delivered-To: mailing list commits@predictionio.incubator.apache.org Received: (qmail 31725 invoked by uid 99); 29 Aug 2017 05:15:45 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Aug 2017 05:15:45 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 3361E183E33 for ; Tue, 29 Aug 2017 05:15:44 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.522 X-Spam-Level: X-Spam-Status: No, score=-4.522 tagged_above=-999 required=6.31 tests=[KAM_NUMSUBJECT=0.5, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id OUgefFwOODRk for ; Tue, 29 Aug 2017 05:15:27 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id 7F4DB60EF3 for ; Tue, 29 Aug 2017 05:15:25 +0000 (UTC) Received: (qmail 30452 invoked by uid 99); 29 Aug 2017 05:15:24 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Aug 2017 05:15:24 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id DB74EF5547; Tue, 29 Aug 2017 05:15:23 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: git-site-role@apache.org To: commits@predictionio.incubator.apache.org Date: Tue, 29 Aug 2017 05:15:28 -0000 Message-Id: In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [06/51] [abbrv] [partial] incubator-predictionio-site git commit: Documentation based on apache/incubator-predictionio#9c68d86c61a06b08d618b8cd8177c9d76cd86618 archived-at: Tue, 29 Aug 2017 05:15:49 -0000 http://git-wip-us.apache.org/repos/asf/incubator-predictionio-site/blob/f6f3c69c/api/current/org/apache/predictionio/controller/package.html ---------------------------------------------------------------------- diff --git a/api/current/org/apache/predictionio/controller/package.html b/api/current/org/apache/predictionio/controller/package.html new file mode 100644 index 0000000..c500988 --- /dev/null +++ b/api/current/org/apache/predictionio/controller/package.html @@ -0,0 +1,1149 @@ + + + + + controller - PredictionIO Scala API 0.11.0-incubating - org.apache.predictionio.controller + + + + + + + + + + +
+ +

org.apache.predictionio

+

controller

+
+ +

+ + + package + + + controller + +

+ +

Provides building blocks for writing a complete prediction engine +consisting of DataSource, Preparator, Algorithm, Serving, and Evaluation.

Start Building an Engine

The starting point of a prediction engine is the Engine class.

The DASE Paradigm

The building blocks together form the DASE paradigm. Learn more about DASE +here.

Types of Building Blocks

Depending on the problem you are solving, you would need to pick appropriate +flavors of building blocks.

Engines

There are 3 typical engine configurations:

  1. PDataSource, PPreparator, P2LAlgorithm, LServing + 2. PDataSource, PPreparator, PAlgorithm, LServing + 3. LDataSource, LPreparator, LAlgorithm, LServing

In both configurations 1 and 2, data is sourced and prepared in a +parallelized fashion, with data type as RDD.

The difference between configurations 1 and 2 come at the algorithm stage. +In configuration 1, the algorithm operates on potentially large data as RDDs +in the Spark cluster, and eventually outputs a model that is small enough to +fit in a single machine.

On the other hand, configuration 2 outputs a model that is potentially too +large to fit in a single machine, and must reside in the Spark cluster as +RDD(s).

With configuration 1 (P2LAlgorithm), PredictionIO will automatically +try to persist the model to local disk or HDFS if the model is serializable.

With configuration 2 (PAlgorithm), PredictionIO will not automatically +try to persist the model, unless the model implements the PersistentModel +trait.

In special circumstances where both the data and the model are small, +configuration 3 may be used. Beware that RDDs cannot be used with +configuration 3.

Data Source

PDataSource is probably the most used data source base class with the +ability to process RDD-based data. LDataSource cannot handle +RDD-based data. Use only when you have a special requirement.

Preparator

With PDataSource, you must pick PPreparator. The same applies to +LDataSource and LPreparator.

Algorithm

The workhorse of the engine comes in 3 different flavors.

P2LAlgorithm

Produces a model that is small enough to fit in a single machine from +PDataSource and PPreparator. The model cannot contain any RDD. +If the produced model is serializable, PredictionIO will try to +automatically persist it. In addition, P2LAlgorithm.batchPredict is +already implemented for Evaluation purpose.

PAlgorithm

Produces a model that could contain RDDs from PDataSource and +PPreparator. PredictionIO will not try to persist it automatically +unless the model implements PersistentModel. PAlgorithm.batchPredict +must be implemented for Evaluation.

LAlgorithm

Produces a model that is small enough to fit in a single machine from +LDataSource and LPreparator. The model cannot contain any RDD. +If the produced model is serializable, PredictionIO will try to +automatically persist it. In addition, LAlgorithm.batchPredict is +already implemented for Evaluation purpose.

Serving

The serving component comes with only 1 flavor--LServing. At the serving +stage, it is assumed that the result being served is already at a human- +consumable size.

Model Persistence

PredictionIO tries its best to persist trained models automatically. Please +refer to LAlgorithm.makePersistentModel, +P2LAlgorithm.makePersistentModel, and PAlgorithm.makePersistentModel +for descriptions on different strategies. +

+ Linear Supertypes +
AnyRef, Any
+
+ + +
+
+
+ Ordering +
    +
  1. Grouped
    2. +
  2. Alphabetic
    3. +
  3. By inheritance
    4. +
+
+
+ Inherited
+ +
    +
  1. controller
  2. AnyRef
  3. Any
    4. +
+
+ +
    +
  1. Hide All
    2. +
  2. Show all
    3. +
+ Learn more about member selection +
+
+ Visibility +
  1. Public
  2. All
+
+
+ +
+
+ + +
+

Type Members

+
  1. + + +

    + + abstract + class + + + AverageMetric[EI, Q, P, A] extends Metric[EI, Q, P, A, Double] with StatsMetricHelper[EI, Q, P, A] with QPAMetric[Q, P, A, Double] + +

    +

    Returns the global average of the score returned by the calculate method.

    +
  2. + + +

    + + + trait + + + CustomQuerySerializer extends BaseQuerySerializer + +

    +

    If your query class cannot be automatically serialized/deserialized to/from +JSON, implement a trait by extending this trait, and overriding the +querySerializer member with your +custom JSON4S serializer.

    +
  3. + + +

    + + + trait + + + Deployment extends EngineFactory + +

    +

    Defines a deployment that contains an Engine +

    +
  4. + + +

    + + + type + + + EmptyActualResult = SerializableClass + +

    +

    Empty actual result.

    +
  5. + + +

    + + + type + + + EmptyAlgorithmParams = EmptyParams + +

    +

    Empty algorithm parameters.

    +
  6. + + +

    + + + type + + + EmptyDataParams = EmptyParams + +

    +

    Empty data parameters.

    +
  7. + + +

    + + + type + + + EmptyDataSourceParams = EmptyParams + +

    +

    Empty data source parameters.

    +
  8. + + +

    + + + type + + + EmptyEvaluationInfo = SerializableClass + +

    +

    Empty evaluation info.

    +
  9. + + +

    + + + type + + + EmptyMetricsParams = EmptyParams + +

    +

    Empty metrics parameters.

    +
  10. + + +

    + + + type + + + EmptyModel = SerializableClass + +

    +

    Empty model.

    +
  11. + + +

    + + + case class + + + EmptyParams() extends Params with Product with Serializable + +

    +

    A concrete implementation of Params representing empty parameters.

    +
  12. + + +

    + + + type + + + EmptyPreparatorParams = EmptyParams + +

    +

    Empty preparator parameters.

    +
  13. + + +

    + + + type + + + EmptyPreparedData = SerializableClass + +

    +

    Empty prepared data.

    +
  14. + + +

    + + + type + + + EmptyServingParams = EmptyParams + +

    +

    Empty serving parameters.

    +
  15. + + +

    + + + type + + + EmptyTrainingData = SerializableClass + +

    +

    Empty training data.

    +
  16. + + +

    + + + class + + + Engine[TD, EI, PD, Q, P, A] extends BaseEngine[EI, Q, P, A] + +

    +

    This class chains up the entire data process.

    +
  17. + + +

    + + abstract + class + + + EngineFactory extends AnyRef + +

    +

    If you intend to let PredictionIO create workflow and deploy serving +automatically, you will need to implement an object that extends this class +and return an Engine.

    +
  18. + + +

    + + + class + + + EngineParams extends Serializable + +

    +

    This class serves as a logical grouping of all required engine's parameters.

    +
  19. + + +

    + + + trait + + + EngineParamsGenerator extends AnyRef + +

    +

    Defines an engine parameters generator.

    +
  20. + + +

    + + + trait + + + Evaluation extends EngineFactory with Deployment + +

    +

    Defines an evaluation that contains an engine and a metric.

    +
  21. + + +

    + + + class + + + FastEvalEngine[TD, EI, PD, Q, P, A] extends Engine[TD, EI, PD, Q, P, A] + +

    +

    :: Experimental :: +FastEvalEngine is a subclass of Engine that exploits the immutability of +controllers to optimize the evaluation process +

    +
  22. + + +

    + + + class + + + FastEvalEngineWorkflow[TD, EI, PD, Q, P, A] extends Serializable + +

    +

    :: Experimental :: +Workflow based on FastEvalEngine +

    +
  23. + + +

    + + + class + + + IdentityPreparator[TD] extends BasePreparator[TD, TD] + +

    +

    A helper concrete implementation of org.apache.predictionio.core.BasePreparator +that passes training data through without any special preparation.

    +
  24. + + +

    + + abstract + class + + + LAlgorithm[PD, M, Q, P] extends BaseAlgorithm[RDD[PD], RDD[M], Q, P] + +

    +

    Base class of a local algorithm.

    +
  25. + + +

    + + + class + + + LAverageServing[Q] extends LServing[Q, Double] + +

    +

    A concrete implementation of LServing returning the average of all +algorithms' predictions, where their classes are expected to be all Double.

    +
  26. + + +

    + + abstract + class + + + LDataSource[TD, EI, Q, A] extends BaseDataSource[RDD[TD], EI, Q, A] + +

    +

    Base class of a local data source.

    +
  27. + + +

    + + + class + + + LFirstServing[Q, P] extends LServing[Q, P] + +

    +

    A concrete implementation of LServing returning the first algorithm's +prediction result directly without any modification.

    +
  28. + + +

    + + + class + + + LIdentityPreparator[TD] extends IdentityPreparator[TD] + +

    +

    DEPRECATED.

    +
  29. + + +

    + + abstract + class + + + LPreparator[TD, PD] extends BasePreparator[RDD[TD], RDD[PD]] + +

    +

    Base class of a local preparator.

    +
  30. + + +

    + + abstract + class + + + LServing[Q, P] extends BaseServing[Q, P] + +

    +

    Base class of serving.

    +
  31. + + +

    + + + trait + + + LocalFileSystemPersistentModel[AP <: Params] extends PersistentModel[AP] + +

    +

    This trait is a convenience helper for persisting your model to the local +filesystem.

    +
  32. + + +

    + + + trait + + + LocalFileSystemPersistentModelLoader[AP <: Params, M] extends PersistentModelLoader[AP, M] + +

    +

    Implement an object that extends this trait for PredictionIO to support +loading a persisted model from local filesystem during serving deployment.

    +
  33. + + +

    + + abstract + class + + + Metric[EI, Q, P, A, R] extends Serializable + +

    +

    Base class of a Metric.

    +
  34. + + +

    + + + class + + + MetricEvaluator[EI, Q, P, A, R] extends BaseEvaluator[EI, Q, P, A, MetricEvaluatorResult[R]] + +

    +

    :: DeveloperApi :: +Do no use this directly.

    +
  35. + + +

    + + + case class + + + MetricEvaluatorResult[R](bestScore: MetricScores[R], bestEngineParams: EngineParams, bestIdx: Int, metricHeader: String, otherMetricHeaders: Seq[String], engineParamsScores: Seq[(EngineParams, MetricScores[R])], outputPath: Option[String]) extends BaseEvaluatorResult with Product with Serializable + +

    +

    Contains all results of a MetricEvaluator +

    +
  36. + + +

    + + + case class + + + MetricScores[R](score: R, otherScores: Seq[Any]) extends Product with Serializable + +

    +

    Case class storing a primary score, and other scores +

    +
  37. + + +

    + + abstract + class + + + OptionAverageMetric[EI, Q, P, A] extends Metric[EI, Q, P, A, Double] with StatsOptionMetricHelper[EI, Q, P, A] with QPAMetric[Q, P, A, Option[Double]] + +

    +

    Returns the global average of the non-None score returned by the calculate +method.

    +
  38. + + +

    + + abstract + class + + + OptionStdevMetric[EI, Q, P, A] extends Metric[EI, Q, P, A, Double] with StatsOptionMetricHelper[EI, Q, P, A] with QPAMetric[Q, P, A, Option[Double]] + +

    +

    Returns the global standard deviation of the non-None score returned by the calculate method

    +
  39. + + +

    + + abstract + class + + + P2LAlgorithm[PD, M, Q, P] extends BaseAlgorithm[PD, M, Q, P] + +

    +

    Base class of a parallel-to-local algorithm.

    +
  40. + + +

    + + abstract + class + + + PAlgorithm[PD, M, Q, P] extends BaseAlgorithm[PD, M, Q, P] + +

    +

    Base class of a parallel algorithm.

    +
  41. + + +

    + + abstract + class + + + PDataSource[TD, EI, Q, A] extends BaseDataSource[TD, EI, Q, A] + +

    +

    Base class of a parallel data source.

    +
  42. + + +

    + + + class + + + PIdentityPreparator[TD] extends IdentityPreparator[TD] + +

    +

    DEPRECATED.

    +
  43. + + +

    + + abstract + class + + + PPreparator[TD, PD] extends BasePreparator[TD, PD] + +

    +

    Base class of a parallel preparator.

    +
  44. + + +

    + + + trait + + + Params extends Serializable + +

    +

    Base trait for all kinds of parameters that will be passed to constructors +of different controller classes.

    +
  45. + + +

    + + + trait + + + PersistentModel[AP <: Params] extends AnyRef + +

    +

    Mix in and implement this trait if your model cannot be persisted by +PredictionIO automatically.

    +
  46. + + +

    + + + trait + + + PersistentModelLoader[AP <: Params, M] extends AnyRef + +

    +

    Implement an object that extends this trait for PredictionIO to support +loading a persisted model during serving deployment.

    +
  47. + + +

    + + + trait + + + QPAMetric[Q, P, A, R] extends AnyRef + +

    +

    Trait for metric which returns a score based on Query, PredictedResult, +and ActualResult +

    +
  48. + + +

    + + + trait + + + SanityCheck extends AnyRef + +

    +

    Extends a data class with this trait if you want PredictionIO to +automatically perform sanity check on your data classes during training.

    +
  49. + + +

    + + + class + + + SerializableClass extends Serializable + +

    +

    Base class of several helper types that represent emptiness +

    +
  50. + + +

    + + + class + + + SimpleEngine[TD, EI, Q, P, A] extends Engine[TD, EI, TD, Q, P, A] + +

    +

    SimpleEngine has only one algorithm, and uses default preparator and serving +layer.

    +
  51. + + +

    + + + class + + + SimpleEngineParams extends EngineParams + +

    +

    This shorthand class serves the SimpleEngine class.

    +
  52. + + +

    + + abstract + class + + + StdevMetric[EI, Q, P, A] extends Metric[EI, Q, P, A, Double] with StatsMetricHelper[EI, < span class="extype" name="org.apache.predictionio.controller.StdevMetric.Q">Q, P, A] with QPAMetric[Q, P, A, Double] + +

    +

    Returns the global standard deviation of the score returned by the calculate method

    +
  53. + + +

    + + abstract + class + + + SumMetric[EI, Q, P, A, R] extends Metric[EI, Q, P, A, R] with QPAMetric[Q, P, A, R] + +

    +

    Returns the sum of the score returned by the calculate method.

    +
  54. + + +

    + + + class + + + ZeroMetric[EI, Q, P, A] extends Metric[EI, Q, P, A, Double] + +

    +

    Returns zero.

    +
  55. + + +

    + + + trait + + + IEngineFactory extends EngineFactory + +

    +

    DEPRECATED.

    +
  56. + + +

    + + + trait + + + IFSPersistentModel[AP <: Params] extends LocalFileSystemPersistentModel[AP] + +

    +

    DEPRECATED.

    +
  57. + + +

    + + + trait + + + IFSPersistentModelLoader[AP <: Params, M] extends LocalFileSystemPersistentModelLoader[AP, M] + +

    +

    DEPRECATED.

    +
  58. + + +

    + + + trait + + + IPersistentModel[AP <: Params] extends PersistentModel[AP] + +

    +

    DEPRECATED.

    +
  59. + + +

    + + + trait + + + IPersistentModelLoader[AP <: Params, M] extends PersistentModelLoader[AP, M] + +

    +

    DEPRECATED.

    +
  60. + + +

    + + + trait + + + WithPrId extends AnyRef + +

    +

    Mix in this trait for queries that contain prId (PredictedResultId).

    +
  61. + + +

    + + + trait + + + WithQuerySerializer extends CustomQuerySerializer + +

    +

    DEPRECATED.

    +
+
+ + + +
+

Value Members

+
  1. + + +

    + + + object + + + Engine extends Serializable + +

    +

    This object contains concrete implementation for some methods of the +Engine class.

    +
  2. + + +

    + + + object + + + EngineParams extends Serializable + +

    +

    Companion object for creating EngineParams instances.

    +
  3. + + +

    + + + object + + + FastEvalEngineWorkflow extends Serializable + +

    +

    :: Experimental :: +Workflow based on FastEvalEngine +

    +
  4. + + +

    + + + object + + + IdentityPreparator extends Serializable + +

    +

    Companion object of