predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel O' Shaughnessy" <danieljamesda...@gmail.com>
Subject Re: Adding Spark Context (sc) to predict method?
Date Thu, 05 Oct 2017 21:07:44 GMT
Hi Shane,

Your RFAlgorithm class needs to use PAlgorithm instead of P2Algorithm. You
then need to write some code to save and load your model and spark context
etc.

There should be examples of this on the predictionio site somewhere

On Thu, Oct 5, 2017 at 10:00 PM Shane Johnson <shanewaldenjohnson@gmail.com>
wrote:

> Hi team,
>
> Can someone guide how I can add SparkContext into the predict method. I am
> using unique ids that I gather from Query and pulling back additional
> attributes from the PEventStore and am getting an error that "sc" cannot be
> found. When I add SparkContext to the method I get the following error.
>
> Can anyone provide direction here?
>
> Thank you
>
> Adding SparkContext eliminates the first error but produces another.
>
>   def predict(sc: SparkContext, model: RFModel, query: Query):
> PredictedResult = {
>
>     val featureIndex = model.featureIndex
>     val featureCategoricalIntMap = model.featureCategoricalIntMap
>
>
>     val responses: List[PredictionResponses] = query.predictionRequests
>     .map {
>         Predictions =>
>
>             val oppPost = PEventStore.aggregateProperties(
>               appName = sys.env("PIO_EVENTSERVER_APP_NAME"),
>               entityType = "Opportunity"
>             )(sc)
>
>
> First error when sc: SparkContext is not added to the method parameters:
>
> not found: value sc
> [INFO] [Engine$] [error]             )(sc)
> [INFO] [Engine$] [error]               ^
>
>
> Error after adding SparkContext:
>
> class RFAlgorithm needs to be abstract, since method predict in class
> P2LAlgorithm of type (model: org.template.liftscori
> ng.RFModel, query: org.template.liftscoring.Query)org.template.liftscoring
> .PredictedResult is not defined
> [INFO] [Engine$] [error] class RFAlgorithm(val ap: RFAlgorithmParams)
> [INFO] [Engine$] [error]       ^
> [INFO] [Engine$] [error] one error found
> [INFO] [Engine$] [error] (compile:compileIncremental) Compilation failed
> [INFO] [Engine$] [error] Total time: 6 s, completed Oct 5, 2017 2:44:51 PM
>
> *Shane Johnson | 801.360.3350 <(801)%20360-3350>*
> LinkedIn <https://www.linkedin.com/in/shanewjohnson> | Facebook
> <https://www.facebook.com/shane.johnson.71653>
>

Mime
View raw message