predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shane Johnson <shanewaldenjohn...@gmail.com>
Subject Re: Adding Spark Context (sc) to predict method?
Date Thu, 05 Oct 2017 21:13:49 GMT
Thanks Daniel. I had the P2Algorithm working before I had to query the
EventStore within the predict method. Do you think this is still the issue
with the context that I had it working before attempting to add the
SparkContext?

*Shane Johnson | 801.360.3350*
LinkedIn <https://www.linkedin.com/in/shanewjohnson> | Facebook
<https://www.facebook.com/shane.johnson.71653>

On Thu, Oct 5, 2017 at 3:07 PM, Daniel O' Shaughnessy <
danieljamesdavid@gmail.com> wrote:

> Hi Shane,
>
> Your RFAlgorithm class needs to use PAlgorithm instead of P2Algorithm.
> You then need to write some code to save and load your model and spark
> context etc.
>
> There should be examples of this on the predictionio site somewhere
>
> On Thu, Oct 5, 2017 at 10:00 PM Shane Johnson <
> shanewaldenjohnson@gmail.com> wrote:
>
>> Hi team,
>>
>> Can someone guide how I can add SparkContext into the predict method. I
>> am using unique ids that I gather from Query and pulling back additional
>> attributes from the PEventStore and am getting an error that "sc" cannot be
>> found. When I add SparkContext to the method I get the following error.
>>
>> Can anyone provide direction here?
>>
>> Thank you
>>
>> Adding SparkContext eliminates the first error but produces another.
>>
>>   def predict(sc: SparkContext, model: RFModel, query: Query):
>> PredictedResult = {
>>
>>     val featureIndex = model.featureIndex
>>     val featureCategoricalIntMap = model.featureCategoricalIntMap
>>
>>
>>     val responses: List[PredictionResponses] = query.predictionRequests
>>     .map {
>>         Predictions =>
>>
>>             val oppPost = PEventStore.aggregateProperties(
>>               appName = sys.env("PIO_EVENTSERVER_APP_NAME"),
>>               entityType = "Opportunity"
>>             )(sc)
>>
>>
>> First error when sc: SparkContext is not added to the method parameters:
>>
>> not found: value sc
>> [INFO] [Engine$] [error]             )(sc)
>> [INFO] [Engine$] [error]               ^
>>
>>
>> Error after adding SparkContext:
>>
>> class RFAlgorithm needs to be abstract, since method predict in class
>> P2LAlgorithm of type (model: org.template.liftscori
>> ng.RFModel, query: org.template.liftscoring.Query)org.template
>> .liftscoring.PredictedResult is not defined
>> [INFO] [Engine$] [error] class RFAlgorithm(val ap: RFAlgorithmParams)
>> [INFO] [Engine$] [error]       ^
>> [INFO] [Engine$] [error] one error found
>> [INFO] [Engine$] [error] (compile:compileIncremental) Compilation failed
>> [INFO] [Engine$] [error] Total time: 6 s, completed Oct 5, 2017 2:44:51
>> PM
>>
>> *Shane Johnson | 801.360.3350 <(801)%20360-3350>*
>> LinkedIn <https://www.linkedin.com/in/shanewjohnson> | Facebook
>> <https://www.facebook.com/shane.johnson.71653>
>>
>

Mime
View raw message