predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Gabrieli <dgabri...@salesforce.com>
Subject Re: pio train fails in Intelij - [Error] com.esotericsoftware.kryo.Kryo.setInstantiatorStrategy(Lorg/objenesis/strategy/InstantiatorStrategy;)V
Date Mon, 22 May 2017 23:58:58 GMT
Hi Shane,

I am not an expert but also came to PIO from a jupyter notebook background.

Yes, you can print and log dataframes as PIO is running.  But I've been
using the spark-shell interactively with the :paste and :load commands
heavily.  If you can get the dataframes loaded into the spark-shell then
you can interactively explore them directly in the REPL, which is nice.
>From pio you can also write your dataframe to csv files and then load it
into the spark-shell to work with that way.

As far as text editor support, I am using Sublime with Ensime
http://ensime.org/editors/sublime/ .   Ensime will catch type errors
without having to rebuild the program, which speeds up the cycle a bit.
Sublime doesn't have as many features as IntelliJ but it is probably easier
to set.




On Mon, May 22, 2017 at 4:31 PM Shane Johnson <shanewaldenjohnson@gmail.com>
wrote:

> My apologies, my last e-mail was sent on accident.
>
> *Primary Question:*
>
> I am following this tutorial (
> http://predictionio.incubator.apache.org/resources/intellij/) and am
> getting stuck running `pio train`. Error code is above.
>
> Exception in thread "main" java.lang.NoSuchMethodError:
> com.esotericsoftware.kryo.Kryo.setInstantiatorStrategy(
> Lorg/objenesis/strategy/InstantiatorStrategy;)V
>
> *Secondary Question:*
>
> What is a good way to print out the contents of a RDD or dataframe when
> working with PredictionIO. I am new to Scala and PredictionIO. My
> experience is in Jupyter Notebooks and python where you can view the data
> frames and objects as you are proving out the model. Is there a good method
> to do this within the predictionIO development cycles. I was thinking
> IntelliJ would be the answer but perhaps there is an easy way to print out
> the dataframes in the logs as well and view it when you run `pio build`.
>
> Your mentorship is very much appreciated.
>
> class TrainingData(
>   val ratings: RDD[Rating]
> ) extends Serializable {
>   override def toString = {
>     s"ratings: [${ratings.count()}] (${ratings.take(2).toList}...)"
>   }
> }
>
> *Shane Johnson | 801.360.3350 <(801)%20360-3350>*
> LinkedIn <https://www.linkedin.com/in/shanewjohnson> | Facebook
> <https://www.facebook.com/shane.johnson.71653>
>
> On Mon, May 22, 2017 at 5:23 PM, Shane Johnson <
> shanewaldenjohnson@gmail.com> wrote:
>
>> Team,
>>
>> I am trying to get IntelliJ working with predictionIO. I have been
>> following this turorial and am getting close to getting the PIO Train to
>> work but it looks like it is failing as it goes to persist the model. This
>> in turn seems to be causing the `pio deploy` in intelliJ to fail as well.
>>
>> What I really am trying to do is get into an environment where I can test
>> the code and view the dataframes as they are being created. I am new to
>> PredictionIO and Scala. I am trying to find a way to view the contents of class
>> TrainingData(
>>
>>   val ratings: RDD[Rating]
>> ) extends Serializable {
>>   override def toString = {
>>     s"ratings: [${ratings.count()}] (${ratings.take(2).toList}...)"
>>   }
>> }
>>
>>
>> [INFO] [Engine] Extracting datasource params...
>> [INFO] [WorkflowUtils$] No 'name' is found. Default empty String will be
>> used.
>> [INFO] [Engine] Datasource params: (,DataSourceParams(MyApp1,None))
>> [INFO] [Engine] Extracting preparator params...
>> [INFO] [Engine] Preparator params: (,Empty)
>> [INFO] [Engine] Extracting serving params...
>> [INFO] [Engine] Serving params: (,Empty)
>> [INFO] [Remoting] Starting remoting
>> [INFO] [Remoting] Remoting started; listening on addresses :[akka.tcp://
>> sparkDriverActorSystem@192.168.1.21:57099]
>> [WARN] [AbstractLifeCycle] FAILED SelectChannelConnector@0.0.0.0:4040:
>> java.net.BindException: Address already in use
>> [WARN] [AbstractLifeCycle] FAILED
>> org.spark-project.jetty.server.Server@53c1179a: java.net.BindException:
>> Address already in use
>> [WARN] [Utils] Service 'SparkUI' could not bind on port 4040. Attempting
>> port 4041.
>> [INFO] [Engine$] EngineWorkflow.train
>> [INFO] [Engine$] DataSource:
>> org.example.recommendation.DataSource@19bedeb8
>> [INFO] [Engine$] Preparator:
>> org.example.recommendation.Preparator@7ef8eda7
>> [INFO] [Engine$] AlgorithmList:
>> List(org.example.recommendation.ALSAlgorithm@7f287b98)
>> [INFO] [Engine$] Data sanity check is on.
>> [INFO] [Engine$] org.example.recommendation.TrainingData does not support
>> data sanity check. Skipping check.
>> [INFO] [Engine$] org.example.recommendation.PreparedData does not support
>> data sanity check. Skipping check.
>> [WARN] [BLAS] Failed to load implementation from:
>> com.github.fommil.netlib.NativeSystemBLAS
>> [WARN] [BLAS] Failed to load implementation from:
>> com.github.fommil.netlib.NativeRefBLAS
>> [WARN] [LAPACK] Failed to load implementation from:
>> com.github.fommil.netlib.NativeSystemLAPACK
>> [WARN] [LAPACK] Failed to load implementation from:
>> com.github.fommil.netlib.NativeRefLAPACK
>> [INFO] [Engine$] org.apache.spark.mllib.recommendation.ALSModel does not
>> support data sanity check. Skipping check.
>> [INFO] [Engine$] EngineWorkflow.train completed
>> [INFO] [Engine] engineInstanceId=6fb987b7-7674-4b95-a938-a1190e215638
>> [INFO] [CoreWorkflow$] Inserting persistent model
>> Exception in thread "main" java.lang.NoSuchMethodError:
>> com.esotericsoftware.kryo.Kryo.setInstantiatorStrategy(Lorg/objenesis/strategy/InstantiatorStrategy;)V
>> at com.twitter.chill.KryoBase.setInstantiatorStrategy(KryoBase.scala:86)
>> at
>> com.twitter.chill.EmptyScalaKryoInstantiator.newKryo(ScalaKryoInstantiator.scala:59)
>> at
>> com.twitter.chill.ScalaKryoInstantiator.newKryo(ScalaKryoInstantiator.scala:94)
>> at
>> org.apache.predictionio.workflow.KryoInstantiator.newKryo(CreateServer.scala:60)
>> at
>> org.apache.predictionio.workflow.KryoInstantiator.newKryo(CreateServer.scala:58)
>> at com.twitter.chill.KryoPool$2.newInstance(KryoPool.java:59)
>> at com.twitter.chill.KryoPool$2.newInstance(KryoPool.java:57)
>> at com.twitter.chill.ResourcePool.borrow(ResourcePool.java:37)
>> at com.twitter.chill.KryoPool.toBytesWithClass(KryoPool.java:114)
>> at com.twitter.chill.KryoInjectionInstance.apply(KryoInjection.scala:64)
>> at com.twitter.chill.KryoInjectionInstance.apply(KryoInjection.scala:55)
>> at
>> org.apache.predictionio.workflow.CoreWorkflow$.runTrain(CoreWorkflow.scala:81)
>> at
>> org.apache.predictionio.workflow.CreateWorkflow$.main(CreateWorkflow.scala:250)
>> at
>> org.apache.predictionio.workflow.CreateWorkflow.main(CreateWorkflow.scala)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
>>
>> *Shane Johnson | 801.360.3350 <(801)%20360-3350>*
>> LinkedIn <https://www.linkedin.com/in/shanewjohnson> | Facebook
>> <https://www.facebook.com/shane.johnson.71653>
>>
>
>

Mime
View raw message