predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shane Johnson <shanewaldenjohn...@gmail.com>
Subject Re: pio train fails in Intelij - [Error] com.esotericsoftware.kryo.Kryo.setInstantiatorStrategy(Lorg/objenesis/strategy/InstantiatorStrategy;)V
Date Tue, 23 May 2017 15:42:48 GMT
Daniel,

Thank you for your mentorship. I am using both of your methods

   - pio shell (:paste and :load)
   - ensime

 and I am cruising. I appreciate your help getting me past a roadblock and
will continue developing with sublime!

Thanks

*Shane Johnson | 801.360.3350*
LinkedIn <https://www.linkedin.com/in/shanewjohnson> | Facebook
<https://www.facebook.com/shane.johnson.71653>

On Mon, May 22, 2017 at 5:58 PM, Daniel Gabrieli <dgabrieli@salesforce.com>
wrote:

> Hi Shane,
>
> I am not an expert but also came to PIO from a jupyter notebook background.
>
> Yes, you can print and log dataframes as PIO is running.  But I've been
> using the spark-shell interactively with the :paste and :load commands
> heavily.  If you can get the dataframes loaded into the spark-shell then
> you can interactively explore them directly in the REPL, which is nice.
> From pio you can also write your dataframe to csv files and then load it
> into the spark-shell to work with that way.
>
> As far as text editor support, I am using Sublime with Ensime
> http://ensime.org/editors/sublime/ .   Ensime will catch type errors
> without having to rebuild the program, which speeds up the cycle a bit.
> Sublime doesn't have as many features as IntelliJ but it is probably easier
> to set.
>
>
>
>
> On Mon, May 22, 2017 at 4:31 PM Shane Johnson <
> shanewaldenjohnson@gmail.com> wrote:
>
>> My apologies, my last e-mail was sent on accident.
>>
>> *Primary Question:*
>>
>> I am following this tutorial (http://predictionio.incubator.apache.org/
>> resources/intellij/) and am getting stuck running `pio train`. Error
>> code is above.
>>
>> Exception in thread "main" java.lang.NoSuchMethodError:
>> com.esotericsoftware.kryo.Kryo.setInstantiatorStrategy(Lorg/
>> objenesis/strategy/InstantiatorStrategy;)V
>>
>> *Secondary Question:*
>>
>> What is a good way to print out the contents of a RDD or dataframe when
>> working with PredictionIO. I am new to Scala and PredictionIO. My
>> experience is in Jupyter Notebooks and python where you can view the data
>> frames and objects as you are proving out the model. Is there a good method
>> to do this within the predictionIO development cycles. I was thinking
>> IntelliJ would be the answer but perhaps there is an easy way to print out
>> the dataframes in the logs as well and view it when you run `pio build`.
>>
>> Your mentorship is very much appreciated.
>>
>> class TrainingData(
>>   val ratings: RDD[Rating]
>> ) extends Serializable {
>>   override def toString = {
>>     s"ratings: [${ratings.count()}] (${ratings.take(2).toList}...)"
>>   }
>> }
>>
>> *Shane Johnson | 801.360.3350 <(801)%20360-3350>*
>> LinkedIn <https://www.linkedin.com/in/shanewjohnson> | Facebook
>> <https://www.facebook.com/shane.johnson.71653>
>>
>> On Mon, May 22, 2017 at 5:23 PM, Shane Johnson <
>> shanewaldenjohnson@gmail.com> wrote:
>>
>>> Team,
>>>
>>> I am trying to get IntelliJ working with predictionIO. I have been
>>> following this turorial and am getting close to getting the PIO Train to
>>> work but it looks like it is failing as it goes to persist the model. This
>>> in turn seems to be causing the `pio deploy` in intelliJ to fail as well.
>>>
>>> What I really am trying to do is get into an environment where I can
>>> test the code and view the dataframes as they are being created. I am new
>>> to PredictionIO and Scala. I am trying to find a way to view the contents
>>> of class TrainingData(
>>>
>>>   val ratings: RDD[Rating]
>>> ) extends Serializable {
>>>   override def toString = {
>>>     s"ratings: [${ratings.count()}] (${ratings.take(2).toList}...)"
>>>   }
>>> }
>>>
>>>
>>> [INFO] [Engine] Extracting datasource params...
>>> [INFO] [WorkflowUtils$] No 'name' is found. Default empty String will be
>>> used.
>>> [INFO] [Engine] Datasource params: (,DataSourceParams(MyApp1,None))
>>> [INFO] [Engine] Extracting preparator params...
>>> [INFO] [Engine] Preparator params: (,Empty)
>>> [INFO] [Engine] Extracting serving params...
>>> [INFO] [Engine] Serving params: (,Empty)
>>> [INFO] [Remoting] Starting remoting
>>> [INFO] [Remoting] Remoting started; listening on addresses :[akka.tcp://
>>> sparkDriverActorSystem@192.168.1.21:57099]
>>> [WARN] [AbstractLifeCycle] FAILED SelectChannelConnector@0.0.0.0:4040:
>>> java.net.BindException: Address already in use
>>> [WARN] [AbstractLifeCycle] FAILED org.spark-project.jetty.
>>> server.Server@53c1179a: java.net.BindException: Address already in use
>>> [WARN] [Utils] Service 'SparkUI' could not bind on port 4040. Attempting
>>> port 4041.
>>> [INFO] [Engine$] EngineWorkflow.train
>>> [INFO] [Engine$] DataSource: org.example.recommendation.
>>> DataSource@19bedeb8
>>> [INFO] [Engine$] Preparator: org.example.recommendation.
>>> Preparator@7ef8eda7
>>> [INFO] [Engine$] AlgorithmList: List(org.example.
>>> recommendation.ALSAlgorithm@7f287b98)
>>> [INFO] [Engine$] Data sanity check is on.
>>> [INFO] [Engine$] org.example.recommendation.TrainingData does not
>>> support data sanity check. Skipping check.
>>> [INFO] [Engine$] org.example.recommendation.PreparedData does not
>>> support data sanity check. Skipping check.
>>> [WARN] [BLAS] Failed to load implementation from:
>>> com.github.fommil.netlib.NativeSystemBLAS
>>> [WARN] [BLAS] Failed to load implementation from:
>>> com.github.fommil.netlib.NativeRefBLAS
>>> [WARN] [LAPACK] Failed to load implementation from:
>>> com.github.fommil.netlib.NativeSystemLAPACK
>>> [WARN] [LAPACK] Failed to load implementation from:
>>> com.github.fommil.netlib.NativeRefLAPACK
>>> [INFO] [Engine$] org.apache.spark.mllib.recommendation.ALSModel does
>>> not support data sanity check. Skipping check.
>>> [INFO] [Engine$] EngineWorkflow.train completed
>>> [INFO] [Engine] engineInstanceId=6fb987b7-7674-4b95-a938-a1190e215638
>>> [INFO] [CoreWorkflow$] Inserting persistent model
>>> Exception in thread "main" java.lang.NoSuchMethodError:
>>> com.esotericsoftware.kryo.Kryo.setInstantiatorStrategy(
>>> Lorg/objenesis/strategy/InstantiatorStrategy;)V
>>> at com.twitter.chill.KryoBase.setInstantiatorStrategy(KryoBase.scala:86)
>>> at com.twitter.chill.EmptyScalaKryoInstantiator.
>>> newKryo(ScalaKryoInstantiator.scala:59)
>>> at com.twitter.chill.ScalaKryoInstantiator.newKryo(
>>> ScalaKryoInstantiator.scala:94)
>>> at org.apache.predictionio.workflow.KryoInstantiator.
>>> newKryo(CreateServer.scala:60)
>>> at org.apache.predictionio.workflow.KryoInstantiator.
>>> newKryo(CreateServer.scala:58)
>>> at com.twitter.chill.KryoPool$2.newInstance(KryoPool.java:59)
>>> at com.twitter.chill.KryoPool$2.newInstance(KryoPool.java:57)
>>> at com.twitter.chill.ResourcePool.borrow(ResourcePool.java:37)
>>> at com.twitter.chill.KryoPool.toBytesWithClass(KryoPool.java:114)
>>> at com.twitter.chill.KryoInjectionInstance.apply(KryoInjection.scala:64)
>>> at com.twitter.chill.KryoInjectionInstance.apply(KryoInjection.scala:55)
>>> at org.apache.predictionio.workflow.CoreWorkflow$.
>>> runTrain(CoreWorkflow.scala:81)
>>> at org.apache.predictionio.workflow.CreateWorkflow$.main(
>>> CreateWorkflow.scala:250)
>>> at org.apache.predictionio.workflow.CreateWorkflow.main(
>>> CreateWorkflow.scala)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke(
>>> NativeMethodAccessorImpl.java:62)
>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>> DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
>>>
>>> *Shane Johnson | 801.360.3350 <(801)%20360-3350>*
>>> LinkedIn <https://www.linkedin.com/in/shanewjohnson> | Facebook
>>> <https://www.facebook.com/shane.johnson.71653>
>>>
>>
>>

Mime
View raw message