Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id B559A200C8C for ; Tue, 23 May 2017 01:31:29 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id B123C160BD4; Mon, 22 May 2017 23:31:29 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 10812160BBF for ; Tue, 23 May 2017 01:31:27 +0200 (CEST) Received: (qmail 37754 invoked by uid 500); 22 May 2017 23:31:27 -0000 Mailing-List: contact user-help@predictionio.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@predictionio.incubator.apache.org Delivered-To: mailing list user@predictionio.incubator.apache.org Received: (qmail 37739 invoked by uid 99); 22 May 2017 23:31:26 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 22 May 2017 23:31:26 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 664B0C05AA for ; Mon, 22 May 2017 23:31:26 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.895 X-Spam-Level: X-Spam-Status: No, score=-0.895 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, NORMAL_HTTP_TO_IP=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-2.796, SPF_PASS=-0.001, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id cHP7S0XVMyQI for ; Mon, 22 May 2017 23:31:23 +0000 (UTC) Received: from mail-ua0-f175.google.com (mail-ua0-f175.google.com [209.85.217.175]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id E47485FDFE for ; Mon, 22 May 2017 23:31:22 +0000 (UTC) Received: by mail-ua0-f175.google.com with SMTP id j17so68515050uag.3 for ; Mon, 22 May 2017 16:31:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=2ePJcEMu+4WjMw2KXRS2SKVWaCDj//fN5IXunf5dCas=; b=BTKvcfoxuEDZB/X4VttzTGiEjRztnuGOHoAf5aoQJFiBVGYDw7BcwSYIDypakNjGF9 bUhvFMsSSnhTU/hReEbDoNS5bsBdkC0QsZvjSsoxjkZbJoaS6mSo5aUpQroySlxUP9AW EOLkpA/4gQNtc7lq+ztK5WfW1h+q/fiaR3UeJF7eq+K5KTzao6uuWzE2QNOvB/TVwVLr uxAJMLIXmf/x2ZURG6yqYv690xqSu1VqDla49ec2n7aIZ4VQ25XIVwvnDr5+T+dgRj4s Cr2fVydyNbZtqz0UuGlbzUfpDjFboEP8qL8BShfYgOOKwY0k6J79HW/FUpKkysy/ET8b kgKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=2ePJcEMu+4WjMw2KXRS2SKVWaCDj//fN5IXunf5dCas=; b=anFnAA5H1+cx5g6elRrnGPvdNP0sXRdWjPeC+fzelJ7bS06bX+J6Bpikjhrf93Npxj tgxgXJZsZuOyr4nxNzg1H62d3qfIucdY8YfBNbHQy4oYN0Pu7ZGThKnbshxiHF9fdZZC M15rmZhoxMmJI2/Gw6CLRHHbzpV8Kl3nT/4OJiQVjdZrEBtFDfPyqIpYOfEjxMB+qd0Z bfSvDFktz2dnVfCL+Y0RfvdudMO7uWG/V8huFPWolh+YtKSwWpyq0pffM55ChrqG4vSs Nc271Vvm3VckvJjtwMcG8akkzt6lcu2nvrBM0nWPhBqoatASinGtfP/zCNlb4Sbth5p5 unrA== X-Gm-Message-State: AODbwcCr3AY+beJ8WZKHTEvcWThNlkJrBS5p1paADE0fk44j+ylZVYEh aDuPok4Bbc7DQ5wd2vw3eiP/3GAvOKjj X-Received: by 10.176.81.73 with SMTP id f9mr12901542uaa.41.1495495881637; Mon, 22 May 2017 16:31:21 -0700 (PDT) MIME-Version: 1.0 Received: by 10.159.39.193 with HTTP; Mon, 22 May 2017 16:31:21 -0700 (PDT) In-Reply-To: References: From: Shane Johnson Date: Mon, 22 May 2017 17:31:21 -0600 Message-ID: Subject: Re: pio train fails in Intelij - [Error] com.esotericsoftware.kryo.Kryo.setInstantiatorStrategy(Lorg/objenesis/strategy/InstantiatorStrategy;)V To: user@predictionio.incubator.apache.org Content-Type: multipart/alternative; boundary="94eb2c191536c9c60d055025440e" archived-at: Mon, 22 May 2017 23:31:29 -0000 --94eb2c191536c9c60d055025440e Content-Type: text/plain; charset="UTF-8" My apologies, my last e-mail was sent on accident. *Primary Question:* I am following this tutorial ( http://predictionio.incubator.apache.org/resources/intellij/) and am getting stuck running `pio train`. Error code is above. Exception in thread "main" java.lang.NoSuchMethodError: com.esotericsoftware.kryo.Kryo.setInstantiatorStrategy( Lorg/objenesis/strategy/InstantiatorStrategy;)V *Secondary Question:* What is a good way to print out the contents of a RDD or dataframe when working with PredictionIO. I am new to Scala and PredictionIO. My experience is in Jupyter Notebooks and python where you can view the data frames and objects as you are proving out the model. Is there a good method to do this within the predictionIO development cycles. I was thinking IntelliJ would be the answer but perhaps there is an easy way to print out the dataframes in the logs as well and view it when you run `pio build`. Your mentorship is very much appreciated. class TrainingData( val ratings: RDD[Rating] ) extends Serializable { override def toString = { s"ratings: [${ratings.count()}] (${ratings.take(2).toList}...)" } } *Shane Johnson | 801.360.3350* LinkedIn | Facebook On Mon, May 22, 2017 at 5:23 PM, Shane Johnson wrote: > Team, > > I am trying to get IntelliJ working with predictionIO. I have been > following this turorial and am getting close to getting the PIO Train to > work but it looks like it is failing as it goes to persist the model. This > in turn seems to be causing the `pio deploy` in intelliJ to fail as well. > > What I really am trying to do is get into an environment where I can test > the code and view the dataframes as they are being created. I am new to > PredictionIO and Scala. I am trying to find a way to view the contents of class > TrainingData( > > val ratings: RDD[Rating] > ) extends Serializable { > override def toString = { > s"ratings: [${ratings.count()}] (${ratings.take(2).toList}...)" > } > } > > > [INFO] [Engine] Extracting datasource params... > [INFO] [WorkflowUtils$] No 'name' is found. Default empty String will be > used. > [INFO] [Engine] Datasource params: (,DataSourceParams(MyApp1,None)) > [INFO] [Engine] Extracting preparator params... > [INFO] [Engine] Preparator params: (,Empty) > [INFO] [Engine] Extracting serving params... > [INFO] [Engine] Serving params: (,Empty) > [INFO] [Remoting] Starting remoting > [INFO] [Remoting] Remoting started; listening on addresses :[akka.tcp:// > sparkDriverActorSystem@192.168.1.21:57099] > [WARN] [AbstractLifeCycle] FAILED SelectChannelConnector@0.0.0.0:4040: > java.net.BindException: Address already in use > [WARN] [AbstractLifeCycle] FAILED org.spark-project.jetty. > server.Server@53c1179a: java.net.BindException: Address already in use > [WARN] [Utils] Service 'SparkUI' could not bind on port 4040. Attempting > port 4041. > [INFO] [Engine$] EngineWorkflow.train > [INFO] [Engine$] DataSource: org.example.recommendation. > DataSource@19bedeb8 > [INFO] [Engine$] Preparator: org.example.recommendation. > Preparator@7ef8eda7 > [INFO] [Engine$] AlgorithmList: List(org.example. > recommendation.ALSAlgorithm@7f287b98) > [INFO] [Engine$] Data sanity check is on. > [INFO] [Engine$] org.example.recommendation.TrainingData does not support > data sanity check. Skipping check. > [INFO] [Engine$] org.example.recommendation.PreparedData does not support > data sanity check. Skipping check. > [WARN] [BLAS] Failed to load implementation from: com.github.fommil.netlib. > NativeSystemBLAS > [WARN] [BLAS] Failed to load implementation from: com.github.fommil.netlib. > NativeRefBLAS > [WARN] [LAPACK] Failed to load implementation from: > com.github.fommil.netlib.NativeSystemLAPACK > [WARN] [LAPACK] Failed to load implementation from: > com.github.fommil.netlib.NativeRefLAPACK > [INFO] [Engine$] org.apache.spark.mllib.recommendation.ALSModel does not > support data sanity check. Skipping check. > [INFO] [Engine$] EngineWorkflow.train completed > [INFO] [Engine] engineInstanceId=6fb987b7-7674-4b95-a938-a1190e215638 > [INFO] [CoreWorkflow$] Inserting persistent model > Exception in thread "main" java.lang.NoSuchMethodError: > com.esotericsoftware.kryo.Kryo.setInstantiatorStrategy( > Lorg/objenesis/strategy/InstantiatorStrategy;)V > at com.twitter.chill.KryoBase.setInstantiatorStrategy(KryoBase.scala:86) > at com.twitter.chill.EmptyScalaKryoInstantiator. > newKryo(ScalaKryoInstantiator.scala:59) > at com.twitter.chill.ScalaKryoInstantiator.newKryo( > ScalaKryoInstantiator.scala:94) > at org.apache.predictionio.workflow.KryoInstantiator. > newKryo(CreateServer.scala:60) > at org.apache.predictionio.workflow.KryoInstantiator. > newKryo(CreateServer.scala:58) > at com.twitter.chill.KryoPool$2.newInstance(KryoPool.java:59) > at com.twitter.chill.KryoPool$2.newInstance(KryoPool.java:57) > at com.twitter.chill.ResourcePool.borrow(ResourcePool.java:37) > at com.twitter.chill.KryoPool.toBytesWithClass(KryoPool.java:114) > at com.twitter.chill.KryoInjectionInstance.apply(KryoInjection.scala:64) > at com.twitter.chill.KryoInjectionInstance.apply(KryoInjection.scala:55) > at org.apache.predictionio.workflow.CoreWorkflow$. > runTrain(CoreWorkflow.scala:81) > at org.apache.predictionio.workflow.CreateWorkflow$.main( > CreateWorkflow.scala:250) > at org.apache.predictionio.workflow.CreateWorkflow.main( > CreateWorkflow.scala) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147) > > *Shane Johnson | 801.360.3350 <(801)%20360-3350>* > LinkedIn | Facebook > > --94eb2c191536c9c60d055025440e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
My apologies, my last e-mail was sent on accident.

Primary Question:

I = am following this tutorial (http://predictionio.incubator.apache.org/resource= s/intellij/) and am getting stuck running `pio train`. Error code is ab= ove.

Exception in= thread "main" java.lang.NoSuchMethodError: com.esotericsoftware.= kryo.Lorg/objenesis/strategy/InstantiatorStrategy;)V

Secondary Question:
=
What is a good way to print out the contents of a RDD= or dataframe when working with PredictionIO. I am new to Scala and Predict= ionIO. My experience is in Jupyter Notebooks and python where you can view = the data frames and objects as you are proving out the model. Is there a go= od method to do this within the predictionIO development cycles. I was thin= king IntelliJ would be the answer but perhaps there is an easy way to print= out the dataframes in the logs as well and view it when you run `pio build= `.

Your mentorship is very much appreciated.
= 
class TrainingData(
  val ratings: RDD[Rating]
) extends Serializable {
  =
override def =
toString =3D {
    s"ratings: [${ratings.count()}] (${ratings.take(2).toList}...)"
  }
}

Shane Johnson | 801.360.3350

LinkedIn=C2=A0|=C2=A0Facebook

On Mon, May 22, 2017 at 5:23 PM, Shane Johns= on <shanewaldenjohnson@gmail.com> wrote:
Team,

<= div>I am trying to get IntelliJ working with predictionIO. I have been foll= owing this turorial and am getting close to getting the PIO Train to work b= ut it looks like it is failing as it goes to persist the model. This in tur= n seems to be causing the `pio deploy` in intelliJ to fail as well.

What I really am trying to do is get into an environment = where I can test the code and view the dataframes as they are being created= . I am new to PredictionIO and Scala. I am trying to find a way to view the= contents of=C2=A0class Train= ingData(
  
val rat=
ings: RDD[Rating]
) extends Serializable {
  override def toString =3D {
    s"=
;ratings: [${ratings.count()}] (${ratings.take(2).toList}...)"
  }
}

[INFO] [Engine] Extracting datasource params...
[INFO] [WorkflowUtils$] No 'name' is fou= nd. Default empty String will be used.
[INFO] [Engine] Datasource params: (,DataSourceParams(MyApp1,None= ))
[INFO] [Engine] Extracting prep= arator params...
[INFO] [Engine] P= reparator params: (,Empty)
[INFO] = [Engine] Extracting serving params...
[INFO] [Engine] Serving params: (,Empty)
[INFO] [Remoting] Starting remoting
[INFO] [Remoting] Remoting started; listening on addresses :[a= kka.tcp://sparkDriverActorSystem@192.168.1.21:57099]
[WARN] [AbstractLifeCycle] FAILED SelectCha= nnelConnector@0.0.0.0:4040: java.net.BindException: Address alread= y in use
[WARN] [AbstractLifeCycle= ] FAILED org.spark-project.jetty.server.Server@53c1179a: java.net.Bind= Exception: Address already in use
= [WARN] [Utils] Service 'SparkUI' could not bind on port 4040. Attem= pting port 4041.
[INFO] [Engine$] = EngineWorkflow.train
[INFO] [Engin= e$] DataSource: org.example.recommendation.DataSource@19bedeb8
[INFO] [Engine$] Preparator: org.example.= recommendation.Preparator@7ef8eda7
[INFO] [Engine$] AlgorithmList: List(org.example.recommendation.= ALSAlgorithm@7f287b98)
[INFO]= [Engine$] Data sanity check is on.
[INFO] [Engine$] org.example.recommendation.TrainingData does not su= pport data sanity check. Skipping check.
[INFO] [Engine$] org.example.recommendation.PreparedData does n= ot support data sanity check. Skipping check.
[WARN] [BLAS] Failed to load implementation from: com.github.fo= mmil.netlib.NativeSystemBLAS
= [WARN] [BLAS] Failed to load implementation from: com.github.fommil.netlib.= NativeRefBLAS
[WARN] [LAPACK]= Failed to load implementation from: com.github.fommil.netlib.NativeSy= stemLAPACK
[WARN] [LAPACK] Failed = to load implementation from: com.github.fommil.netlib.NativeRefLAPACK<= /div>
[INFO] [Engine$] org.apache.spark.= mllib.recommendation.ALSModel does not support data sanity check. Skip= ping check.
[INFO] [Engine$] Engin= eWorkflow.train completed
[INFO] [= Engine] engineInstanceId=3D6fb987b7-7674-4b95-a938-a1190e215638
<= div class=3D"m_-1872143685776559972gmail-hljs" style=3D"color:rgb(51,51,51)= ;overflow-x:auto;padding:0.5em;background-image:initial;background-position= :initial;background-size:initial;background-repeat:initial;background-origi= n:initial;background-clip:initial">[INFO] [CoreWorkflow$] Inserting persist= ent model
Exception in thread = "main" java.lang.NoSuchMethodError: com.esotericsoftware.kryo.Kryo.setInstantiatorStrategy(Lorg/objenesis/strategy/Instantiat= orStrategy;)V
at com.twitter.chill.KryoBase.setInstantiatorStrategy(KryoBase.scala:86)
at com.twitter.chill.EmptyScalaKryoInstantiator.newKryo(= ScalaKryoInstantiator.scala:59)
at com.twitter.chill.ScalaKryoInstantiator.n= ewKryo(ScalaKryoInstantiator.scala:94)
at org.apache.predictionio.workf= low.KryoInstantiator.newKryo(CreateServer.scala:60)
at org.apache.predictionio.workflow.KryoInstantiator.newKryo(CreateServer.scala:58)
at com.twitter.chill.= KryoPool$2.newInstance(KryoPool.java:59)
at com.twitter.chill.KryoPool$2.new= Instance(KryoPool.java:57)
at com.twitter.chill.ResourcePool.borrow(Resource= Pool.java:37)
at com.twitter.chill.KryoPool.toBytesWithClass(KryoPool.java:1= 14)
at com.= twitter.chill.KryoInjectionInstance.apply(KryoInjection.scala:64)=
at com.twi= tter.chill.KryoInjectionInstance.apply(KryoInjection.scala:55)
at org.apache= .predictionio.workflow.CoreWorkflow$.runTrain(CoreWorkflow.scala:= 81)
at= org.apache.predictionio.workflow.CreateWorkflow$.main(CreateWork= flow.scala:250)
<= /span>at org.apache.predictionio.workflow.CreateWorkflow.main(Cre= ateWorkflow.scala)
= at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Me= thod)
at su= n.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccess= orImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(= Method.java:498)
<= /span>at com.intellij.rt.execution.application.AppMain.main(AppMa= in.java:147)

Shane Johnson | 801.360.3350=

LinkedIn=C2=A0|= =C2=A0Facebook
=

--94eb2c191536c9c60d055025440e--