predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Darshan A.N." <darshanan...@gmail.com>
Subject Re: Pio build success with error, pio train is faililng.
Date Thu, 27 Jul 2017 08:59:20 GMT
hi Tom Chan,
i did tried with directory u told me to do.

but got an error, i know where did that error emerging from, but as for the
document i myself edited the appName to appId,

now the error is like this,




darshan@darshu:~/PredictionIO/tapster-episode-similar$ pio train
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/home/darshan/PredictionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/darshan/PredictionIO/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
[INFO] [Runner$] Submission command:
/home/darshan/PredictionIO/vendors//spark-1.6.3/bin/spark-submit --class
org.apache.predictionio.workflow.CreateWorkflow --jars
file:/home/darshan/PredictionIO/lib/mysql-connector-java-5.1.37.jar,file:/home/darshan/PredictionIO/tapster-episode-similar/target/scala-2.10/template-scala-parallel-similarproduct_2.10-0.1-SNAPSHOT.jar,file:/home/darshan/PredictionIO/tapster-episode-similar/target/scala-2.10/template-scala-parallel-similarproduct-assembly-0.1-SNAPSHOT-deps.jar,file:/home/darshan/PredictionIO/lib/spark/pio-data-hbase-assembly-0.11.0-incubating.jar,file:/home/darshan/PredictionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar,file:/home/darshan/PredictionIO/lib/spark/pio-data-jdbc-assembly-0.11.0-incubating.jar,file:/home/darshan/PredictionIO/lib/spark/pio-data-elasticsearch1-assembly-0.11.0-incubating.jar,file:/home/darshan/PredictionIO/lib/spark/pio-data-localfs-assembly-0.11.0-incubating.jar
--files file:/home/darshan/PredictionIO/conf/log4j.properties
--driver-class-path
/home/darshan/PredictionIO/conf:/home/darshan/PredictionIO/lib/mysql-connector-java-5.1.37.jar
--driver-java-options -Dpio.log.dir=/home/darshan
file:/home/darshan/PredictionIO/lib/pio-assembly-0.11.0-incubating.jar
--engine-id org.example.similarproduct.SimilarProductEngine
--engine-version 7ff66e8607f7d36b79cb9e9f3b97b53287e553f7 --engine-variant
file:/home/darshan/PredictionIO/tapster-episode-similar/engine.json
--verbosity 0 --json-extractor Both --env
PIO_ENV_LOADED=1,PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio,PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta,PIO_FS_BASEDIR=/home/darshan/.pio_store,PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost:3306/pio,PIO_HOME=/home/darshan/PredictionIO,PIO_FS_ENGINESDIR=/home/darshan/.pio_store/engines,PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc,PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=MYSQL,PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=MYSQL,PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio,PIO_FS_TMPDIR=/home/darshan/.pio_store/tmp,PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model,PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=MYSQL,PIO_CONF_DIR=/home/darshan/PredictionIO/conf
[INFO] [Engine] Extracting datasource params...
[INFO] [WorkflowUtils$] No 'name' is found. Default empty String will be
used.
[INFO] [Engine] Datasource params: (,DataSourceParams(null))
[INFO] [Engine] Extracting preparator params...
[INFO] [Engine] Preparator params: (,Empty)
[INFO] [Engine] Extracting serving params...
[INFO] [Engine] Serving params: (,Empty)
[WARN] [Utils] Your hostname, darshu resolves to a loopback address:
127.0.0.1; using 192.168.2.103 instead (on interface wlx001ea6631cc7)
[WARN] [Utils] Set SPARK_LOCAL_IP if you need to bind to another address
[INFO] [Remoting] Starting remoting
[INFO] [Remoting] Remoting started; listening on addresses :[akka.tcp://
sparkDriverActorSystem@192.168.2.103:45309]
[INFO] [Engine$] EngineWorkflow.train
[INFO] [Engine$] DataSource: org.example.similarproduct.DataSource@168b4cb0
[INFO] [Engine$] Preparator: org.example.similarproduct.Preparator@45545e7a
[INFO] [Engine$] AlgorithmList:
List(org.example.similarproduct.ALSAlgorithm@23cbbd07)
[INFO] [Engine$] Data sanity check is on.
[ERROR] [Common$] Invalid app name null
Exception in thread "main" java.lang.IllegalArgumentException: Invalid app
name null
    at
org.apache.predictionio.data.store.Common$$anonfun$appNameToId$2.apply(Common.scala:50)
    at
org.apache.predictionio.data.store.Common$$anonfun$appNameToId$2.apply(Common.scala:48)
    at scala.Option.getOrElse(Option.scala:120)
    at
org.apache.predictionio.data.store.Common$.appNameToId(Common.scala:48)
    at
org.apache.predictionio.data.store.PEventStore$.aggregateProperties(PEventStore.scala:108)
    at
org.example.similarproduct.DataSource.readTraining(DataSource.scala:31)
    at
org.example.similarproduct.DataSource.readTraining(DataSource.scala:18)
    at
org.apache.predictionio.controller.PDataSource.readTrainingBase(PDataSource.scala:40)
    at org.apache.predictionio.controller.Engine$.train(Engine.scala:644)
    at org.apache.predictionio.controller.Engine.train(Engine.scala:177)
    at
org.apache.predictionio.workflow.CoreWorkflow$.runTrain(CoreWorkflow.scala:67)
    at
org.apache.predictionio.workflow.CreateWorkflow$.main(CreateWorkflow.scala:250)
    at
org.apache.predictionio.workflow.CreateWorkflow.main(CreateWorkflow.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
    at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
    at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

i m using MySQL as my repository. i followed the article from official
website only, still i m not able to execute the pio build or pio train. My
src/main/scala DataSource.scala program is still as it is in previously
mentioned.

Thanks,
-DAN

On Wed, Jul 26, 2017 at 3:22 PM, Tom Chan <yukhei.chan@gmail.com> wrote:

> darshan@darshu:~/PredictionIO/tapster-episode-similar/src/main/scala$ pio
> train
>
> Can you try the command from the tapster-episode-similar directory?
>
> Tom
>
> On Jul 26, 2017 2:45 AM, "Darshan A.N." <darshanan.24@gmail.com> wrote:
>
>> hi team,
>> i am trying to install demo tapster. i  followed
>> http://predictionio.incubator.apache.org/demo/tapster/.     It took more
>> than a week to install predictionio.
>> now that i m installed the PIO, its throwing me error while running the
>> $pio build command. the error goes like this:
>> darshan@darshu:~/PredictionIO/tapster-episode-similar$ pio build
>> SLF4J: Class path contains multiple SLF4J bindings.
>> SLF4J: Found binding in [jar:file:/home/darshan/Predic
>> tionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.ja
>> r!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in [jar:file:/home/darshan/Predic
>> tionIO/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/imp
>> l/StaticLoggerBinder.class]
>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>> explanation.
>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> [INFO] [Engine$] Using command '/home/darshan/PredictionIO/sbt/sbt' at
>> /home/darshan/PredictionIO/tapster-episode-similar to build.
>> [INFO] [Engine$] If the path above is incorrect, this process will fail.
>> [INFO] [Engine$] Uber JAR disabled. Making sure
>> lib/pio-assembly-0.11.0-incubating.jar is absent.
>> [INFO] [Engine$] Going to run: /home/darshan/PredictionIO/sbt/sbt
>> package assemblyPackageDependency in /home/darshan/PredictionIO/tap
>> ster-episode-similar
>> [ERROR] [Engine$] [error] /home/darshan/PredictionIO/tap
>> ster-episode-similar/src/main/scala/DataSource.scala:63: not found:
>> value eventsDb
>> [ERROR] [Engine$] [error]     val viewEventsRDD: RDD[ViewEvent] =
>> eventsDb.find(
>> [ERROR] [Engine$] [error]                                         ^
>> [ERROR] [Engine$] [error] one error found
>> [ERROR] [Engine$] [error] (compile:compileIncremental) Compilation failed
>> [ERROR] [Engine$] [error] Total time: 5 s, completed 26 Jul, 2017 1:56:50
>> AM
>> [ERROR] [Engine$] Return code of build command:
>> /home/darshan/PredictionIO/sbt/sbt  package assemblyPackageDependency is
>> 1. Aborting.
>> [INFO] [Engine$] Looking for an engine...
>> [INFO] [Engine$] Found template-scala-parallel-simila
>> rproduct_2.10-0.1-SNAPSHOT.jar
>> [INFO] [Engine$] Found template-scala-parallel-simila
>> rproduct-assembly-0.1-SNAPSHOT-deps.jar
>> [INFO] [Engine$] Build finished successfully.
>> [INFO] [Pio$] Your engine is ready for training.
>>
>>
>> and the file DataSource.scala,
>>
>>
>> package org.example.similarproduct
>>
>> import org.apache.predictionio.controller.PDataSource
>> import org.apache.predictionio.controller.EmptyEvaluationInfo
>> import org.apache.predictionio.controller.EmptyActualResult
>> import org.apache.predictionio.controller.Params
>> import org.apache.predictionio.data.storage.Event
>> import org.apache.predictionio.data.store.PEventStore
>>
>> import org.apache.spark.SparkContext
>> import org.apache.spark.SparkContext._
>> import org.apache.spark.rdd.RDD
>>
>> import grizzled.slf4j.Logger
>>
>> case class DataSourceParams(appName: String) extends Params
>>
>> class DataSource(val dsp: DataSourceParams)
>>   extends PDataSource[TrainingData,
>>       EmptyEvaluationInfo, Query, EmptyActualResult] {
>>
>>   @transient lazy val logger = Logger[this.type]
>>
>>   override
>>   def readTraining(sc: SparkContext): TrainingData = {
>>
>>     // create a RDD of (entityID, User)
>>     val usersRDD: RDD[(String, User)] = PEventStore.aggregateProperties(
>>       appName = dsp.appName,
>>       entityType = "user"
>>     )(sc).map { case (entityId, properties) =>
>>       val user = try {
>>         User()
>>       } catch {
>>         case e: Exception => {
>>           logger.error(s"Failed to get properties ${properties} of" +
>>             s" user ${entityId}. Exception: ${e}.")
>>           throw e
>>         }
>>       }
>>       (entityId, user)
>>     }.cache()
>>
>>     // create a RDD of (entityID, Item)
>>     val itemsRDD: RDD[(String, Item)] = PEventStore.aggregateProperties(
>>       appName = dsp.appName,
>>       entityType = "item"
>>     )(sc).map { case (entityId, properties) =>
>>       val item = try {
>>         // Assume categories is optional property of item.
>>         Item(categories = properties.getOpt[List[String]]("categories"))
>>       } catch {
>>         case e: Exception => {
>>           logger.error(s"Failed to get properties ${properties} of" +
>>             s" item ${entityId}. Exception: ${e}.")
>>           throw e
>>         }
>>       }
>>       (entityId, item)
>>     }.cache()
>>
>>     // get all "user" "view" "item" events
>>     val viewEventsRDD: RDD[ViewEvent] = eventsDb.find(
>>       appId = dsp.appId,
>>       entityType = Some("user"),
>>       eventNames = Some(List("like")),
>>       // targetEntityType is optional field of an event.
>>       targetEntityType = Some(Some("item")))(sc)
>>       // eventsDb.find() returns RDD[Event]
>>       .map { event =>
>>         val viewEvent = try {
>>           event.event match {
>>             case "like" => ViewEvent(
>>               user = event.entityId,
>>               item = event.targetEntityId.get,
>>               t = event.eventTime.getMillis)
>>             case _ => throw new Exception(s"Unexpected event ${event} is
>> read.")
>>           }
>>         } catch {
>>           case e: Exception => {
>>             logger.error(s"Cannot convert ${event} to ViewEvent." +
>>               s" Exception: ${e}.")
>>             throw e
>>           }
>>         }
>>         viewEvent
>>       }.cache()
>>
>>     new TrainingData(
>>       users = usersRDD,
>>       items = itemsRDD,
>>       viewEvents = viewEventsRDD
>>     )
>>   }
>> }
>>
>> case class User()
>>
>> case class Item(categories: Option[List[String]])
>>
>> case class ViewEvent(user: String, item: String, t: Long)
>>
>> class TrainingData(
>>   val users: RDD[(String, User)],
>>   val items: RDD[(String, Item)],
>>   val viewEvents: RDD[ViewEvent]
>> ) extends Serializable {
>>   override def toString = {
>>     s"users: [${users.count()} (${users.take(2).toList}...)]" +
>>     s"items: [${items.count()} (${items.take(2).toList}...)]" +
>>     s"viewEvents: [${viewEvents.count()}] (${viewEvents.take(2).toList}.
>> ..)"
>>   }
>> }
>>
>>
>>
>>
>> while training, it gives  me following error,
>>
>>
>>
>>
>> darshan@darshu:~/PredictionIO/tapster-episode-similar/src/main/scala$
>> pio train
>> SLF4J: Class path contains multiple SLF4J bindings.
>> SLF4J: Found binding in [jar:file:/home/darshan/Predic
>> tionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.ja
>> r!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in [jar:file:/home/darshan/Predic
>> tionIO/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/imp
>> l/StaticLoggerBinder.class]
>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>> explanation.
>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> [WARN] [Template$] /home/darshan/PredictionIO/tap
>> ster-episode-similar/src/main/scala/template.json does not exist.
>> Template metadata will not be available. (This is safe to ignore if you are
>> not working on a template.)
>> Exception in thread "main" java.io.FileNotFoundException:
>> /home/darshan/PredictionIO/tapster-episode-similar/src/main/scala/engine.json
>> (No such file or directory)
>>     at java.io.FileInputStream.open0(Native Method)
>>     at java.io.FileInputStream.open(FileInputStream.java:195)
>>     at java.io.FileInputStream.<init>(FileInputStream.java:138)
>>     at scala.io.Source$.fromFile(Source.scala:90)
>>     at scala.io.Source$.fromFile(Source.scala:75)
>>     at org.apache.predictionio.tools.console.Console$.getEngineInfo
>> (Console.scala:724)
>>     at org.apache.predictionio.tools.RunWorkflow$.runWorkflow(RunWo
>> rkflow.scala:54)
>>     at org.apache.predictionio.tools.commands.Engine$.train(Engine.
>> scala:186)
>>     at org.apache.predictionio.tools.console.Pio$.train(Pio.scala:85)
>>     at org.apache.predictionio.tools.console.Console$$anonfun$main$
>> 1.apply(Console.scala:626)
>>     at org.apache.predictionio.tools.console.Console$$anonfun$main$
>> 1.apply(Console.scala:611)
>>     at scala.Option.map(Option.scala:145)
>>     at org.apache.predictionio.tools.console.Console$.main(Console.
>> scala:611)
>>     at org.apache.predictionio.tools.console.Console.main(Console.scala)
>>
>>
>>
>> i tried almost all the things, but could not find the proper solution.
>> please help me...
>> i know you may feel bit odd about this mail, but in need of your help.
>>
>> thanks,
>>
>>
>> -DAN
>>
>

Mime
View raw message