predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vaghawan Ojha <vaghawan...@gmail.com>
Subject Re: Pio build success with error, pio train is faililng.
Date Thu, 27 Jul 2017 09:01:41 GMT
As per the error "[ERROR] [Common$] Invalid app name null" you haven't
configured the engine.jso properly it seems.

On Thu, Jul 27, 2017 at 2:44 PM, Darshan A.N. <darshanan.24@gmail.com>
wrote:

> hi Tom Chan,
> i did tried with directory u told me to do.
>
> but got an error, i know where did that error emerging from, but as for
> the document i myself edited the appName to appId,
>
> now the error is like this,
>
>
>
>
> darshan@darshu:~/PredictionIO/tapster-episode-similar$ pio train
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/home/darshan/
> PredictionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-
> incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/home/darshan/
> PredictionIO/lib/pio-assembly-0.11.0-incubating.jar!/org/
> slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> [INFO] [Runner$] Submission command: /home/darshan/PredictionIO/
> vendors//spark-1.6.3/bin/spark-submit --class org.apache.predictionio.workflow.CreateWorkflow
> --jars file:/home/darshan/PredictionIO/lib/mysql-
> connector-java-5.1.37.jar,file:/home/darshan/PredictionIO/tapster-episode-
> similar/target/scala-2.10/template-scala-parallel-similarproduct_2.10-0.1-
> SNAPSHOT.jar,file:/home/darshan/PredictionIO/tapster-
> episode-similar/target/scala-2.10/template-scala-parallel-
> similarproduct-assembly-0.1-SNAPSHOT-deps.jar,file:/home/
> darshan/PredictionIO/lib/spark/pio-data-hbase-assembly-
> 0.11.0-incubating.jar,file:/home/darshan/PredictionIO/lib/
> spark/pio-data-hdfs-assembly-0.11.0-incubating.jar,file:/
> home/darshan/PredictionIO/lib/spark/pio-data-jdbc-assembly-
> 0.11.0-incubating.jar,file:/home/darshan/PredictionIO/lib/
> spark/pio-data-elasticsearch1-assembly-0.11.0-incubating.
> jar,file:/home/darshan/PredictionIO/lib/spark/pio-
> data-localfs-assembly-0.11.0-incubating.jar --files file:/home/darshan/
> PredictionIO/conf/log4j.properties --driver-class-path
> /home/darshan/PredictionIO/conf:/home/darshan/PredictionIO/lib/mysql-connector-java-5.1.37.jar
> --driver-java-options -Dpio.log.dir=/home/darshan file:/home/darshan/
> PredictionIO/lib/pio-assembly-0.11.0-incubating.jar --engine-id
> org.example.similarproduct.SimilarProductEngine --engine-version
> 7ff66e8607f7d36b79cb9e9f3b97b53287e553f7 --engine-variant
> file:/home/darshan/PredictionIO/tapster-episode-similar/engine.json
> --verbosity 0 --json-extractor Both --env PIO_ENV_LOADED=1,PIO_STORAGE_
> SOURCES_MYSQL_PASSWORD=pio,PIO_STORAGE_REPOSITORIES_
> METADATA_NAME=pio_meta,PIO_FS_BASEDIR=/home/darshan/.pio_
> store,PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://
> localhost:3306/pio,PIO_HOME=/home/darshan/PredictionIO,PIO_
> FS_ENGINESDIR=/home/darshan/.pio_store/engines,PIO_STORAGE_
> SOURCES_MYSQL_TYPE=jdbc,PIO_STORAGE_REPOSITORIES_METADATA_
> SOURCE=MYSQL,PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=MYSQL,PIO_STORAGE_
> REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_
> MYSQL_USERNAME=pio,PIO_FS_TMPDIR=/home/darshan/.pio_store/tmp,PIO_STORAGE_
> REPOSITORIES_MODELDATA_NAME=pio_model,PIO_STORAGE_
> REPOSITORIES_EVENTDATA_SOURCE=MYSQL,PIO_CONF_DIR=/home/
> darshan/PredictionIO/conf
> [INFO] [Engine] Extracting datasource params...
> [INFO] [WorkflowUtils$] No 'name' is found. Default empty String will be
> used.
> [INFO] [Engine] Datasource params: (,DataSourceParams(null))
> [INFO] [Engine] Extracting preparator params...
> [INFO] [Engine] Preparator params: (,Empty)
> [INFO] [Engine] Extracting serving params...
> [INFO] [Engine] Serving params: (,Empty)
> [WARN] [Utils] Your hostname, darshu resolves to a loopback address:
> 127.0.0.1; using 192.168.2.103 instead (on interface wlx001ea6631cc7)
> [WARN] [Utils] Set SPARK_LOCAL_IP if you need to bind to another address
> [INFO] [Remoting] Starting remoting
> [INFO] [Remoting] Remoting started; listening on addresses :[akka.tcp://
> sparkDriverActorSystem@192.168.2.103:45309]
> [INFO] [Engine$] EngineWorkflow.train
> [INFO] [Engine$] DataSource: org.example.similarproduct.
> DataSource@168b4cb0
> [INFO] [Engine$] Preparator: org.example.similarproduct.
> Preparator@45545e7a
> [INFO] [Engine$] AlgorithmList: List(org.example.
> similarproduct.ALSAlgorithm@23cbbd07)
> [INFO] [Engine$] Data sanity check is on.
> [ERROR] [Common$] Invalid app name null
> Exception in thread "main" java.lang.IllegalArgumentException: Invalid
> app name null
>     at org.apache.predictionio.data.store.Common$$anonfun$
> appNameToId$2.apply(Common.scala:50)
>     at org.apache.predictionio.data.store.Common$$anonfun$
> appNameToId$2.apply(Common.scala:48)
>     at scala.Option.getOrElse(Option.scala:120)
>     at org.apache.predictionio.data.store.Common$.appNameToId(
> Common.scala:48)
>     at org.apache.predictionio.data.store.PEventStore$.
> aggregateProperties(PEventStore.scala:108)
>     at org.example.similarproduct.DataSource.readTraining(
> DataSource.scala:31)
>     at org.example.similarproduct.DataSource.readTraining(
> DataSource.scala:18)
>     at org.apache.predictionio.controller.PDataSource.
> readTrainingBase(PDataSource.scala:40)
>     at org.apache.predictionio.controller.Engine$.train(Engine.scala:644)
>     at org.apache.predictionio.controller.Engine.train(Engine.scala:177)
>     at org.apache.predictionio.workflow.CoreWorkflow$.
> runTrain(CoreWorkflow.scala:67)
>     at org.apache.predictionio.workflow.CreateWorkflow$.main(
> CreateWorkflow.scala:250)
>     at org.apache.predictionio.workflow.CreateWorkflow.main(
> CreateWorkflow.scala)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:498)
>     at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$
> deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
>     at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(
> SparkSubmit.scala:181)
>     at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
>     at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
>     at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>
> i m using MySQL as my repository. i followed the article from official
> website only, still i m not able to execute the pio build or pio train. My
> src/main/scala DataSource.scala program is still as it is in previously
> mentioned.
>
> Thanks,
> -DAN
>
> On Wed, Jul 26, 2017 at 3:22 PM, Tom Chan <yukhei.chan@gmail.com> wrote:
>
>> darshan@darshu:~/PredictionIO/tapster-episode-similar/src/main/scala$
>> pio train
>>
>> Can you try the command from the tapster-episode-similar directory?
>>
>> Tom
>>
>> On Jul 26, 2017 2:45 AM, "Darshan A.N." <darshanan.24@gmail.com> wrote:
>>
>>> hi team,
>>> i am trying to install demo tapster. i  followed
>>> http://predictionio.incubator.apache.org/demo/tapster/.     It took
>>> more than a week to install predictionio.
>>> now that i m installed the PIO, its throwing me error while running the
>>> $pio build command. the error goes like this:
>>> darshan@darshu:~/PredictionIO/tapster-episode-similar$ pio build
>>> SLF4J: Class path contains multiple SLF4J bindings.
>>> SLF4J: Found binding in [jar:file:/home/darshan/Predic
>>> tionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.ja
>>> r!/org/slf4j/impl/StaticLoggerBinder.class]
>>> SLF4J: Found binding in [jar:file:/home/darshan/Predic
>>> tionIO/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/imp
>>> l/StaticLoggerBinder.class]
>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>>> explanation.
>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>>> [INFO] [Engine$] Using command '/home/darshan/PredictionIO/sbt/sbt' at
>>> /home/darshan/PredictionIO/tapster-episode-similar to build.
>>> [INFO] [Engine$] If the path above is incorrect, this process will fail.
>>> [INFO] [Engine$] Uber JAR disabled. Making sure
>>> lib/pio-assembly-0.11.0-incubating.jar is absent.
>>> [INFO] [Engine$] Going to run: /home/darshan/PredictionIO/sbt/sbt
>>> package assemblyPackageDependency in /home/darshan/PredictionIO/tap
>>> ster-episode-similar
>>> [ERROR] [Engine$] [error] /home/darshan/PredictionIO/tap
>>> ster-episode-similar/src/main/scala/DataSource.scala:63: not found:
>>> value eventsDb
>>> [ERROR] [Engine$] [error]     val viewEventsRDD: RDD[ViewEvent] =
>>> eventsDb.find(
>>> [ERROR] [Engine$] [error]                                         ^
>>> [ERROR] [Engine$] [error] one error found
>>> [ERROR] [Engine$] [error] (compile:compileIncremental) Compilation failed
>>> [ERROR] [Engine$] [error] Total time: 5 s, completed 26 Jul, 2017
>>> 1:56:50 AM
>>> [ERROR] [Engine$] Return code of build command:
>>> /home/darshan/PredictionIO/sbt/sbt  package assemblyPackageDependency
>>> is 1. Aborting.
>>> [INFO] [Engine$] Looking for an engine...
>>> [INFO] [Engine$] Found template-scala-parallel-simila
>>> rproduct_2.10-0.1-SNAPSHOT.jar
>>> [INFO] [Engine$] Found template-scala-parallel-simila
>>> rproduct-assembly-0.1-SNAPSHOT-deps.jar
>>> [INFO] [Engine$] Build finished successfully.
>>> [INFO] [Pio$] Your engine is ready for training.
>>>
>>>
>>> and the file DataSource.scala,
>>>
>>>
>>> package org.example.similarproduct
>>>
>>> import org.apache.predictionio.controller.PDataSource
>>> import org.apache.predictionio.controller.EmptyEvaluationInfo
>>> import org.apache.predictionio.controller.EmptyActualResult
>>> import org.apache.predictionio.controller.Params
>>> import org.apache.predictionio.data.storage.Event
>>> import org.apache.predictionio.data.store.PEventStore
>>>
>>> import org.apache.spark.SparkContext
>>> import org.apache.spark.SparkContext._
>>> import org.apache.spark.rdd.RDD
>>>
>>> import grizzled.slf4j.Logger
>>>
>>> case class DataSourceParams(appName: String) extends Params
>>>
>>> class DataSource(val dsp: DataSourceParams)
>>>   extends PDataSource[TrainingData,
>>>       EmptyEvaluationInfo, Query, EmptyActualResult] {
>>>
>>>   @transient lazy val logger = Logger[this.type]
>>>
>>>   override
>>>   def readTraining(sc: SparkContext): TrainingData = {
>>>
>>>     // create a RDD of (entityID, User)
>>>     val usersRDD: RDD[(String, User)] = PEventStore.aggregateProperties(
>>>       appName = dsp.appName,
>>>       entityType = "user"
>>>     )(sc).map { case (entityId, properties) =>
>>>       val user = try {
>>>         User()
>>>       } catch {
>>>         case e: Exception => {
>>>           logger.error(s"Failed to get properties ${properties} of" +
>>>             s" user ${entityId}. Exception: ${e}.")
>>>           throw e
>>>         }
>>>       }
>>>       (entityId, user)
>>>     }.cache()
>>>
>>>     // create a RDD of (entityID, Item)
>>>     val itemsRDD: RDD[(String, Item)] = PEventStore.aggregateProperties(
>>>       appName = dsp.appName,
>>>       entityType = "item"
>>>     )(sc).map { case (entityId, properties) =>
>>>       val item = try {
>>>         // Assume categories is optional property of item.
>>>         Item(categories = properties.getOpt[List[String]]("categories"))
>>>       } catch {
>>>         case e: Exception => {
>>>           logger.error(s"Failed to get properties ${properties} of" +
>>>             s" item ${entityId}. Exception: ${e}.")
>>>           throw e
>>>         }
>>>       }
>>>       (entityId, item)
>>>     }.cache()
>>>
>>>     // get all "user" "view" "item" events
>>>     val viewEventsRDD: RDD[ViewEvent] = eventsDb.find(
>>>       appId = dsp.appId,
>>>       entityType = Some("user"),
>>>       eventNames = Some(List("like")),
>>>       // targetEntityType is optional field of an event.
>>>       targetEntityType = Some(Some("item")))(sc)
>>>       // eventsDb.find() returns RDD[Event]
>>>       .map { event =>
>>>         val viewEvent = try {
>>>           event.event match {
>>>             case "like" => ViewEvent(
>>>               user = event.entityId,
>>>               item = event.targetEntityId.get,
>>>               t = event.eventTime.getMillis)
>>>             case _ => throw new Exception(s"Unexpected event ${event} is
>>> read.")
>>>           }
>>>         } catch {
>>>           case e: Exception => {
>>>             logger.error(s"Cannot convert ${event} to ViewEvent." +
>>>               s" Exception: ${e}.")
>>>             throw e
>>>           }
>>>         }
>>>         viewEvent
>>>       }.cache()
>>>
>>>     new TrainingData(
>>>       users = usersRDD,
>>>       items = itemsRDD,
>>>       viewEvents = viewEventsRDD
>>>     )
>>>   }
>>> }
>>>
>>> case class User()
>>>
>>> case class Item(categories: Option[List[String]])
>>>
>>> case class ViewEvent(user: String, item: String, t: Long)
>>>
>>> class TrainingData(
>>>   val users: RDD[(String, User)],
>>>   val items: RDD[(String, Item)],
>>>   val viewEvents: RDD[ViewEvent]
>>> ) extends Serializable {
>>>   override def toString = {
>>>     s"users: [${users.count()} (${users.take(2).toList}...)]" +
>>>     s"items: [${items.count()} (${items.take(2).toList}...)]" +
>>>     s"viewEvents: [${viewEvents.count()}] (${viewEvents.take(2).toList}.
>>> ..)"
>>>   }
>>> }
>>>
>>>
>>>
>>>
>>> while training, it gives  me following error,
>>>
>>>
>>>
>>>
>>> darshan@darshu:~/PredictionIO/tapster-episode-similar/src/main/scala$
>>> pio train
>>> SLF4J: Class path contains multiple SLF4J bindings.
>>> SLF4J: Found binding in [jar:file:/home/darshan/Predic
>>> tionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.ja
>>> r!/org/slf4j/impl/StaticLoggerBinder.class]
>>> SLF4J: Found binding in [jar:file:/home/darshan/Predic
>>> tionIO/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/imp
>>> l/StaticLoggerBinder.class]
>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>>> explanation.
>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>>> [WARN] [Template$] /home/darshan/PredictionIO/tap
>>> ster-episode-similar/src/main/scala/template.json does not exist.
>>> Template metadata will not be available. (This is safe to ignore if you are
>>> not working on a template.)
>>> Exception in thread "main" java.io.FileNotFoundException:
>>> /home/darshan/PredictionIO/tapster-episode-similar/src/main/scala/engine.json
>>> (No such file or directory)
>>>     at java.io.FileInputStream.open0(Native Method)
>>>     at java.io.FileInputStream.open(FileInputStream.java:195)
>>>     at java.io.FileInputStream.<init>(FileInputStream.java:138)
>>>     at scala.io.Source$.fromFile(Source.scala:90)
>>>     at scala.io.Source$.fromFile(Source.scala:75)
>>>     at org.apache.predictionio.tools.console.Console$.getEngineInfo
>>> (Console.scala:724)
>>>     at org.apache.predictionio.tools.RunWorkflow$.runWorkflow(RunWo
>>> rkflow.scala:54)
>>>     at org.apache.predictionio.tools.commands.Engine$.train(Engine.
>>> scala:186)
>>>     at org.apache.predictionio.tools.console.Pio$.train(Pio.scala:85)
>>>     at org.apache.predictionio.tools.console.Console$$anonfun$main$
>>> 1.apply(Console.scala:626)
>>>     at org.apache.predictionio.tools.console.Console$$anonfun$main$
>>> 1.apply(Console.scala:611)
>>>     at scala.Option.map(Option.scala:145)
>>>     at org.apache.predictionio.tools.console.Console$.main(Console.
>>> scala:611)
>>>     at org.apache.predictionio.tools.console.Console.main(Console.scala)
>>>
>>>
>>>
>>> i tried almost all the things, but could not find the proper solution.
>>> please help me...
>>> i know you may feel bit odd about this mail, but in need of your help.
>>>
>>> thanks,
>>>
>>>
>>> -DAN
>>>
>>
>

Mime
View raw message