predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Darshan A.N." <darshanan...@gmail.com>
Subject Re: Pio build success with error, pio train is faililng.
Date Thu, 27 Jul 2017 09:09:53 GMT
VO,
i was just replying to ur response, here u are... with quick response on my
post. thanks to that.
as far as error, i know that where did i commit mistake, but i did changed
the appName as it mentioned in official article, so could u please check
DataSource.scala file of mine which i hv mentioned in my first mail.

Thanks,
-DAN

On Thu, Jul 27, 2017 at 2:31 PM, Vaghawan Ojha <vaghawan781@gmail.com>
wrote:

> As per the error "[ERROR] [Common$] Invalid app name null" you haven't
> configured the engine.jso properly it seems.
>
> On Thu, Jul 27, 2017 at 2:44 PM, Darshan A.N. <darshanan.24@gmail.com>
> wrote:
>
>> hi Tom Chan,
>> i did tried with directory u told me to do.
>>
>> but got an error, i know where did that error emerging from, but as for
>> the document i myself edited the appName to appId,
>>
>> now the error is like this,
>>
>>
>>
>>
>> darshan@darshu:~/PredictionIO/tapster-episode-similar$ pio train
>> SLF4J: Class path contains multiple SLF4J bindings.
>> SLF4J: Found binding in [jar:file:/home/darshan/Predic
>> tionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-incubatin
>> g.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in [jar:file:/home/darshan/Predic
>> tionIO/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/
>> impl/StaticLoggerBinder.class]
>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>> explanation.
>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> [INFO] [Runner$] Submission command: /home/darshan/PredictionIO/ven
>> dors//spark-1.6.3/bin/spark-submit --class org.apache.predictionio.workflow.CreateWorkflow
>> --jars file:/home/darshan/PredictionIO/lib/mysql-connector-java-5.
>> 1.37.jar,file:/home/darshan/PredictionIO/tapster-episode-sim
>> ilar/target/scala-2.10/template-scala-parallel-similarproduc
>> t_2.10-0.1-SNAPSHOT.jar,file:/home/darshan/PredictionIO/
>> tapster-episode-similar/target/scala-2.10/template-
>> scala-parallel-similarproduct-assembly-0.1-SNAPSHOT-deps.
>> jar,file:/home/darshan/PredictionIO/lib/spark/pio-
>> data-hbase-assembly-0.11.0-incubating.jar,file:/home/
>> darshan/PredictionIO/lib/spark/pio-data-hdfs-assembly-0.11.
>> 0-incubating.jar,file:/home/darshan/PredictionIO/lib/spark
>> /pio-data-jdbc-assembly-0.11.0-incubating.jar,file:/home/
>> darshan/PredictionIO/lib/spark/pio-data-elasticsearch1-assem
>> bly-0.11.0-incubating.jar,file:/home/darshan/PredictionI
>> O/lib/spark/pio-data-localfs-assembly-0.11.0-incubating.jar --files
>> file:/home/darshan/PredictionIO/conf/log4j.properties
>> --driver-class-path /home/darshan/PredictionIO/con
>> f:/home/darshan/PredictionIO/lib/mysql-connector-java-5.1.37.jar
>> --driver-java-options -Dpio.log.dir=/home/darshan
>> file:/home/darshan/PredictionIO/lib/pio-assembly-0.11.0-incubating.jar
>> --engine-id org.example.similarproduct.SimilarProductEngine
>> --engine-version 7ff66e8607f7d36b79cb9e9f3b97b53287e553f7
>> --engine-variant file:/home/darshan/PredictionIO/tapster-episode-similar/engine.json
>> --verbosity 0 --json-extractor Both --env PIO_ENV_LOADED=1,PIO_STORAGE_S
>> OURCES_MYSQL_PASSWORD=pio,PIO_STORAGE_REPOSITORIES_METADATA_
>> NAME=pio_meta,PIO_FS_BASEDIR=/home/darshan/.pio_store,PIO_
>> STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost:3306/
>> pio,PIO_HOME=/home/darshan/PredictionIO,PIO_FS_
>> ENGINESDIR=/home/darshan/.pio_store/engines,PIO_STORAGE_SOUR
>> CES_MYSQL_TYPE=jdbc,PIO_STORAGE_REPOSITORIES_METADATA_SOURCE
>> =MYSQL,PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=MYSQL,PIO_
>> STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_MYSQL_
>> USERNAME=pio,PIO_FS_TMPDIR=/home/darshan/.pio_store/tmp,
>> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model,PIO_
>> STORAGE_REPOSITORIES_EVENTDATA_SOURCE=MYSQL,PIO_CONF_DIR=/home/darshan/
>> PredictionIO/conf
>> [INFO] [Engine] Extracting datasource params...
>> [INFO] [WorkflowUtils$] No 'name' is found. Default empty String will be
>> used.
>> [INFO] [Engine] Datasource params: (,DataSourceParams(null))
>> [INFO] [Engine] Extracting preparator params...
>> [INFO] [Engine] Preparator params: (,Empty)
>> [INFO] [Engine] Extracting serving params...
>> [INFO] [Engine] Serving params: (,Empty)
>> [WARN] [Utils] Your hostname, darshu resolves to a loopback address:
>> 127.0.0.1; using 192.168.2.103 instead (on interface wlx001ea6631cc7)
>> [WARN] [Utils] Set SPARK_LOCAL_IP if you need to bind to another address
>> [INFO] [Remoting] Starting remoting
>> [INFO] [Remoting] Remoting started; listening on addresses :[akka.tcp://
>> sparkDriverActorSystem@192.168.2.103:45309]
>> [INFO] [Engine$] EngineWorkflow.train
>> [INFO] [Engine$] DataSource: org.example.similarproduct.Dat
>> aSource@168b4cb0
>> [INFO] [Engine$] Preparator: org.example.similarproduct.Pre
>> parator@45545e7a
>> [INFO] [Engine$] AlgorithmList: List(org.example.similarproduc
>> t.ALSAlgorithm@23cbbd07)
>> [INFO] [Engine$] Data sanity check is on.
>> [ERROR] [Common$] Invalid app name null
>> Exception in thread "main" java.lang.IllegalArgumentException: Invalid
>> app name null
>>     at org.apache.predictionio.data.store.Common$$anonfun$appNameTo
>> Id$2.apply(Common.scala:50)
>>     at org.apache.predictionio.data.store.Common$$anonfun$appNameTo
>> Id$2.apply(Common.scala:48)
>>     at scala.Option.getOrElse(Option.scala:120)
>>     at org.apache.predictionio.data.store.Common$.appNameToId(Commo
>> n.scala:48)
>>     at org.apache.predictionio.data.store.PEventStore$.aggregatePro
>> perties(PEventStore.scala:108)
>>     at org.example.similarproduct.DataSource.readTraining(DataSourc
>> e.scala:31)
>>     at org.example.similarproduct.DataSource.readTraining(DataSourc
>> e.scala:18)
>>     at org.apache.predictionio.controller.PDataSource.readTrainingB
>> ase(PDataSource.scala:40)
>>     at org.apache.predictionio.controller.Engine$.train(Engine.scala:644)
>>     at org.apache.predictionio.controller.Engine.train(Engine.scala:177)
>>     at org.apache.predictionio.workflow.CoreWorkflow$.runTrain(
>> CoreWorkflow.scala:67)
>>     at org.apache.predictionio.workflow.CreateWorkflow$.main(Create
>> Workflow.scala:250)
>>     at org.apache.predictionio.workflow.CreateWorkflow.main(CreateW
>> orkflow.scala)
>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>> ssorImpl.java:62)
>>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> thodAccessorImpl.java:43)
>>     at java.lang.reflect.Method.invoke(Method.java:498)
>>     at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy
>> $SparkSubmit$$runMain(SparkSubmit.scala:731)
>>     at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit
>> .scala:181)
>>     at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
>>     at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
>>     at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>>
>> i m using MySQL as my repository. i followed the article from official
>> website only, still i m not able to execute the pio build or pio train. My
>> src/main/scala DataSource.scala program is still as it is in previously
>> mentioned.
>>
>> Thanks,
>> -DAN
>>
>> On Wed, Jul 26, 2017 at 3:22 PM, Tom Chan <yukhei.chan@gmail.com> wrote:
>>
>>> darshan@darshu:~/PredictionIO/tapster-episode-similar/src/main/scala$
>>> pio train
>>>
>>> Can you try the command from the tapster-episode-similar directory?
>>>
>>> Tom
>>>
>>> On Jul 26, 2017 2:45 AM, "Darshan A.N." <darshanan.24@gmail.com> wrote:
>>>
>>>> hi team,
>>>> i am trying to install demo tapster. i  followed
>>>> http://predictionio.incubator.apache.org/demo/tapster/.     It took
>>>> more than a week to install predictionio.
>>>> now that i m installed the PIO, its throwing me error while running the
>>>> $pio build command. the error goes like this:
>>>> darshan@darshu:~/PredictionIO/tapster-episode-similar$ pio build
>>>> SLF4J: Class path contains multiple SLF4J bindings.
>>>> SLF4J: Found binding in [jar:file:/home/darshan/Predic
>>>> tionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.ja
>>>> r!/org/slf4j/impl/StaticLoggerBinder.class]
>>>> SLF4J: Found binding in [jar:file:/home/darshan/Predic
>>>> tionIO/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/imp
>>>> l/StaticLoggerBinder.class]
>>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>>>> explanation.
>>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>>>> [INFO] [Engine$] Using command '/home/darshan/PredictionIO/sbt/sbt' at
>>>> /home/darshan/PredictionIO/tapster-episode-similar to build.
>>>> [INFO] [Engine$] If the path above is incorrect, this process will fail.
>>>> [INFO] [Engine$] Uber JAR disabled. Making sure
>>>> lib/pio-assembly-0.11.0-incubating.jar is absent.
>>>> [INFO] [Engine$] Going to run: /home/darshan/PredictionIO/sbt/sbt
>>>> package assemblyPackageDependency in /home/darshan/PredictionIO/tap
>>>> ster-episode-similar
>>>> [ERROR] [Engine$] [error] /home/darshan/PredictionIO/tap
>>>> ster-episode-similar/src/main/scala/DataSource.scala:63: not found:
>>>> value eventsDb
>>>> [ERROR] [Engine$] [error]     val viewEventsRDD: RDD[ViewEvent] =
>>>> eventsDb.find(
>>>> [ERROR] [Engine$] [error]                                         ^
>>>> [ERROR] [Engine$] [error] one error found
>>>> [ERROR] [Engine$] [error] (compile:compileIncremental) Compilation
>>>> failed
>>>> [ERROR] [Engine$] [error] Total time: 5 s, completed 26 Jul, 2017
>>>> 1:56:50 AM
>>>> [ERROR] [Engine$] Return code of build command:
>>>> /home/darshan/PredictionIO/sbt/sbt  package assemblyPackageDependency
>>>> is 1. Aborting.
>>>> [INFO] [Engine$] Looking for an engine...
>>>> [INFO] [Engine$] Found template-scala-parallel-simila
>>>> rproduct_2.10-0.1-SNAPSHOT.jar
>>>> [INFO] [Engine$] Found template-scala-parallel-simila
>>>> rproduct-assembly-0.1-SNAPSHOT-deps.jar
>>>> [INFO] [Engine$] Build finished successfully.
>>>> [INFO] [Pio$] Your engine is ready for training.
>>>>
>>>>
>>>> and the file DataSource.scala,
>>>>
>>>>
>>>> package org.example.similarproduct
>>>>
>>>> import org.apache.predictionio.controller.PDataSource
>>>> import org.apache.predictionio.controller.EmptyEvaluationInfo
>>>> import org.apache.predictionio.controller.EmptyActualResult
>>>> import org.apache.predictionio.controller.Params
>>>> import org.apache.predictionio.data.storage.Event
>>>> import org.apache.predictionio.data.store.PEventStore
>>>>
>>>> import org.apache.spark.SparkContext
>>>> import org.apache.spark.SparkContext._
>>>> import org.apache.spark.rdd.RDD
>>>>
>>>> import grizzled.slf4j.Logger
>>>>
>>>> case class DataSourceParams(appName: String) extends Params
>>>>
>>>> class DataSource(val dsp: DataSourceParams)
>>>>   extends PDataSource[TrainingData,
>>>>       EmptyEvaluationInfo, Query, EmptyActualResult] {
>>>>
>>>>   @transient lazy val logger = Logger[this.type]
>>>>
>>>>   override
>>>>   def readTraining(sc: SparkContext): TrainingData = {
>>>>
>>>>     // create a RDD of (entityID, User)
>>>>     val usersRDD: RDD[(String, User)] = PEventStore.aggregatePropertie
>>>> s(
>>>>       appName = dsp.appName,
>>>>       entityType = "user"
>>>>     )(sc).map { case (entityId, properties) =>
>>>>       val user = try {
>>>>         User()
>>>>       } catch {
>>>>         case e: Exception => {
>>>>           logger.error(s"Failed to get properties ${properties} of" +
>>>>             s" user ${entityId}. Exception: ${e}.")
>>>>           throw e
>>>>         }
>>>>       }
>>>>       (entityId, user)
>>>>     }.cache()
>>>>
>>>>     // create a RDD of (entityID, Item)
>>>>     val itemsRDD: RDD[(String, Item)] = PEventStore.aggregatePropertie
>>>> s(
>>>>       appName = dsp.appName,
>>>>       entityType = "item"
>>>>     )(sc).map { case (entityId, properties) =>
>>>>       val item = try {
>>>>         // Assume categories is optional property of item.
>>>>         Item(categories = properties.getOpt[List[String]
>>>> ]("categories"))
>>>>       } catch {
>>>>         case e: Exception => {
>>>>           logger.error(s"Failed to get properties ${properties} of" +
>>>>             s" item ${entityId}. Exception: ${e}.")
>>>>           throw e
>>>>         }
>>>>       }
>>>>       (entityId, item)
>>>>     }.cache()
>>>>
>>>>     // get all "user" "view" "item" events
>>>>     val viewEventsRDD: RDD[ViewEvent] = eventsDb.find(
>>>>       appId = dsp.appId,
>>>>       entityType = Some("user"),
>>>>       eventNames = Some(List("like")),
>>>>       // targetEntityType is optional field of an event.
>>>>       targetEntityType = Some(Some("item")))(sc)
>>>>       // eventsDb.find() returns RDD[Event]
>>>>       .map { event =>
>>>>         val viewEvent = try {
>>>>           event.event match {
>>>>             case "like" => ViewEvent(
>>>>               user = event.entityId,
>>>>               item = event.targetEntityId.get,
>>>>               t = event.eventTime.getMillis)
>>>>             case _ => throw new Exception(s"Unexpected event ${event}
>>>> is read.")
>>>>           }
>>>>         } catch {
>>>>           case e: Exception => {
>>>>             logger.error(s"Cannot convert ${event} to ViewEvent." +
>>>>               s" Exception: ${e}.")
>>>>             throw e
>>>>           }
>>>>         }
>>>>         viewEvent
>>>>       }.cache()
>>>>
>>>>     new TrainingData(
>>>>       users = usersRDD,
>>>>       items = itemsRDD,
>>>>       viewEvents = viewEventsRDD
>>>>     )
>>>>   }
>>>> }
>>>>
>>>> case class User()
>>>>
>>>> case class Item(categories: Option[List[String]])
>>>>
>>>> case class ViewEvent(user: String, item: String, t: Long)
>>>>
>>>> class TrainingData(
>>>>   val users: RDD[(String, User)],
>>>>   val items: RDD[(String, Item)],
>>>>   val viewEvents: RDD[ViewEvent]
>>>> ) extends Serializable {
>>>>   override def toString = {
>>>>     s"users: [${users.count()} (${users.take(2).toList}...)]" +
>>>>     s"items: [${items.count()} (${items.take(2).toList}...)]" +
>>>>     s"viewEvents: [${viewEvents.count()}] (${viewEvents.take(2).toList}.
>>>> ..)"
>>>>   }
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>> while training, it gives  me following error,
>>>>
>>>>
>>>>
>>>>
>>>> darshan@darshu:~/PredictionIO/tapster-episode-similar/src/main/scala$
>>>> pio train
>>>> SLF4J: Class path contains multiple SLF4J bindings.
>>>> SLF4J: Found binding in [jar:file:/home/darshan/Predic
>>>> tionIO/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.ja
>>>> r!/org/slf4j/impl/StaticLoggerBinder.class]
>>>> SLF4J: Found binding in [jar:file:/home/darshan/Predic
>>>> tionIO/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/imp
>>>> l/StaticLoggerBinder.class]
>>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>>>> explanation.
>>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>>>> [WARN] [Template$] /home/darshan/PredictionIO/tap
>>>> ster-episode-similar/src/main/scala/template.json does not exist.
>>>> Template metadata will not be available. (This is safe to ignore if you are
>>>> not working on a template.)
>>>> Exception in thread "main" java.io.FileNotFoundException:
>>>> /home/darshan/PredictionIO/tapster-episode-similar/src/main/scala/engine.json
>>>> (No such file or directory)
>>>>     at java.io.FileInputStream.open0(Native Method)
>>>>     at java.io.FileInputStream.open(FileInputStream.java:195)
>>>>     at java.io.FileInputStream.<init>(FileInputStream.java:138)
>>>>     at scala.io.Source$.fromFile(Source.scala:90)
>>>>     at scala.io.Source$.fromFile(Source.scala:75)
>>>>     at org.apache.predictionio.tools.console.Console$.getEngineInfo
>>>> (Console.scala:724)
>>>>     at org.apache.predictionio.tools.RunWorkflow$.runWorkflow(RunWo
>>>> rkflow.scala:54)
>>>>     at org.apache.predictionio.tools.commands.Engine$.train(Engine.
>>>> scala:186)
>>>>     at org.apache.predictionio.tools.console.Pio$.train(Pio.scala:85)
>>>>     at org.apache.predictionio.tools.console.Console$$anonfun$main$
>>>> 1.apply(Console.scala:626)
>>>>     at org.apache.predictionio.tools.console.Console$$anonfun$main$
>>>> 1.apply(Console.scala:611)
>>>>     at scala.Option.map(Option.scala:145)
>>>>     at org.apache.predictionio.tools.console.Console$.main(Console.
>>>> scala:611)
>>>>     at org.apache.predictionio.tools.console.Console.main(Console.s
>>>> cala)
>>>>
>>>>
>>>>
>>>> i tried almost all the things, but could not find the proper solution.
>>>> please help me...
>>>> i know you may feel bit odd about this mail, but in need of your help.
>>>>
>>>> thanks,
>>>>
>>>>
>>>> -DAN
>>>>
>>>
>>
>

Mime
View raw message