predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tajrij kawakibi <tajrij.kawak...@gmail.com>
Subject Text Classification Engine Tutorial
Date Thu, 07 Sep 2017 08:53:56 GMT
Hello,

i just finish this 2 tutorial successfully,

https://predictionio.incubator.apache.org/templates/recommendation/quickstart/

https://predictionio.incubator.apache.org/templates/classification/quickstart/

then i try

https://predictionio.incubator.apache.org/demo/textclassification/ ( the
first sample , which is Sample email data for spam classification )

i follow step by step successfully ( create new app, import data, pio
build)  but When i try to train i encounter error

*pio train*
*SLF4J: Class path contains multiple SLF4J bindings.*
*SLF4J: Found binding in
[jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]*
*SLF4J: Found binding in
[jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]*
*SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings
<http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.*
*SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]*
*[WARN] [WorkflowUtils$] Environment variable POSTGRES_JDBC_DRIVER is
pointing to a nonexistent file
/home/admin/PredictionIO-0.11.0-incubating/lib/postgresql-42.0.0.jar.
Ignoring.*
*[WARN] [WorkflowUtils$] Environment variable MYSQL_JDBC_DRIVER is pointing
to a nonexistent file
/home/admin/PredictionIO-0.11.0-incubating/lib/mysql-connector-java-5.1.41.jar.
Ignoring.*
*[INFO] [Runner$] Submission command:
/home/admin/PredictionIO-0.11.0-incubating/vendors/spark-1.6.3-bin-hadoop2.6/bin/spark-submit
--class org.apache.predictionio.workflow.CreateWorkflow --jars
file:/home/admin/text-classifier/target/scala-2.10/org-example-textclassification_2.10-0.1-SNAPSHOT.jar,file:/home/admin/text-classifier/target/scala-2.10/org.example.textclassification-assembly-0.1-SNAPSHOT-deps.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-elasticsearch1-assembly-0.11.0-incubating.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-localfs-assembly-0.11.0-incubating.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hbase-assembly-0.11.0-incubating.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-jdbc-assembly-0.11.0-incubating.jar
--files
file:/home/admin/PredictionIO-0.11.0-incubating/conf/log4j.properties
--driver-class-path
/home/admin/PredictionIO-0.11.0-incubating/conf:/home/admin/PredictionIO-0.11.0-incubating/lib/postgresql-42.0.0.jar:/home/admin/PredictionIO-0.11.0-incubating/lib/mysql-connector-java-5.1.41.jar
--driver-java-options -Dpio.log.dir=/root
file:/home/admin/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar
--engine-id org.example.textclassification.TextClassificationEngine
--engine-version 3aeef180d388b6b932edf67acade44b6904d9135 --engine-variant
file:/home/admin/text-classifier/engine.json --verbosity 0 --json-extractor
Both --env
PIO_STORAGE_SOURCES_HBASE_TYPE=hbase,PIO_ENV_LOADED=1,PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta,PIO_FS_BASEDIR=/root/.pio_store,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost,PIO_STORAGE_SOURCES_HBASE_HOME=/home/admin/PredictionIO-0.11.0-incubating/vendors/hbase-1.2.6,PIO_HOME=/home/admin/PredictionIO-0.11.0-incubating,PIO_FS_ENGINESDIR=/root/.pio_store/engines,PIO_STORAGE_SOURCES_LOCALFS_PATH=/root/.pio_store/models,PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch,PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH,PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS,PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/home/admin/PredictionIO-0.11.0-incubating/vendors/elasticsearch-1.7.6,PIO_FS_TMPDIR=/root/.pio_store/tmp,PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model,PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE,PIO_CONF_DIR=/home/admin/PredictionIO-0.11.0-incubating/conf,PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300,PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs*
*[INFO] [Engine] Extracting datasource params...*
*[INFO] [WorkflowUtils$] No 'name' is found. Default empty String will be
used.*
*[INFO] [Engine] Datasource params: (,DataSourceParams(MyTextApp,None))*
*[INFO] [Engine] Extracting preparator params...*
*[INFO] [WorkflowUtils$] No 'name' is found. Default empty String will be
used.*
*[INFO] [Engine] Preparator params: (,PreparatorParams(1,500))*
*[INFO] [Engine] Extracting serving params...*
*[INFO] [Engine] Serving params: (,Empty)*
*[ERROR] [Storage$] Error initializing storage client for source
ELASTICSEARCH*
*Exception in thread "main"
org.apache.predictionio.data.storage.StorageClientException: Data source
ELASTICSEARCH was not properly initialized.*
* at
org.apache.predictionio.data.storage.Storage$$anonfun$10.apply(Storage.scala:285)*
* at
org.apache.predictionio.data.storage.Storage$$anonfun$10.apply(Storage.scala:285)*
* at scala.Option.getOrElse(Option.scala:120)*
* at
org.apache.predictionio.data.storage.Storage$.getDataObject(Storage.scala:284)*
* at
org.apache.predictionio.data.storage.Storage$.getDataObjectFromRepo(Storage.scala:269)*
* at
org.apache.predictionio.data.storage.Storage$.getMetaDataEngineInstances(Storage.scala:371)*
* at
org.apache.predictionio.workflow.CreateWorkflow$.main(CreateWorkflow.scala:247)*
* at
org.apache.predictionio.workflow.CreateWorkflow.main(CreateWorkflow.scala)*
* at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)*
* at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)*
* at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)*
* at java.lang.reflect.Method.invoke(Method.java:498)*
* at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)*
* at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)*
* at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)*
* at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)*
* at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)*

My conf at pio-env.sh is

- Elastic search as Metadata repo
- Hbase as eventdata repo
- localfs as model repo

result from pio status

pio status
*SLF4J: Class path contains multiple SLF4J bindings.*
*SLF4J: Found binding in
[jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]*
*SLF4J: Found binding in
[jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]*
*SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings
<http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.*
*SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]*
*[INFO] [Management$] Inspecting PredictionIO...*
*[INFO] [Management$] PredictionIO 0.11.0-incubating is installed at
/home/admin/PredictionIO-0.11.0-incubating*
*[INFO] [Management$] Inspecting Apache Spark...*
*[INFO] [Management$] Apache Spark is installed at
/home/admin/PredictionIO-0.11.0-incubating/vendors/spark-1.6.3-bin-hadoop2.6*
*[INFO] [Management$] Apache Spark 1.6.3 detected (meets minimum
requirement of 1.3.0)*
*[INFO] [Management$] Inspecting storage backend connections...*
*[INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...*
*[INFO] [Storage$] Verifying Model Data Backend (Source: LOCALFS)...*
*[INFO] [Storage$] Verifying Event Data Backend (Source: HBASE)...*
*[INFO] [Storage$] Test writing to Event Store (App Id 0)...*
*[INFO] [HBLEvents] The table pio_event:events_0 doesn't exist yet.
Creating now...*
*[INFO] [HBLEvents] Removing table pio_event:events_0...*
*[INFO] [Management$] Your system is all ready to go.*


result from pio build

pio build --verbose
*SLF4J: Class path contains multiple SLF4J bindings.*
*SLF4J: Found binding in
[jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]*
*SLF4J: Found binding in
[jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]*
*SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings
<http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.*
*SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]*
*[INFO] [Engine$] Using command
'/home/admin/PredictionIO-0.11.0-incubating/sbt/sbt' at
/home/admin/text-classifier to build.*
*[INFO] [Engine$] If the path above is incorrect, this process will fail.*
*[INFO] [Engine$] Uber JAR disabled. Making sure
lib/pio-assembly-0.11.0-incubating.jar is absent.*
*[INFO] [Engine$] Going to run:
/home/admin/PredictionIO-0.11.0-incubating/sbt/sbt  package
assemblyPackageDependency in /home/admin/text-classifier*
*[INFO] [Engine$] [info] Loading project definition from
/home/admin/text-classifier/project*
*[INFO] [Engine$] [info] Set current project to
org.example.textclassification (in build file:/home/admin/text-classifier/)*
*[INFO] [Engine$] [success] Total time: 2 s, completed Sep 7, 2017 8:43:48
AM*
*[INFO] [Engine$] [info] Including from cache: scala-library.jar*
*[INFO] [Engine$] [info] Including from cache: lucene-core-6.5.1.jar*
*[INFO] [Engine$] [info] Checking every *.class/*.jar file's SHA-1.*
*[INFO] [Engine$] [info] Merging files...*
*[INFO] [Engine$] [warn] Merging 'META-INF/MANIFEST.MF' with strategy
'discard'*
*[INFO] [Engine$] [warn] Strategy 'discard' was applied to a file*
*[INFO] [Engine$] [info] Assembly up to date:
/home/admin/text-classifier/target/scala-2.10/org.example.textclassification-assembly-0.1-SNAPSHOT-deps.jar*
*[INFO] [Engine$] [success] Total time: 4 s, completed Sep 7, 2017 8:43:53
AM*
*[INFO] [Engine$] Compilation finished successfully.*
*[INFO] [Engine$] Looking for an engine...*
*[INFO] [Engine$] Found
org-example-textclassification_2.10-0.1-SNAPSHOT.jar*
*[INFO] [Engine$] Found
org.example.textclassification-assembly-0.1-SNAPSHOT-deps.jar*
*[INFO] [Engine$] Build finished successfully.*
*[INFO] [Pio$] Your engine is ready for training.*

*is there something wrong with my conf ? or am i missing something ?*

Mime
View raw message