predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tajrij kawakibi <tajrij.kawak...@gmail.com>
Subject Re: Text Classification Engine Tutorial
Date Thu, 14 Sep 2017 08:40:21 GMT
Hi donald,

ES version is 1.7.6

this is log when i ran "pio train "

2017-09-14 15:38:06,133 INFO  org.apache.predictionio.tools.Runner$ [main]
- Submission command:
/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/vendors/spark-1.6.3-bin-hadoop2.6/bin/spark-submit
--class org.apache.predictionio.workflow.CreateWorkflow --jars
file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/tex-classifier/target/scala-2.10/org-example-textclassification_2.10-0.1-SNAPSHOT.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/tex-classifier/target/scala-2.10/org.example.textclassification-assembly-0.1-SNAPSHOT-deps.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/spark/pio-data-elasticsearch1-assembly-0.11.0-incubating.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hbase-assembly-0.11.0-incubating.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/spark/pio-data-jdbc-assembly-0.11.0-incubating.jar,file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/spark/pio-data-localfs-assembly-0.11.0-incubating.jar
--files
file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/conf/log4j.properties
--driver-class-path
/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/conf
--driver-java-options -Dpio.log.dir=/Users/tajrijkawakibi
file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar
--engine-id org.example.textclassification.TextClassificationEngine
--engine-version 440c18b2a720f53b28d2ec839cf1f5d7660485d9 --engine-variant
file:/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/tex-classifier/engine.json
--verbosity 0 --json-extractor Both --env
PIO_STORAGE_SOURCES_HBASE_TYPE=hbase,PIO_ENV_LOADED=1,PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta,PIO_FS_BASEDIR=/Users/tajrijkawakibi/.pio_store,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost,PIO_STORAGE_SOURCES_HBASE_HOME=/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/vendors/hbase-1.2.6,PIO_HOME=/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating,PIO_FS_ENGINESDIR=/Users/tajrijkawakibi/.pio_store/engines,PIO_STORAGE_SOURCES_LOCALFS_PATH=/Users/tajrijkawakibi/.pio_store/models,PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch,PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH,PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS,PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/vendors/elasticsearch-1.7.6,PIO_FS_TMPDIR=/Users/tajrijkawakibi/.pio_store/tmp,PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model,PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE,PIO_CONF_DIR=/Users/tajrijkawakibi/jobs/predictionIO/PredictionIO-0.11.0-incubating/conf,PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300,PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
2017-09-14 15:38:08,668 INFO  org.apache.predictionio.controller.Engine
[main] - Extracting datasource params...
2017-09-14 15:38:08,754 INFO
org.apache.predictionio.workflow.WorkflowUtils$ [main] - No 'name' is
found. Default empty String will be used.
2017-09-14 15:38:08,993 INFO  org.apache.predictionio.controller.Engine
[main] - Datasource params: (,DataSourceParams(Text-1,None))
2017-09-14 15:38:08,993 INFO  org.apache.predictionio.controller.Engine
[main] - Extracting preparator params...
2017-09-14 15:38:08,995 INFO
org.apache.predictionio.workflow.WorkflowUtils$ [main] - No 'name' is
found. Default empty String will be used.
2017-09-14 15:38:09,003 INFO  org.apache.predictionio.controller.Engine
[main] - Preparator params: (,PreparatorParams(1,500))
2017-09-14 15:38:09,020 INFO  org.apache.predictionio.controller.Engine
[main] - Extracting serving params...
2017-09-14 15:38:09,023 INFO  org.apache.predictionio.controller.Engine
[main] - Serving params: (,Empty)
2017-09-14 15:38:09,427 ERROR org.apache.predictionio.data.storage.Storage$
[main] - Error initializing storage client for source ELASTICSEARCH
java.lang.NoSuchFieldError: LUCENE_3_6
at org.elasticsearch.Version.<clinit>(Version.java:43)
at
org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:172)
at
org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:133)
at
org.apache.predictionio.data.storage.elasticsearch.StorageClient.liftedTree1$1(StorageClient.scala:40)
at
org.apache.predictionio.data.storage.elasticsearch.StorageClient.<init>(StorageClient.scala:33)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.predictionio.data.storage.Storage$.getClient(Storage.scala:223)
at
org.apache.predictionio.data.storage.Storage$.org$apache$predictionio$data$storage$Storage$$updateS2CM(Storage.scala:254)
at
org.apache.predictionio.data.storage.Storage$$anonfun$sourcesToClientMeta$1.apply(Storage.scala:215)
at
org.apache.predictionio.data.storage.Storage$$anonfun$sourcesToClientMeta$1.apply(Storage.scala:215)
at scala.collection.mutable.MapLike$class.getOrElseUpdate(MapLike.scala:189)
at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:91)
at
org.apache.predictionio.data.storage.Storage$.sourcesToClientMeta(Storage.scala:215)
at
org.apache.predictionio.data.storage.Storage$.getDataObject(Storage.scala:284)
at
org.apache.predictionio.data.storage.Storage$.getDataObjectFromRepo(Storage.scala:269)
at
org.apache.predictionio.data.storage.Storage$.getMetaDataEngineInstances(Storage.scala:371)
at
org.apache.predictionio.workflow.CreateWorkflow$.main(CreateWorkflow.scala:247)
at
org.apache.predictionio.workflow.CreateWorkflow.main(CreateWorkflow.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)



On Tue, Sep 12, 2017 at 10:52 PM, Donald Szeto <donald@apache.org> wrote:

> Hi,
>
> What version of Elasticsearch are you running? Do you see more information
> from pio.log at where you ran "pio train"?
>
> Regards,
> Donald
>
> On Thu, Sep 7, 2017 at 1:54 AM tajrij kawakibi <tajrij.kawakibi@gmail.com>
> wrote:
>
>> Hello,
>>
>> i just finish this 2 tutorial successfully,
>>
>> https://predictionio.incubator.apache.org/templates/recommendation/
>> quickstart/
>>
>> https://predictionio.incubator.apache.org/templates/classification/
>> quickstart/
>>
>> then i try
>>
>> https://predictionio.incubator.apache.org/demo/textclassification/ ( the
>> first sample , which is Sample email data for spam classification )
>>
>> i follow step by step successfully ( create new app, import data, pio
>> build)  but When i try to train i encounter error
>>
>> *pio train*
>> *SLF4J: Class path contains multiple SLF4J bindings.*
>> *SLF4J: Found binding in
>> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]*
>> *SLF4J: Found binding in
>> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]*
>> *SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings
>> <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.*
>> *SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]*
>> *[WARN] [WorkflowUtils$] Environment variable POSTGRES_JDBC_DRIVER is
>> pointing to a nonexistent file
>> /home/admin/PredictionIO-0.11.0-incubating/lib/postgresql-42.0.0.jar.
>> Ignoring.*
>> *[WARN] [WorkflowUtils$] Environment variable MYSQL_JDBC_DRIVER is
>> pointing to a nonexistent file
>> /home/admin/PredictionIO-0.11.0-incubating/lib/mysql-connector-java-5.1.41.jar.
>> Ignoring.*
>> *[INFO] [Runner$] Submission command:
>> /home/admin/PredictionIO-0.11.0-incubating/vendors/spark-1.6.3-bin-hadoop2.6/bin/spark-submit
>> --class org.apache.predictionio.workflow.CreateWorkflow --jars
>> file:/home/admin/text-classifier/target/scala-2.10/org-example-textclassification_2.10-0.1-SNAPSHOT.jar,file:/home/admin/text-classifier/target/scala-2.10/org.example.textclassification-assembly-0.1-SNAPSHOT-deps.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-elasticsearch1-assembly-0.11.0-incubating.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-localfs-assembly-0.11.0-incubating.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hbase-assembly-0.11.0-incubating.jar,file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-jdbc-assembly-0.11.0-incubating.jar
>> --files
>> file:/home/admin/PredictionIO-0.11.0-incubating/conf/log4j.properties
>> --driver-class-path
>> /home/admin/PredictionIO-0.11.0-incubating/conf:/home/admin/PredictionIO-0.11.0-incubating/lib/postgresql-42.0.0.jar:/home/admin/PredictionIO-0.11.0-incubating/lib/mysql-connector-java-5.1.41.jar
>> --driver-java-options -Dpio.log.dir=/root
>> file:/home/admin/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar
>> --engine-id org.example.textclassification.TextClassificationEngine
>> --engine-version 3aeef180d388b6b932edf67acade44b6904d9135 --engine-variant
>> file:/home/admin/text-classifier/engine.json --verbosity 0 --json-extractor
>> Both --env
>> PIO_STORAGE_SOURCES_HBASE_TYPE=hbase,PIO_ENV_LOADED=1,PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta,PIO_FS_BASEDIR=/root/.pio_store,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost,PIO_STORAGE_SOURCES_HBASE_HOME=/home/admin/PredictionIO-0.11.0-incubating/vendors/hbase-1.2.6,PIO_HOME=/home/admin/PredictionIO-0.11.0-incubating,PIO_FS_ENGINESDIR=/root/.pio_store/engines,PIO_STORAGE_SOURCES_LOCALFS_PATH=/root/.pio_store/models,PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch,PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH,PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS,PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/home/admin/PredictionIO-0.11.0-incubating/vendors/elasticsearch-1.7.6,PIO_FS_TMPDIR=/root/.pio_store/tmp,PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model,PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE,PIO_CONF_DIR=/home/admin/PredictionIO-0.11.0-incubating/conf,PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300,PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs*
>> *[INFO] [Engine] Extracting datasource params...*
>> *[INFO] [WorkflowUtils$] No 'name' is found. Default empty String will be
>> used.*
>> *[INFO] [Engine] Datasource params: (,DataSourceParams(MyTextApp,None))*
>> *[INFO] [Engine] Extracting preparator params...*
>> *[INFO] [WorkflowUtils$] No 'name' is found. Default empty String will be
>> used.*
>> *[INFO] [Engine] Preparator params: (,PreparatorParams(1,500))*
>> *[INFO] [Engine] Extracting serving params...*
>> *[INFO] [Engine] Serving params: (,Empty)*
>> *[ERROR] [Storage$] Error initializing storage client for source
>> ELASTICSEARCH*
>> *Exception in thread "main"
>> org.apache.predictionio.data.storage.StorageClientException: Data source
>> ELASTICSEARCH was not properly initialized.*
>> * at
>> org.apache.predictionio.data.storage.Storage$$anonfun$10.apply(Storage.scala:285)*
>> * at
>> org.apache.predictionio.data.storage.Storage$$anonfun$10.apply(Storage.scala:285)*
>> * at scala.Option.getOrElse(Option.scala:120)*
>> * at
>> org.apache.predictionio.data.storage.Storage$.getDataObject(Storage.scala:284)*
>> * at
>> org.apache.predictionio.data.storage.Storage$.getDataObjectFromRepo(Storage.scala:269)*
>> * at
>> org.apache.predictionio.data.storage.Storage$.getMetaDataEngineInstances(Storage.scala:371)*
>> * at
>> org.apache.predictionio.workflow.CreateWorkflow$.main(CreateWorkflow.scala:247)*
>> * at
>> org.apache.predictionio.workflow.CreateWorkflow.main(CreateWorkflow.scala)*
>> * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)*
>> * at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)*
>> * at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)*
>> * at java.lang.reflect.Method.invoke(Method.java:498)*
>> * at
>> org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)*
>> * at
>> org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)*
>> * at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)*
>> * at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)*
>> * at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)*
>>
>> My conf at pio-env.sh is
>>
>> - Elastic search as Metadata repo
>> - Hbase as eventdata repo
>> - localfs as model repo
>>
>> result from pio status
>>
>> pio status
>> *SLF4J: Class path contains multiple SLF4J bindings.*
>> *SLF4J: Found binding in
>> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]*
>> *SLF4J: Found binding in
>> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]*
>> *SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings
>> <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.*
>> *SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]*
>> *[INFO] [Management$] Inspecting PredictionIO...*
>> *[INFO] [Management$] PredictionIO 0.11.0-incubating is installed at
>> /home/admin/PredictionIO-0.11.0-incubating*
>> *[INFO] [Management$] Inspecting Apache Spark...*
>> *[INFO] [Management$] Apache Spark is installed at
>> /home/admin/PredictionIO-0.11.0-incubating/vendors/spark-1.6.3-bin-hadoop2.6*
>> *[INFO] [Management$] Apache Spark 1.6.3 detected (meets minimum
>> requirement of 1.3.0)*
>> *[INFO] [Management$] Inspecting storage backend connections...*
>> *[INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...*
>> *[INFO] [Storage$] Verifying Model Data Backend (Source: LOCALFS)...*
>> *[INFO] [Storage$] Verifying Event Data Backend (Source: HBASE)...*
>> *[INFO] [Storage$] Test writing to Event Store (App Id 0)...*
>> *[INFO] [HBLEvents] The table pio_event:events_0 doesn't exist yet.
>> Creating now...*
>> *[INFO] [HBLEvents] Removing table pio_event:events_0...*
>> *[INFO] [Management$] Your system is all ready to go.*
>>
>>
>> result from pio build
>>
>> pio build --verbose
>> *SLF4J: Class path contains multiple SLF4J bindings.*
>> *SLF4J: Found binding in
>> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]*
>> *SLF4J: Found binding in
>> [jar:file:/home/admin/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]*
>> *SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings
>> <http://www.slf4j.org/codes.html#multiple_bindings> for an explanation.*
>> *SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]*
>> *[INFO] [Engine$] Using command
>> '/home/admin/PredictionIO-0.11.0-incubating/sbt/sbt' at
>> /home/admin/text-classifier to build.*
>> *[INFO] [Engine$] If the path above is incorrect, this process will fail.*
>> *[INFO] [Engine$] Uber JAR disabled. Making sure
>> lib/pio-assembly-0.11.0-incubating.jar is absent.*
>> *[INFO] [Engine$] Going to run:
>> /home/admin/PredictionIO-0.11.0-incubating/sbt/sbt  package
>> assemblyPackageDependency in /home/admin/text-classifier*
>> *[INFO] [Engine$] [info] Loading project definition from
>> /home/admin/text-classifier/project*
>> *[INFO] [Engine$] [info] Set current project to
>> org.example.textclassification (in build file:/home/admin/text-classifier/)*
>> *[INFO] [Engine$] [success] Total time: 2 s, completed Sep 7, 2017
>> 8:43:48 AM*
>> *[INFO] [Engine$] [info] Including from cache: scala-library.jar*
>> *[INFO] [Engine$] [info] Including from cache: lucene-core-6.5.1.jar*
>> *[INFO] [Engine$] [info] Checking every *.class/*.jar file's SHA-1.*
>> *[INFO] [Engine$] [info] Merging files...*
>> *[INFO] [Engine$] [warn] Merging 'META-INF/MANIFEST.MF' with strategy
>> 'discard'*
>> *[INFO] [Engine$] [warn] Strategy 'discard' was applied to a file*
>> *[INFO] [Engine$] [info] Assembly up to date:
>> /home/admin/text-classifier/target/scala-2.10/org.example.textclassification-assembly-0.1-SNAPSHOT-deps.jar*
>> *[INFO] [Engine$] [success] Total time: 4 s, completed Sep 7, 2017
>> 8:43:53 AM*
>> *[INFO] [Engine$] Compilation finished successfully.*
>> *[INFO] [Engine$] Looking for an engine...*
>> *[INFO] [Engine$] Found
>> org-example-textclassification_2.10-0.1-SNAPSHOT.jar*
>> *[INFO] [Engine$] Found
>> org.example.textclassification-assembly-0.1-SNAPSHOT-deps.jar*
>> *[INFO] [Engine$] Build finished successfully.*
>> *[INFO] [Pio$] Your engine is ready for training.*
>>
>> *is there something wrong with my conf ? or am i missing something ?*
>>
>>

Mime
View raw message