predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From amal kumar <amal.kmr.si...@gmail.com>
Subject pio train -- --master yarn-cluster | Error
Date Tue, 18 Oct 2016 10:32:35 GMT
Hi,

We are using Spark on yarn and after referring the below URL, we have been
able to submit the jobs to yarn from remote machine (i.e. PredictionIO
Server).

http://theckang.com/2015/remote-spark-jobs-on-yarn/

1. copied core-site.xml and yarn-site.xml from Yarn cluster onto remote
machine (i.e. PredictionIO Server)
2. set the HADOOP_CONF_DIR environment variable in spark-env.sh (locally
installed copy) on the remote machine to locate the files core-site.xml and
yarn-site.xml


Now, when I am trying to train the model using the below command, I get a
new error.

pio train -- --master yarn-cluster

*Console Error Logs:*
java.io.FileNotFoundException: File does not exist:
hdfs://ip-172-31-45-33.us-west-2.compute.internal:8020/user/root/.sparkStaging/application_1476763882145_0069/hbase-site.xml

*Yarn Error Logs:*
[ERROR] [CreateWorkflow$] Error reading from file: File
file:/home/user/PredictionIO/SimilarProductRecommendation/engine.json does
not exist. Aborting workflow


I also tried to pass the file path, but no luck.

pio train -- --master yarn-cluster  --files
file:/home/user/PredictionIO/SimilarProductRecommendation/engine.json


Thanks,
Amal

Mime
View raw message