spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Udit Mehta <>
Subject How to use the --files arg
Date Fri, 10 Apr 2015 21:56:07 GMT

Suppose I have a command and I pass the --files arg as below:

bin/spark-submit --class com.test.HelloWorld --master yarn-cluster
--num-executors 8 --driver-memory 512m --executor-memory 2048m
--executor-cores 4 --queue public * --files $HOME/myfile.txt*  --name
test_1 ~/test_code-1.0-SNAPSHOT.jar

Can anyone tell me how do I access this file in my executors?
Basically I want to read this file to get some configs. I tries to read
from my HDFS Home dir but that doesnt work.


View raw message