Thank you, Narasingu and Rekha for your help.
So there are no way for hadoop streaming to read an environment variable from the OS? Guess that I'll have to use the "-cmdenv" command to specify the PATH and CLASSPATH variables.
Again, appreciate your help.
What I said means first he has set install java and then pwd(present working directory) for JAVA_HOME directory and then set to classpath for the hadoop installation.
Thanks & Regards,
On Tue, Sep 4, 2012 at 1:36 PM, Joshi, Rekha <Rekha_Joshi@intuit.com> wrote:Hi Andy,
If you are referring to HADOOP_CLASSPATH, that is env variable on your cluster or effected via config xml.But if you need your own environment variables for streaming you may use -cmdenv PATH= on your streaming command.Or if you have specific jars for the streaming process -libjars <application.jar> on command works.
I wish to use Hadoop streaming to run a program which requires specific PATH and CLASSPATH variables. I have set these two variables in both "/etc/profile" and "~/.bashrc" on all slaves (and restarted these slaves). However, when I run the hadoop streaming job, the program generates error messages saying that these environment variables are not correctly set.
Does it mean that I did not set these environment variables in the correct place? How should I define these environment variables?I use Hadoop 1.0.3 and all machines are running Ubuntu 12.04
Thank you for your time and help. Have a great day.