samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From George Li <...@ca.ibm.com>
Subject Re: Deploying samza job to a remote yarn resource manager
Date Tue, 23 Jun 2015 15:44:56 GMT
Hi Navina and Milinda,

Thanks for your reply. I get it working by setting HADOOP_YARN_HOME and 
HADOOP_CONF_DIR

export HADOOP_YARN_HOME=$HOME/hadoop-2.6.0/
export HADOOP_CONF_DIR=$HOME/hadoop-2.6.0/etc/hadoop

,i.e., similar to what run-class.sh does


I am also trying to figure out how to deploy samza job by code. By porting 
over org.apache.samza.job.JobRunner, I am able to deploy a job as long as 
yarn resource manager is on the same host, but how do I pass the remote 
resource manager address? Maybe I should include  HADOOP_YARN_HOME and 
HADOOP_CONF_DIR as classpath?

George




From:   Navina Ramesh <nramesh@linkedin.com.INVALID>
To:     "dev@samza.apache.org" <dev@samza.apache.org>, 
Date:   22/06/2015 05:57 PM
Subject:        Re: Deploying samza job to a remote yarn resource manager



Hi George,

Looks like there is a formatting issue on the samza website. The
environment variable name should be HADOOP_YARN_HOME.
In your case, it should point to hadoop home: $HOME/hadoop-2.6.0/

Please let me know if it works.

Cheers!
Navina

On 6/22/15, 2:11 PM, "George Li" <gli@ca.ibm.com> wrote:

>Hi,
>
>According to
>
>https://samza.apache.org/learn/documentation/0.9/jobs/yarn-jobs.html
>
>, to deploy samza jobs to a remote yarn resource manager, I need to set
>the HADOOPYARNHOME environment variable. What should be the value of this
>variable if my yarn-site.xml is at
>$HOME/hadoop-2.6.0/etc/hadoop/yarn-site.xml?
>
>Thanks,
>
>George



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message