Return-Path: X-Original-To: apmail-samza-dev-archive@minotaur.apache.org Delivered-To: apmail-samza-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 26E6F18B83 for ; Tue, 23 Jun 2015 15:45:22 +0000 (UTC) Received: (qmail 50022 invoked by uid 500); 23 Jun 2015 15:45:22 -0000 Delivered-To: apmail-samza-dev-archive@samza.apache.org Received: (qmail 49963 invoked by uid 500); 23 Jun 2015 15:45:22 -0000 Mailing-List: contact dev-help@samza.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@samza.apache.org Delivered-To: mailing list dev@samza.apache.org Received: (qmail 49950 invoked by uid 99); 23 Jun 2015 15:45:21 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 23 Jun 2015 15:45:21 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 45CE0C006D for ; Tue, 23 Jun 2015 15:45:21 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.991 X-Spam-Level: ** X-Spam-Status: No, score=2.991 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=3, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id nx1nln0vaLjH for ; Tue, 23 Jun 2015 15:45:09 +0000 (UTC) Received: from e18.ny.us.ibm.com (e18.ny.us.ibm.com [129.33.205.208]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id 9E49524971 for ; Tue, 23 Jun 2015 15:45:08 +0000 (UTC) Received: from /spool/local by e18.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 23 Jun 2015 11:45:07 -0400 Received: from d01dlp01.pok.ibm.com (9.56.250.166) by e18.ny.us.ibm.com (146.89.104.205) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 23 Jun 2015 11:45:06 -0400 X-Helo: d01dlp01.pok.ibm.com X-MailFrom: gli@ca.ibm.com X-RcptTo: dev@samza.apache.org Received: from b01cxnp23034.gho.pok.ibm.com (b01cxnp23034.gho.pok.ibm.com [9.57.198.29]) by d01dlp01.pok.ibm.com (Postfix) with ESMTP id 665D438C8046 for ; Tue, 23 Jun 2015 11:45:05 -0400 (EDT) Received: from d01av05.pok.ibm.com (d01av05.pok.ibm.com [9.56.224.195]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id t5NFj5qa66715840 for ; Tue, 23 Jun 2015 15:45:05 GMT Received: from d01av05.pok.ibm.com (localhost [127.0.0.1]) by d01av05.pok.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id t5NFj4hj012479 for ; Tue, 23 Jun 2015 11:45:04 -0400 Received: from d25ml02.torolab.ibm.com (d25ml02.torolab.ibm.com [9.26.29.95]) by d01av05.pok.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id t5NFj3cV012344 for ; Tue, 23 Jun 2015 11:45:04 -0400 In-Reply-To: References: To: dev@samza.apache.org MIME-Version: 1.0 Subject: Re: Deploying samza job to a remote yarn resource manager X-KeepSent: C220099C:094E09F3-85257E6D:005427A6; type=4; name=$KeepSent X-Mailer: Lotus Notes Release 8.5.3FP5SHF238 December 19, 2013 Message-ID: From: George Li Date: Tue, 23 Jun 2015 11:44:56 -0400 X-MIMETrack: Serialize by Router on D25ML02/25/M/IBM(Release 9.0.1FP3HF243 | March 21, 2015) at 06/23/2015 11:45:04, Serialize complete at 06/23/2015 11:45:04 Content-Type: multipart/alternative; boundary="=_alternative 0056805085257E6D_=" X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15062315-0045-0000-0000-000000852514 --=_alternative 0056805085257E6D_= Content-Type: text/plain; charset="US-ASCII" Hi Navina and Milinda, Thanks for your reply. I get it working by setting HADOOP_YARN_HOME and HADOOP_CONF_DIR export HADOOP_YARN_HOME=$HOME/hadoop-2.6.0/ export HADOOP_CONF_DIR=$HOME/hadoop-2.6.0/etc/hadoop ,i.e., similar to what run-class.sh does I am also trying to figure out how to deploy samza job by code. By porting over org.apache.samza.job.JobRunner, I am able to deploy a job as long as yarn resource manager is on the same host, but how do I pass the remote resource manager address? Maybe I should include HADOOP_YARN_HOME and HADOOP_CONF_DIR as classpath? George From: Navina Ramesh To: "dev@samza.apache.org" , Date: 22/06/2015 05:57 PM Subject: Re: Deploying samza job to a remote yarn resource manager Hi George, Looks like there is a formatting issue on the samza website. The environment variable name should be HADOOP_YARN_HOME. In your case, it should point to hadoop home: $HOME/hadoop-2.6.0/ Please let me know if it works. Cheers! Navina On 6/22/15, 2:11 PM, "George Li" wrote: >Hi, > >According to > >https://samza.apache.org/learn/documentation/0.9/jobs/yarn-jobs.html > >, to deploy samza jobs to a remote yarn resource manager, I need to set >the HADOOPYARNHOME environment variable. What should be the value of this >variable if my yarn-site.xml is at >$HOME/hadoop-2.6.0/etc/hadoop/yarn-site.xml? > >Thanks, > >George --=_alternative 0056805085257E6D_=--