Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4E239DC3C for ; Wed, 13 Mar 2013 18:08:13 +0000 (UTC) Received: (qmail 92310 invoked by uid 500); 13 Mar 2013 18:08:08 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 92208 invoked by uid 500); 13 Mar 2013 18:08:08 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 92200 invoked by uid 99); 13 Mar 2013 18:08:08 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 13 Mar 2013 18:08:08 +0000 X-ASF-Spam-Status: No, hits=1.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of vajjalak009@gmail.com designates 74.125.82.193 as permitted sender) Received: from [74.125.82.193] (HELO mail-we0-f193.google.com) (74.125.82.193) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 13 Mar 2013 18:08:02 +0000 Received: by mail-we0-f193.google.com with SMTP id z53so340601wey.4 for ; Wed, 13 Mar 2013 11:07:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=r8gG1oRSrjRJfAphCoXWskFb7bTNXU0S09GmT0/QfPA=; b=HSmJlcw3iIqmv0vnE6Hp7S+mXaLvzRAoSFTBTRKIDUCnUFlHd0/EyH25pjzKAf1YcU eczI2OjWtzpWKaUUjmGgINYizxpd19Y4ECjSFmoY/DQQrqY5fJfIMm1Dm1igR9ZZvs9Y jVJqMIymk/vRMm4zvMcpLStDqS1TBiOmiCigpmQ7uEqPNIzECpKcXQ07yHx9fBzvrTlU J1P2u5K7aB35LQPE0VA1LCwSWnaKRTq3J5bgYvZZwmVnJ4LK56+ynfWMkGm/QvrPD/vx R80NzVKNhLt/dyedC9Djw3eOcji9dS1lBS5/qvriB8GCX/KJw1z2nUBkqOZr98bHihrU 9dNw== MIME-Version: 1.0 X-Received: by 10.194.235.196 with SMTP id uo4mr35998768wjc.30.1363198061984; Wed, 13 Mar 2013 11:07:41 -0700 (PDT) Received: by 10.216.239.10 with HTTP; Wed, 13 Mar 2013 11:07:41 -0700 (PDT) In-Reply-To: References: Date: Wed, 13 Mar 2013 11:07:41 -0700 Message-ID: Subject: Re: Question regarding hadoop jar command usage From: KayVajj To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=089e01419d7c3e439a04d7d2482a X-Virus-Checked: Checked by ClamAV on apache.org --089e01419d7c3e439a04d7d2482a Content-Type: text/plain; charset=ISO-8859-1 Hi Sandy, I was going through the RunJar source code and the jar executes locally. When the jar fires a mapreduce job, the way I create JobConf is JobConf conf = new JobConf(MyJob.class); > Does this set MyJar as the job jar? Can you explain what is the difference between running an MR job using the jar command vs mapred job command (looks like the hadoop job command is deprecated). Thanks Kay On Wed, Mar 13, 2013 at 10:14 AM, Sandy Ryza wrote: > Hi Kay, > > The jar is just executed locally. If the jar fires up a mapreduce job and > sets itself as the job jar, then mapreduce will handle copying it to the > nodes that will use it. > > -Sandy > > > On Wed, Mar 13, 2013 at 9:01 AM, KayVajj wrote: > >> I have a question regarding the hadoop jar command. In a cluster of say >> nodes n1,n2...n100 >> >> the node n1 has jar Myjar on its local file system. >> >> If I run the command >> hadoop jar local/path/to/Myjar Myclass other-args >> >> Is the MR job executed just on n1 or any arbitrary node n1..n100? >> >> If it is any arbitrary node, since Myjar exists only on n1. Is the jar >> copied across all nodes on the cluster to run the MR code. >> >> >> >> >> > --089e01419d7c3e439a04d7d2482a Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi Sandy,

I was going through the RunJar source code and the jar exe= cutes locally. When the jar fires a mapreduce job,

the way I create= JobConf is

JobConf conf =3D new Jo= bConf(MyJob.class);

Does this set MyJar as the job jar?

Can you explain what is the dif= ference between running an MR job using the jar command vs mapred job comma= nd (looks like the hadoop job command is deprecated).

Thanks
Kay

On Wed, Mar 13, 2013 at 10:14 AM, San= dy Ryza <sandy.ryza@cloudera.com> wrote:
Hi Kay,

The jar is just executed locally. =A0If the jar = fires up a mapreduce job and sets itself as the job jar, then mapreduce wil= l handle copying it to the nodes that will use it.

--089e01419d7c3e439a04d7d2482a--