Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4F0A911713 for ; Sun, 12 May 2013 20:03:21 +0000 (UTC) Received: (qmail 3507 invoked by uid 500); 12 May 2013 20:03:16 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 3402 invoked by uid 500); 12 May 2013 20:03:16 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 3391 invoked by uid 99); 12 May 2013 20:03:16 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 12 May 2013 20:03:15 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_FONT_SIZE_LARGE,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of dechouxb@gmail.com designates 209.85.215.41 as permitted sender) Received: from [209.85.215.41] (HELO mail-la0-f41.google.com) (209.85.215.41) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 12 May 2013 20:03:10 +0000 Received: by mail-la0-f41.google.com with SMTP id lx15so3055660lab.14 for ; Sun, 12 May 2013 13:02:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=9rXNpZrLj5W5zaVuib3GrcTWZ7q8BI7Z+Fy53Rad7mw=; b=GULHn0R6T6waa6MD3+ulvVuHawNg6ReBa3jSMju3C0ZLgcJ9t0KjbhdheiON41zSt1 sCie+F+mURYaQzU456f5QuXYOZbAQFldUTHktM6wGeULq4BGrVTZsXXxqjhrbsTSdRD9 MZXGb5F0CU/aJr7cGKMvQ4QEklup2Q5CI39RlWz63yBPti1b9e5zL6RV/NX/qKT6qNqA SfQEg/E4ewqmtTfXu8zxM6+0+pQ6quPRmneweftMgvAxe9QtEgZ52NMSpxn9e8JQsJAd U+R/wJAnw4Qs/Hp/gb1/fIroPc9lX0zfEv57JVQ6G+o90gklsr5M4BPXR5tR6S41KUqt wb1Q== MIME-Version: 1.0 X-Received: by 10.152.21.74 with SMTP id t10mr11793594lae.52.1368388969327; Sun, 12 May 2013 13:02:49 -0700 (PDT) Received: by 10.112.150.198 with HTTP; Sun, 12 May 2013 13:02:49 -0700 (PDT) In-Reply-To: References: Date: Sun, 12 May 2013 22:02:49 +0200 Message-ID: Subject: Re: Submitting a hadoop job in large clusters. From: Bertrand Dechoux To: "user@hadoop.apache.org" Content-Type: multipart/alternative; boundary=089e014941646e75f204dc8ae207 X-Virus-Checked: Checked by ClamAV on apache.org --089e014941646e75f204dc8ae207 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Which doesn't imply that you should log yourself to the physical machine where the JobTracker is hosted. It only implies that the hadoop client must be able to reach the JobTracker. It could be from any physical machines hosting the slaves (DataNode, Tasktracker) but it is rarely the case. Often, job are submitted from a machine which doesn't belong to the cluster but can reach every machine of it. Regards Bertrand On Sun, May 12, 2013 at 7:59 PM, shashwat shriparv < dwivedishashwat@gmail.com> wrote: > As nitin said , its responsibility of Jobtracker to distribute the job to > task to the tasktrackers so you need to submitt the job to the job tracke= r > > *Thanks & Regards * > > =E2=88=9E > Shashwat Shriparv > > > > On Sun, May 12, 2013 at 11:26 PM, Nitin Pawar wr= ote: > >> nope >> in MRv1 only jobtracker can accept jobs. You can not trigger job on any >> other process in hadoop other than jobtracker. >> >> >> On Sun, May 12, 2013 at 11:25 PM, Shashidhar Rao < >> raoshashidhar123@gmail.com> wrote: >> >>> @shashwat shriparv >>> >>> Can the a hadoop job be submitted to any datanode in the cluster and no= t >>> to jobTracker. >>> >>> Correct me if it I am wrong , I was told that a hadoop job can be >>> submitted to datanode also apart from JobTracker. Is it correct? >>> >>> Advanced thanks >>> >>> >>> On Sun, May 12, 2013 at 11:02 PM, shashwat shriparv < >>> dwivedishashwat@gmail.com> wrote: >>> >>>> >>>> On Sun, May 12, 2013 at 12:19 AM, Nitin Pawar wrote: >>>> >>>>> >>>>> normally if you want to copy the jar then hadoop admins setu >>>>> >>>> >>>> Submit you job to Job tracker it will distribute throughout the >>>> tasktrackers. >>>> >>>> *Thanks & Regards * >>>> >>>> =E2=88=9E >>>> Shashwat Shriparv >>>> >>>> >>> >> >> >> -- >> Nitin Pawar >> > > --089e014941646e75f204dc8ae207 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Which doesn't imply that you should log your= self to the physical machine where the JobTracker is hosted. It only implie= s that the hadoop client must be able to reach the JobTracker. It could be = from any physical machines hosting the slaves (DataNode, Tasktracker) but i= t is rarely the case. Often, job are submitted from a machine which doesn&#= 39;t belong to the cluster but can reach every machine of it.

Regards

Bertrand

<= br>
On Sun, May 12, 2013 at 7:59 PM, shashwat= shriparv <dwivedishashwat@gmail.com> wrote:
As nitin said , its respons= ibility of Jobtracker to distribute the job to task to the tasktrackers so = you need to submitt the job to the job tracker

<= div dir=3D"ltr"> Thanks & Regards =C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=20 =09 =09 =09 =09

<= font size=3D"6">=E2=88=9E

Shash= wat Shriparv



On Sun, May= 12, 2013 at 11:26 PM, Nitin Pawar <nitinpawar432@gmail.com><= /span> wrote:
nope=C2=A0
in MRv1 only jobtracker can accept jobs. Yo= u can not trigger job on any other process in hadoop other than jobtracker.= =C2=A0


On Sun, May 12, 2013 at 11:25 PM, Shashidhar Rao <raoshashidhar12= 3@gmail.com> wrote:
@shashwat shriparv
=
Can the a hadoop job be submitted to any datanode in the= cluster and not to jobTracker.

Correct me if it I am wrong , I was told that a hado= op job can be submitted to datanode also apart from JobTracker. Is it corre= ct?

Advanced thanks


On Sun, May 1= 2, 2013 at 11:02 PM, shashwat shriparv <dwivedishashwat@gmail.com<= /a>> wrote:

On Sun, May 12, 2013 at 12:19 AM, Nitin= Pawar <nitinpawar432@gmail.com> wrote:

normally if you want to = copy the jar then hadoop admins setu

Submit you job to Job tracker it will distribute throughout the tasktracker= s.

Thanks & Regards =C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=20 =09 =09 =09 =09

<= font size=3D"6">=E2=88=9E

Shash= wat Shriparv





<= /div>--
Nitin Pawar

--089e014941646e75f204dc8ae207--