Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6CCA71083B for ; Thu, 21 Nov 2013 19:36:10 +0000 (UTC) Received: (qmail 21119 invoked by uid 500); 21 Nov 2013 19:36:05 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 20958 invoked by uid 500); 21 Nov 2013 19:36:05 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 20951 invoked by uid 99); 21 Nov 2013 19:36:05 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 21 Nov 2013 19:36:05 +0000 X-ASF-Spam-Status: No, hits=1.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of rickylee0815@gmail.com designates 74.125.82.177 as permitted sender) Received: from [74.125.82.177] (HELO mail-we0-f177.google.com) (74.125.82.177) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 21 Nov 2013 19:36:01 +0000 Received: by mail-we0-f177.google.com with SMTP id p61so206126wes.36 for ; Thu, 21 Nov 2013 11:35:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=1eSSNUI9SEu4bfqto9pZRHwMAgRKRQW4Rial6AYyJcM=; b=d0QMH8O4yw+Ua9zM1it+Jah63OQmxJcOeri3/O8l9p39LiZSdAdc6EpYzsu/B2vlT8 QzPWGOtSV/Uo80Kk+HGpFn6F9r97HFmdxllH0eS52FXn7iccoWxCjkvySmXVvQQl5Xv3 HBGm7CfW6HEqSJkoadkq0COEdNsIf1iRD7XF9lGDpiq8NZuoJkXDZ2uZQOQJMNwvgs3z MizFOElInGl7sq+uMsBd56TzMLXm+odeYzkqt7uDjBYlISLN3uu73/73L6+tCoK2QIrQ c0yQsbtS7Rb9Sfq/hbTauRap85gZc5Z3BVuYlEylqVqKwkH0EKigQGrrClTPwqd3EveA vD5A== MIME-Version: 1.0 X-Received: by 10.180.210.130 with SMTP id mu2mr7229228wic.61.1385062540227; Thu, 21 Nov 2013 11:35:40 -0800 (PST) Received: by 10.216.236.65 with HTTP; Thu, 21 Nov 2013 11:35:40 -0800 (PST) In-Reply-To: References: Date: Thu, 21 Nov 2013 14:35:40 -0500 Message-ID: Subject: Re: In YARN, how does a task tracker knows the address of a job tracker? From: ricky l To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a11c26a84b3ba0304ebb500a7 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c26a84b3ba0304ebb500a7 Content-Type: text/plain; charset=ISO-8859-1 Thank you for the answer, Omkar. I read the links that were helpful. Though the concept of job tracker/task tracker does not exist in the YARN MapReduce, doesn't it use the binary of job/task tracker? I though the application master runs job tracker binary and the containers in the node will run task tracker binary. thx On Thu, Nov 21, 2013 at 2:06 PM, Omkar Joshi wrote: > Hi, > > Starting with YARN there is no notion of job tracker and task tracker. > Here is a quick summary > JobTracker :- > 1) Resource management :- Now done by Resource Manager (it does all > scheduling work) > 2) Application state management :- managing and launching new map /reduce > tasks (done by Application Master .. It is per job not one single entity in > the cluster for all jobs like MRv1). > TaskTracker :- replaced by Node Manager > > I would suggest you read the YARN blog post. > This will answer most of your questions. Plus read this (slide > 12) for how job actually gets executed. > > Thanks, > Omkar Joshi > *Hortonworks Inc.* > > > On Thu, Nov 21, 2013 at 7:52 AM, ricky l wrote: > >> Hi all, >> >> I have a question of how a task tracker identifies job tracker address >> when I submit MR job through YARN. As far as I know, both job tracker and >> task trackers are launched through application master and I am curious >> about the details about job and task tracker launch sequence. >> >> thanks. >> > > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. --001a11c26a84b3ba0304ebb500a7 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Thank you for the answer, Omkar.

I read= the links that were helpful. Though the concept of job tracker/task tracke= r does not exist in the YARN MapReduce, doesn't it use the binary of jo= b/task tracker? I though the application master runs job tracker binary and= the containers in the node will run task tracker binary. thx


On Thu,= Nov 21, 2013 at 2:06 PM, Omkar Joshi <ojoshi@hortonworks.com>= wrote:
Hi,

Star= ting with YARN there is no notion of job tracker and task tracker. Here is = a quick summary
JobTracker :-=A0
1) Resource management :- Now done by Resou= rce Manager (it does all scheduling work)
2) Application state management :- managing and launching new map /red= uce tasks (done by Application Master .. It is per job not one single entit= y in the cluster for all jobs like MRv1).
TaskTracker :- replaced= by Node Manager=A0

I would suggest you read the=A0= YARN blog post. This will answer most of your questions. Plus read this=A0(slide 12) for how job actually gets executed.=

Thanks,
Omkar Joshi


On Thu, Nov 21, 2013 at 7:52 AM, ricky l= <rickylee0815@gmail.com> wrote:
Hi all,

I have a question of how a task= tracker identifies job tracker address when I submit MR job through YARN. = As far as I know, both job tracker and task trackers are launched through a= pplication master and I am curious about the details about job and task tra= cker launch sequence.

thanks.=A0


CONFIDENTIALITY NOTICE
NOTICE: This message is = intended for the use of the individual or entity to which it is addressed a= nd may contain information that is confidential, privileged and exempt from= disclosure under applicable law. If the reader of this message is not the = intended recipient, you are hereby notified that any printing, copying, dis= semination, distribution, disclosure or forwarding of this communication is= strictly prohibited. If you have received this communication in error, ple= ase contact the sender immediately and delete it from your system. Thank Yo= u.

--001a11c26a84b3ba0304ebb500a7--