hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ricky l <rickylee0...@gmail.com>
Subject Re: In YARN, how does a task tracker knows the address of a job tracker?
Date Thu, 21 Nov 2013 19:35:40 GMT
Thank you for the answer, Omkar.

I read the links that were helpful. Though the concept of job tracker/task
tracker does not exist in the YARN MapReduce, doesn't it use the binary of
job/task tracker? I though the application master runs job tracker binary
and the containers in the node will run task tracker binary. thx

On Thu, Nov 21, 2013 at 2:06 PM, Omkar Joshi <ojoshi@hortonworks.com> wrote:

> Hi,
> Starting with YARN there is no notion of job tracker and task tracker.
> Here is a quick summary
> JobTracker :-
> 1) Resource management :- Now done by Resource Manager (it does all
> scheduling work)
> 2) Application state management :- managing and launching new map /reduce
> tasks (done by Application Master .. It is per job not one single entity in
> the cluster for all jobs like MRv1).
> TaskTracker :- replaced by Node Manager
> I would suggest you read the YARN blog post<http://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/>.
> This will answer most of your questions. Plus read this<http://www.slideshare.net/ovjforu/yarn-way-to-share-cluster-beyond>
> 12) for how job actually gets executed.
> Thanks,
> Omkar Joshi
> *Hortonworks Inc.* <http://www.hortonworks.com>
> On Thu, Nov 21, 2013 at 7:52 AM, ricky l <rickylee0815@gmail.com> wrote:
>> Hi all,
>> I have a question of how a task tracker identifies job tracker address
>> when I submit MR job through YARN. As far as I know, both job tracker and
>> task trackers are launched through application master and I am curious
>> about the details about job and task tracker launch sequence.
>> thanks.
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

View raw message