hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Graves <tgra...@yahoo-inc.com>
Subject Re: Question regarding "attempt" concept in YARN source code
Date Fri, 09 Nov 2012 00:25:33 GMT
With YARN you now have applications. This is how the RM (resource manager)
views things.  MapReduce is one type of application.  An RMAppAttempt is
an attempt to run one particular application. During the execution of that
application something might go wrong which causes it to die before the
application finishes.  For instance the node the application master is on
dies.   The RM will then try to run that application again - a second
attempt. There is a configuration that controls how many attempts to run
that application you make before giving up.  You application can chose to
handle re-attempts. For instance the MapReduce application knows its a
second (or third or etc..) attempt, reads the job history file and only
re-runs the tasks that haven't finished.

A task-attempt is at the MapReduce application level.  A Task in this case
is either a map or a reduce task.  An attempt again is just one attempt at
running a single map or reduce.  That single attempt can fail for lots of
reasons - host goes down, etc.  The mapreduce application master will
rerun that task (thus a second attempt) up to whatever is configured.

Tom

On 11/8/12 4:22 PM, "Arinto Murdopo" <arinto@gmail.com> wrote:

>Hi guys,
>
>I have several questions related to YARN source code:
>1. what is "task-attempt" in the this document (
>https://issues.apache.org/jira/secure/attachment/12486023/MapR) ?
>2. what is RMAppAttempt ? the documentation in the code mentions that
>RMApp
>may have multiple RMAppAttempt but I still could not reason what is
>RMAppAttempt and why do we need this.
>
>regards,
>
>Arinto Murdopo
>European Master in Distributed Computing (EMDC)
>Universitat Politècnica de Catalunya · BarcelonaTech, Barcelona, Spain
>KTH Royal Institute of Technology, Stockholm, Sweden
>Phone: +46 725 548 759


Mime
View raw message